Behind every great vendor analysis lies a great customer reference program

By Jay Andersen

Clients regularly ask us to help manage industry analyst vendor evaluation reports, specifically Gartner Magic Quadrants, Forrester Waves and IDC MarketScapes. In a perfect world, this process kicks off immediately after the previous year’s report is published and involves laying a foundation with inquiries, briefings and consulting days. So there are few surprises when you receive an invitation to participate in an upcoming report and begin the laborious four-to-six month process of responding to the analyst firm, one that includes presentations, demos and a written survey response.

With so much time and effort put into these deliverables, some vendors don’t appreciate the importance of solid customer references. However, some landscape reports have become increasingly weighted more towards customer feedback than vendor inputs. Even more crucially, customer feedback is crucial in validating vendor claims about performance and results.

In most cases, AR teams and product managers are simply overstretched and are unable to put in place supporting processes, or spend enough time enforcing best practices. Historically, the following are common slip-ups by vendors that have been heard directly from analysts:

  • Selecting references that are not relevant to the product being evaluated, or that do not fit with the vendor’s long-term vision
  • Pushing forward references that have not been properly vetted or prepped for an analyst interview or survey
  • Using references who are unhappy and view the interview as an opportunity to ‘get back’ at the vendor
  • Tapping references who fail to respond to survey requests

To avoid these pitfalls, negate them with the following steps:

  • Establish close ties with your customer reference teams to identify happy references and become fast friends with your customer reference program. Customer references are the ‘coin of the realm’ in AR and these teams should be your mint. Customer reference teams can also provide a sanity check to determine if customers really are happy.  
  • Include the right mix of customers, ideally a healthy blend of bleeding edge ‘transformative’ customers and more conservative, fast-follower customers. The precise mix depends on how much you are defining a space or defending market share. It is also imperative to provide contacts that match the requested profiles; it may seem like a no-brainer but we keep hearing from analysts with this exact complaint.
  • If available, use different clients than the ones provided in previous evaluations. This not only demonstrates to analysts that you have a deep bench of customers, it prevents burnout from your customers who have been bombarded with requests. 
  • Once identified, get on the phone with customers and provide them with the knowledge they’ll need for the call and survey. Prepare them by running mock interviews with expected questions.
  • Know the analyst leading the vendor rating well in advance of the evaluation. Each analyst has their own perspective on the market space and how it’s evolving, and this will affect your optimal responses. Schedule regular inquiries to check their perspectives in the run-up to the next report. This can be an important step in deciding whether to engage in the report or decline participation.

I’d love to hear from people regarding ways that customer references have saved the day or sunk the ship. Drop me an email at jay.andersen (at) blancandotus.com.