The world of tech analysis is changing. Where formerly you had IDC for the numbers on one side and Gartner with its infamous MQ on the other, the abundance of data and near free distribution costs are changing the landscape. Many independent analysts now enjoy a significant presence.
As unofficial members of the technology analyst community, we endeavor to provide rich, qualitative insights into customer buying patterns and the reasons behind customer buying decisions. We're one of many decision points for the informed buyer. We believe that in doing so, we fill an important white space between the opinion makers and mainstream tech media.
G2, formerly known as G2 Crowd, is endeavoring to fill another white space - the data analytics behind what drives decisions and implementation experience. I wanted to understand more about how this works and to that point, I spoke with Mike Fauscette who leads G2's fast-growing analysis group and which already sports a headcount of 50. See the candid podcast recording above.
As background, Fauscette is one of those fairly rare analyst types who worked as a consultant in a variety of software houses before spending close to 10 years with IDC in multiple research-based roles. With G2 for just over three years, it's safe to say that he knows his shit and eschews the kind of marketing BS that so often accompanies vendor-driven analytics.
Prior to our call, I was sent a voluminous methodology document which sets out in excruciating detail how G2 does its work. If you want a copy - feel free to ping me on Twitter and I'll DM the file. As with all docs of this nature, it is an evolving work in progress so do not take it as a definitive statement.
I kicked off our call with my usual analyst poke about statistics and lies. In jocular fashion, Fauscette shot back with:
Tell me the answer you're looking for and I'll give you the question.
Jokes aside that's not what G2 is about. Even so, any set of numbers upon which you rely has to be subject to caveats. In the past, I've seen ridiculous claims to patterns from samples as low as 12. C'mon people. That's a sick joke.
On the other hand, there is much current and earnest debate in tech analyst circles about just how well the enterprise mega-vendors are doing in pushing their cloud ERP agendas.
The vendors know that shareholders want to see growth because that drives the share price upwards. But there are plenty of contra discussions to indicate that interest is not turning into a deployed product. As always, the numbers are all over the map and that in turn makes it hard for decision-makers to both verify claims and find appropriate real-world examples. Here's a good example:
@SAP customers plan to stay on #ECC6 beyond 2025 deadline https://t.co/2uAFUm7iWJ » Well that's what @Riministreet favors. But possible. Comes back to SAP improving the #S4HANA value prop, too early to tell as @dealarchitect states. pic.twitter.com/KDcbOnS2i0
— Holger Mueller (@holgermu) June 22, 2019
For its part, G2 attempts to stand in a middle ground, crowdsourcing reviews, (of which it now counts some 760,000 across all products) which Fauscette insists are carefully vetted, as a service to both buyers and sellers. Fauscette also argues that the methodology it uses facilitates the best quality of answer that in turn allow it to objectively rank vendors.
Don't game me
I understand that and have used G2 on many occasions to get a sense of what a product can do and where a product stands in the market, as evaluated by G2. In my view, it does a good job for the many SMEs who don't have the time or skill to invest in consulting/analysis of their own. But then I pointed out a weakness of these systems in that I have received solicitations from a few vendors asking me to positively review or amend negative reviews I've posted.
Fauscette told me they work hard to ensure such gaming is eradicated and that vendors don't seed with fake eviews.
We ask that any reviewer provide details of their LinkedIn profile so that we can go check to see if they're a real person involved in relevant technology. When we catch vendors out, we do warn them and in some cases have removed their listings entirely.
I have no doubt that G2 works hard to be the best it can be and with Fauscette's reputation on the line (because as analysts that's all you've really got) I'm not surprised that he takes a robust approach to this difficult topic.
The new stats
I was more interested in other stats that G2 collects. These are for vendors and PE/VC consumption only. The one I was provided, which are proprietary, focus on cloud/non-cloud ERP deployments across multiple vendors. They jibe with what I have seen elsewhere but with more detail around time to value, contract periods, and average user counts per deployment. This is valuable in assessing how well a vendor is penetrating a given market, how that's translating into dollar value ARR and how well customers are rep[sonding to offerings in general terms.
VCs and PE firms are highly data-driven. They want to know how firms are progressing. We have a very large pool of crowdsourced data and can spin up what amount to campaigns - think digital marketing. The Gartners of the world are dealing in hundreds of customers, we're dealing in thousands.
This is an interesting play and one which adds credibility to the numbers G2 acquires. Even so, it is early in the game and some of the numbers are open to further question. Fauscette says that to encourage respondents, G2 runs the equivalent of LinkedIn but increasingly Facebook campaigns with Amazon gift card rewards on offer. It's one way to skin the proverbial cat and, according to Fauscette, it works. I think there are other more valuable ways to achieve a possibly better result.
During our conversation, it struck me that G2's analysis serves as good background data against which we can sense test our own customer conversations. So for example, when G2 says that average implementation times for X are Y then we can ask a similar question of customers on the ground with which to gain additional nuance.
So what's missing? In any vendor evaluation, it is good practice to obtain your own customer references. Go to the vendor and you'll get the latest, greatest, and happiest campers. Go to G2 (or Capterra for that matter) and you'll get ranked responses which provide some insights but not quite enough to seal the deal.
It strikes me that where G2 could really add value is through the provision of a referral service. This won't be easy as buyers soon tire of endless reference calls but with such a large pool of referring customers, the opportunity to present both good and bad is there.
Right now, Fauscette is run off his feet and while I might bucket the firm as part of the broad analyst community, he prefers a different positioning:
I'm the only one who's allowed to call himself an analyst.
There are plenty of ways in which to poke holes in the G2 model. Take this assessment of the employee experience engagement market that Larry Dignan picked apart the other day. Definitions that cross tech buckets clearly need more work. What cannot be denied though is that G2 is carving out a valuable position.
Finally, I encourage you to listen to the whole 30-minute conversation. Fauscette responds without hesitation to all my questions in a manner that is candid and authentic. Except one - pricing of the newer products. (Dang!) If there's anything I missed them ping me in the comments.