Bernard Golden: Some Industry Analysts Offer Anecdote Dressed Up as Science

Written by Sean Murphy. Posted in skmurphy

I got Bernard Golden‘s April Navica Open Source Newsletter today, provocatively titled “Everything I Said Last Month was Wrong, and I’m a Lousy Prophet, Too” which contains some good advice for startups as they work with industry analysts (emphasis added):

I had my eyes opened about analyst firms this month. Even though I had a rather jaundiced view of them in the past, I had a couple of interactions that really illustrated the shallowness of their approach and the undoubted unreliability of their research, if it actually can be spoken of with that term.

First up, I got an email from one firm wanting to research an open source topic. The email was something along the lines of “I’m interested in learning more about XXX. I’d like to interview you for 45 minutes to an hour and would like to talk to two of your customers. Please let me know when would be convenient for you to speak.” I found the assumption that I would offer a bunch of my time for free and would impose on clients all in the service of the analyst firm’s business incredibly presumptuous and so naturally, though politely, declined.

The second interaction was an actual conversation. An open source company asked me to speak with another analyst firm to discuss the potential for their application from my perspective – in other words, to discuss my opinion about how their application might be used in the marketplace. I agreed and arranged a time to speak to the analyst. She called me and immediately asked me to describe my firm’s business and then wanted to know how many engagements it had done in the previous year. When I demurred to discuss my business – remember, the purpose of the call was to talk about the potential for an application released by another company – the analyst got very huffy and rather frostily ended the conversation shortly thereafter. And, by the way, the insights offered by the analyst during our conversation were, to my mind, extremely puerile and unsophisticated.

These interactions illustrated the reality of the analyst business. For all its supposed accuracy and sophisticated research methodologies, it is founded on opinions formed on the basis of discussions with people who are willing to have their time wasted in intrusive conversations with interviewers who really have very little deep understanding of the topic. In other words, it’s anecdote dressed up as science.

It’s fascinating, however, that commercial open source companies, despite their brave insistence that they are upending the bloated and irresponsible proprietary software industry, still kowtow to the traditional practices of the industry, including focusing on currying favor with analyst firms in the hope of gaining a coveted endorsement or at least recognition. This despite what last month’s newsletter discussed: most open source software enters enterprises in a very different fashion than the historic patterns of proprietary products. That’s why the traditional handmaidens of proprietary software, the big system integrators, have heretofore played no role in the spread of open source.

There are a number of good analysts out there, Gary Smith in the EDA space comes readily to mind, as does Bryan Lewis for the semiconductor market. But especially in emerging markets like collaboration, SaaS, open source, and social media, we may have yet to separate the wheat from the chaff.

Trackback from your site.

Leave a comment

Quick Links

Bootstrappers Breakfast Link Startup Stages Clients In the News Upcoming Events Office Hours Button Newsletter SignUp