How Pulse is crowdsourcing the wisdom of the CIO

Profile picture for user barb.mosher By Barb Mosher Zinck September 17, 2018
Vendor peer review sites are gaining traction. But they don't solve the problem of executives comparing notes about partners and implementations. Enter Pulse, a Quora-like crowdsourcing community for CIOs.

What’s your opinion on analyst reports? Do they offer you enough insights into the products and services of vendors to help you make a good decision?

Or do they just scratch the surface leaving you to dig deeper on your own (or pay them even more money to learn what they know)? How do you know you aren’t missing out on something that didn’t make the analyst list?

Maybe it’s time to leverage the wisdom of your peers. You know, the people who have been there, done that, are dealing with the same challenges and opportunities you are. This is what Pulse is doing for CIOs, and it seems to be working.

Crowd-sourced insights for CIOs

Mayank Mehta co-founded Pulse because he understood the traditional model of research and analysis was no longer sufficient to help CIOs make sound decisions on technology. His experience interacting with CIOs and learning how they made decisions on vendors, strategy, and hiring showed him that the decision framework they were following was no longer working. CIOs typically follow centralized research models that can’t scale with the number of vendor solutions available.

He also said that a lot of the success organizations had adopting new technologies was not due to selecting the right technology as much as it was selecting the right SI or implementation partner and focusing on what happens after the purchase is complete (last mile changes, meeting business objectives). Mehta saw an opportunity. That opportunity came in the form of a Quora-like crowdsourcing community for CIOs.

Pulse is an agenda-free, vendor-free network, exclusively for IT executives. It started as a chatbot on Facebook Messenger. Mehta said they started there because it offered a frictionless way to connect with CIOs. They would survey and collect data from CIOs and feed the results back (think Glassdoor at this point). It was a way for CIOs to benchmark themselves against their peers, as well as discover new vendors and solutions.

It’s now a complete web and mobile experience with over four thousand IT executives representing 3000 companies, the majority of which are currently in North America. There are weekly roundtables where executives are invited to participate. They answer a series of questions, and the results are given back in a multi-page living report personalized to each member based on company type, tech stack, industry and so on:


Example of roundtable report on Robotic Process Automation - from Pulse Q&A

These roundtables don’t just cover what executives are doing today; they are asked if they will continue using a technology as well, providing insights into where the future is headed.

In parallel with the roundtables, executives are welcome to provide freestyle comments and answer questions. These are shown alongside the report results, dynamically changing as members interact:


Example #2 of roundtable report on Robotic Process Automation - from Pulse Q&A

Members can see the list of who participated in the roundtable, but they don’t see specific responses for each user. Mehta said that this level of transparency cultivates trust among the community.

As for comments, members can choose to show their name or comment anonymously, in which case only their industry is disclosed.

The traditional research model is no longer enough

Traditional approaches to research require vendors to update analysts on their products, strategy, and roadmaps regularly. And most analysts do talk to companies using the products. Typically, vendors supply a list of customers, but analysts may also talk with companies outside the vendor’s customer base. Much of this work only scratches the surface though.

After that, a report is created which provided high-level insights into the vendors and the strength and weaknesses of the product. And the products are ranked according to these strengths and weaknesses.  The onus is on the vendor for many of these types of reports.

But why would CIOs and other executives want to participate in this type of community, especially since participation is free at this level? How do you know they are answering honestly? Mehta said the focus is on collective wisdom/intelligence so one set of responses won’t affect the result set overall. Mehta said the model is gaining traction because much of the technology CIOs are talking about are basics that don’t necessarily set you apart, but you need to get right.

They are also getting a report based on the answers they provide, so everyone will want to be specific and honest. Mehta talked about a recent roundtable they did on robotic process automation that reveals the tools companies use today (examples of the report shown above). If a CIO wants to go deeper, they can request more in-depth information. They can also participate in an ongoing conversation via comments and Q&A, shown to the right of the report.

One of the things that is different about these roundtable reports compared to a traditional report like a Magic Quadrant or a Wave is that they aren’t static reports produced once a quarter or twice a year and then left. The commenting and Q&A continue as long as the report is live. And, Pulse reports are living reports that get updated regularly. Mehta said that the answers expire after one year and members who completed the survey are asked to answer again. At the same time, new members can complete roundtable questions as well ensuring the data is always fresh.

What if you want to go deeper? Now you need to be a premium member. As a premium member, you can request to talk directly to other members in your industry or members working with a specific vendor. You can request a roundtable with a specific set of questions. If you participate in a roundtable and see people using a certain vendor, you can click on that vendor and have a targeted conversation with the members who picked that vendor. You still don’t see what each member says specifically unless they want their name known.

Mehta gave the example of the CIO who wanted to do a benchmark study on her peers’ investment in AI. She received over 50 responses to her study which helped her defend her numbers and budget for AI.

Bringing vendors into the mix - without bias

The next step for Pulse Mehta said, is to bring vendors into the mix to help them better understand their markets while remaining vendor-free. Mehta said they didn’t want to be in the ad business, but they did see an opportunity to help startups and established vendors understand the markets they serve around things like product-market fit, feature set, pricing strategies, messaging and thought leadership.

Vendors can request to have a question sent out to the membership and CIOs can choose to answer. The responses come back to the vendor anonymized with only title, industry, geography and the answer. This, obviously, is not a free service.

I can see the benefit to vendors to ask specific questions to their target audiences to improve how they create and market their products. Not only would this be helpful to vendors starting out or entering a new market/offering a new product, but also to establish vendors who are at risk of losing market share to new entrants into the market who may have better technology or a smarter message.

My take

Right now, Pulse is focused on CIOs, but they are only the beginning. Once they get the vendor component understood and in place, they plan to shift their efforts to helping CMOs and marketing executives with their marketing tech stacks.

This is a group of people that scream for this type of support. And it seems like a group that would be very willing to share their insights. Marketing technologies are amongst the thousands and trying to figure out which ones are the right ones is not simple or straightforward. And you won’t see many of these vendors on a Magic Quadrant or Wave report.

Success today requires organizations to make good decisions as quickly as possible and be able to adapt as necessary. I’m not sure if there will always be a place for traditional research organizations, I suspect they will be around for a while yet. But I do believe there is a place for a community like Pulse – if it remains vendor-free and can attract and retain a well-rounded community who continues to be open to sharing their insights.