The survey epidemic is upon us – and something must be done


The era of the spam survey is upon us, and I’m in need of a padded room. Fortunately, Scott Miller of Vision Critical sees change coming – it can’t come soon enough.

bill-murray-groundhog-dayIt happens, my belaugered friends. It happens A LOT. Companies need our input. They need to know exactly how we feel about the button we just pressed, the web page we just clicked on. They need us to fill out… a survey.

I woke up this morning, checked my caller ID. The first caller of the day? A survey. Every customer interaction, no matter how trivial, triggers a survey. If my toothpaste company knew when I finished brushing (someday they probably definitely will – yikes!) they would send me a freaking survey.

Last night, while performing triage on my mother’s scanner, I unsuccessfully clicked on a broken Canon support web page. But one thing wasn’t broken. Care to venture a guess? Yup, the survey pop-up:


A provider that cut off services in my country a year ago still wants my useless input:

survey-hell two

Southwest sent me a “how was your flight” survey while I was still in the air. A clothier sent me “rate our services” spam before the shirt arrived. Oh, and if your Internet goes down and you call Comcast, you have to listen to their survey pitch – and accept or decline – before you can navigate their phone tree and actually fix your broken whatever. EVERY SINGLE TIME you call.

Something must be done.

But no one wants to read a survey rant – you could easily rattle off your own. So I did an interview with someone who IS trying do something about it: Scott Miller, CEO of Vision Critical, and author of the ebook Why Ad-Hoc Surveys Don’t Work (free with sign up). You can also check out Vision Critical’s infographic, The Tyranny of Spam Surveys.

Let’s be honest here – I didn’t need an interview; I needed a therapy session. Miller handled my survey angst like a skilled social worker. And he shined a small glimmer of hope: it turns out that survey mania ISN’T effective. Miller thinks we’ll see this idiocy replaced by better tactics. As we tick through my burning questions, what better place to start then:

Survey frenzy: why is this happening?

Miller’s short answer: because surveys are still a $20-30 billion industry. Oh, and because of “technology-enabled laziness.” Now we’re cooking! Miller explained that surveys represent a clunky, old school approach to determining sentiment – an approach that is now being disrupted (thank goodness!):

Survey research has been dramatically disintermediated. That disintermediation started about a decade ago when customers – for the first time ever – started being able to talk about brands and companies and products without the company engaging in the conversation. [Prior to the social web], the only way that companies could gather intelligence about customers was to pepper you with surveys.

As the customer gains a public voice and sentiment is easier to capture, the survey is forced to change – or appear tone deaf and intrusive. But Miller acknowledges that the survey industry is slow to accept their shrill shortcomings:

To be very honest, we think most of the conventional survey research world still doesn’t understand what the [empowered customers] means. It doesn’t mean “gather more intelligence from that person because that person is now more powerful.” It means embrace the idea that you have to engage that person on their terms. Our view of the world is that companies should no longer be out trying to gather information about the customer experience or the customer product – they should be trying to make that process part of the customer experience.

Can you make surveys part of a compelling customer experience? We’ll get back to that, but first: if surveys are intrusive and out-of-touch, then, by definition, they are spam.

Ergo, surveys are spam

Miller’s definition of survey spam: “The metric that we use is simple. If customers that you’re reaching out to don’t respond, don’t engage in high volumes, then you’re spamming them.”

In their spam surveys infographic, Vision Critical defines four elements of the “spam survey”:

  • They are one-way conversations, strictly about the results the survey maker wants to acquire, not the interests of the customer
  • They appear out of nowhere, uninvited, and most often, unwelcome
  • They are impersonal and generic, not connected to the data or experience of the individual customer
  • They are often overly long, not respecting the time and effort required of the customer

I’d say 99 percent of the surveys I receive easily qualify as spam – how about you? (I’ll grant exception to industry research surveys like Esteban Kolsky’s, where input is solicited via a blog post and the results are shared with participants afterward).

So how do we change this sorry state off affairs? Miller believes change is afoot, as survey participation rates continue to drop. Check out these numbers from their infographic:

  • 72 percent say surveys interfere with the experience of a web site
  • 80 percent say they have abandoned a survey halfway through
  • 52 percent say they won’t spend more than three minutes filling out a survey

(From OpinionLab – I was going to say “of people surveyed” but that is too dangerously ironic).

We also know that response rates on surveys dropped from 36 percent in 1997 to a meager 9 percent in 2004, as per Pew Research Center. I asked Miller why Pew hasn’t updated that number since 2004. He’s not sure, but thinks the result would be embarrassing for the survey industry.

The problem is that with the rise of technology-triggered surveys, companies can rationalize crummy response rates versus, say, a failed/expensive direct mail survey. That’s the dark side of this outlook, which would imply: more spam surveys coming your way. Miller has a brighter view, but he admits that this tech has become intrusive. When I ranted about getting pinged every time I perform the simplest of web site actions, Miller responded:

When they hit you every time you have some an event or trigger, I would consider that to be technology-enabled laziness. By the way, we do not advocate that.

Surveys are impersonal; companies rarely act on the results – cynicism ensues

I hit Miller with two more points. Surveys are impersonal. They’re not enjoyable. It’s very rare you can stamp what you feel into them. It’s usually an endless list of “rate this from one to five.” It’s almost impossible to get across what you really think. Miller responded:

We agree with you entirely. Our belief is that the closer our engagements are to actual human behavior, human conversations, the more accurate they’re going to be. You know what the number one thing that humans do when they’re having a conversation? They recall things from the past in order to trigger the next conversation, the next question, the next comment.

What non-humans do is they simply look at it at a point in time data and trigger an activity. I don’t care who you are. I don’t care what you told me in the past. You did this, now rate it if you will.

We believe that is dying. We believe that it’s inappropriate. We believe just like you said, that it is too impersonal – and most importantly – it’s going to be inaccurate because it doesn’t build upon intelligence you already have the potential to leverage.

I also take issue with the lack of transparency and follow-through. Surveys accomplish nothing and result in no action. If I knew my feedback would improve services, I’d take the damn survey. Here Miller agrees, but only to a point:

In defense of the industry, they actually are using the information, but what they’re doing is they’re taking your responses, they are aggregating them with those of 50,000 other people, and they’re measuring and making a macro decision based on that. That’s the way that the research world was built, on samples. Quite frankly, in some cases, they still call it samples. You and I were “samples.”

The wrap – can there be a non-spam survey?

So does Miller believe there can be a non-spam survey? Yes, of course – Vision Critical is informed by that belief. A non-spammy survey would be the opposite of what we dismantled here.

Miller outlined characteristics of a quality survey:

  • Demonstrate to your target participant that you know something about them and that you’re reaching out for a reason that’s relevant to them.
  • Deliver the survey in a multi-device manner, so participants can fill it out on any device, and think mobile-first.
  • Do not ask anything that is “nice to know.” Ask only the “must haves.”

Miller’s ultimate criteria is response rates. He says that Vision Critical’s customers respond in three to four times the rates of a typical survey.

But it’s not just what you ask; it’s how. Survey “asks” should be a natural byproduct of an existing customer relationship/community. That gets back to the point Miller made about embedding the survey process in the customer experience. Or, as I put it:

Don’t even think about surveys you are confident that you have an engaged community, and then I would consider a survey as one possible approach. I’ll give Miller the last word, along with a well-earned Vision Critical plug for helping me process my survey angst:

All the things you’ve complained about helped create the momentum that Vision Critical has enjoyed. Let’s face it, we’re in the right place at the right time. We didn’t create the empowered customer. We’re just trying to help companies engage with that person, and gather intelligence and deliver intelligence back as a result. I do think the marketplace will police the type of things that are driving you crazy. It’ll take a while, but most importantly, you have to decide to not participate in the things that don’t matter to you – and that will help purge the bad behaviors out of the marketplace.

No problem Mr. Miller – no more spam surveys for me. And with all the powers invested in me by my inbox, I pray to the merciless gods of spam that you are right.

p.s. the exact second I hit “publish,” my cell phone rang. “Can I speak to Jonathan Reed?”

Updated April 2, 2016 am with a link to the magic toothbrush and a classic picture that reflects the mood of the piece.

Image credit - Desperate employee © olly -

Disclosure - Diginomica has no financial ties to Vision Critical. I reached out to Vision Critical's PR to set an interview, based on their content on spam surveys.

    Comments are closed.

    1. says:

      Have you forgotten Benioff’s Toothbrush already my friend? 😉Reference

      1. Jon Reed says:

        You know – I had repressed that memory into the deeper corners of my noggin, but you’re right. I had been so caught up with toothpaste I overlooked the magic brush. I’ve made an editorial change to that effect. I’m thinking this would also work well with a picture of Bill Murray from Groundhog Day…..

    2. Another twist on survey spam is complete these ad-tech survey questions to continue reading this article: CSmonitor et al

      Perhaps they can make the UX so horribly interruptive subscriptions come back into vogue (or is that Vogue).

      1. Jon Reed says:

        Clive – yes. That’s the hope – enough of us get fed up with that UX-invasive junk that companies give up. I should have mentioned that form of invasive survey spam in the piece. That’s also the “technology enabled laziness” referred to. Alas, enough people seem to click or fill out to make it worthwhile. I hope Scott is right but I fear this crap will dog us for a long while….

        – Jon

        1. Jon — Julia Ferraioli’s talk on building smart applications with machine learning at GCPNext features David Zuckerman of Wix. Who tells if Wix asks survey questions something really interesting happens, conversion rates drop (interrupting flow with surveys kills engagement). Goes on to say realtime ML is key. Clive.

    3. been doing “surveys” and related work for 30+ years — since my dad started a research firm in Argentina that was staffed only by psychologists, anthropologists, and sociologists. i pushed to start the “revolution” of biz mixing surveys with data and results in the early 00s while at Gartner and ended up doing 6-8 hours of inquiries a day on the topic (which, btw, could’ve been recorded and replayed since everyone had the same question).

      I am not bragging, i am stating my credentials.

      I am here to tell you – surveys are the worse thing you can do to find out what your customers want / think / need and even lower value to help you figure out what you have to to in your organization.

      its time to ditch surveys across the board and focus on using data (sorry BIG data) from everything we capture to understand customers better.

      main reason? people lie in surveys – but that’s a subject for a different forum (and no, its not intentional — this is why we used psychologists and sociologists in the past, was necessary).

      ah, surveys — if we could use that $18+ billion in something useful…

      1. Claes Fornell says:

        What good is big data alone without some kind of perspective telling you what the big data means? GA, Omniture, Heat Mapping, Video Replays, etc etc etc are all incomplete without EFFECTIVE customer experience measurement. Getting customer feedback is a necessary evil… getting actionable customer experience data the right way is where most people fail. It requires the survey work to be backed by a proven methodology. Running your business completely off BIG data will result in BIG wastes of time/money chasing incomplete, backward looking, behavioral metrics.

        1. Many of the top researchers (often minted at Stanford where design is integrated into degrees) who were previously with [Xerox] PARC have headed to Google and others who are data driven and now combine ethnography with data driven.

          Examples adding new features to Google is done incrementally and validated 3 ways by measuring progress in data (unlike adding features to enterprise software!)

    4. Claes Fornell says:

      It’s simple… There is a reason all the top brands in every industry (including government) have customer experience measurement solutions that include effective tools (i.e. surveying done correctly through random web intercepts) to interact with their customers directly. Again… top business want to make decisions not based on behavioral metrics alone (google analytics), but also on their customers feedback MEASURED THE RIGHT WAY. Making business decisions based on nothing but big data or junk metrics like CES/NPS is a recipe for failure.

      There is a reason Jeff Bezos introduces the new Fire phone and references Amazon’s #1 position on industry leading, methodology driven CSAT rankings. He understands the customer experience is more important than anything and that it needs to be measured effectively to be made actionable. Big data/GA doesn’t tell you a thing about what those metrics mean from the customers perspective…

      – Time on page is high… that must be a good thing because people are engaging more with the content —– OR maybe they are struggling to find what they are looking for or accomplish their tasks
      – Site traffic went way up… that must be a good thing because more traffic is more traffic right —– OR maybe people are struggling to accomplish tasks and coming back multiple times or getting help from others
      – Traffic from social media referring URL’s is low… that must be bad because we invest a lot of time/money in our social media marketing/outreach/spend. Maybe we should spend less here. —– OR we simply have a large audience that doesn’t click direct through social media links to get to our site. They come to the site directly when timing is better and GA has no way of showing us this. Our social media spend turns out to be an incredibly effective acquisition source for site traffic, but we would have never known unless we asked site visitors what influenced their visit.

      Businesses run on assumptions and best guesses when they rely solely on behavioral tools and don’t get direct survey feedback from their users. Again, the key is surveying the right way using a proven methodology.

      The leaders in every industry aren’t wrong. The last battlefield is the customer experience and surveys are the only way to get the visitors point of view to complement other usability/behavioral tools by giving them actionable/useful perspective.