What Real Conversations Taught Us About Recommendation

If you work in marketing, you’ve lived with NPS for years. You report it every quarter. Maybe your bonus is tied to it. And you’ve probably also seen the endless stream of articles telling you it’s a flawed metric.

Here’s the strange part: researchers have been picking NPS apart for close to two decades, and yet NPS budgets keep growing. So which is it – is NPS broken, or is it the best thing marketing ever got?

We think the honest answer is: both. NPS has real, well-documented problems. But underneath the single number, there’s something genuinely important – the psychology of why people recommend, and why they don’t. At Mercury Research, we ran a qualitative study to dig into exactly that and presented the findings at Best of ESOMAR Bucharest. This article walks through what NPS gets wrong, and – more usefully – what actually makes recommendation work in your industry.

A quick history lesson

In 2002, Andy Taylor, CEO of Enterprise Rent-A-Car, published an HBR article called “Driving Customer Satisfaction,” describing how he grew the company by tying management bonuses to the percentage of “completely satisfied” customers.

That example inspired Frederick Reichheld of Bain & Company to go looking for the one metric most closely linked to growth. In 2003, he published the now-famous HBR article “The One Number You Need to Grow” – and Net Promoter Score was born.

The idea was simple and seductive: ask customers “how likely is it that you would recommend [brand] to a friend or colleague?” on a 0–10 scale. Subtract the percentage of detractors (0–6) from the percentage of promoters (9–10), and you get your score. Companies everywhere adopted it. Many still pay bonuses on it.

And then, as these things go, some other things started to happen – researchers started finding holes in it.

So, what’s wrong with NPS?

Fifteen years on, critics like Jerry Thomas of Decision Analyst – and our own research at Mercury – have surfaced four recurring problems.

1. Ambiguity

The scale isn’t as clean as it looks. Are 7 and 8 really “neutral”? In our own surveys, we regularly found people giving a 9 specifically as a way of signaling why they would not recommend a brand. And plenty of respondents will tell you plainly, “yes, I’d recommend it” – and then simply cannot land on a specific number when you ask them to.

2. Loss of information

Look at what the scoring actually does to your data:

• 10 gets treated exactly like 9
• 7 and 8 are thrown out entirely
• 0 through 6 are all treated as identical

Squint at that, and NPS is really just a yes/no question wearing an 11-point scale as a costume. Which raises the obvious question: why bother with 0–10 in the first place? For a methodology whose creator repeatedly preaches “keep it simple,” that’s a strange bit of hypocrisy.

See this very eloquent example published on Medium by Colin Fraser, Data Scientist at Facebook, with three very different distributions of responses, but the same NPS!

3. The names are not adequate

Is someone who answers 9 actually going to recommend you? We don’t think so – not reliably. We’ve even seen genuinely delighted customers, people who gave a 10, decline to recommend a product because they were worried they and the friend they’d recommend it to might not share the same taste.

Flip it around: is an 8 really consequence-free? Maybe that person would happily recommend part of your service. Maybe a 7 would actively tell a friend to avoid you – which would make them a detractor, not a passive. And is everyone scoring 0–6 actually steering people away from your brand? We’ve met plenty of people who simply don’t want the responsibility of recommending anything to anyone, no matter how happy they are – and they are not detractors.

If the labels “promoter” and “detractor” don’t map cleanly onto behavior, then neither do the actions companies take based on them.

4. The random noise is bigger than you think

This is the one that should worry you most as a marketer, because it directly affects how you interpret your quarterly NPS trend.

We ran our own simulation: 100 samples of 500 cases each. The sampling error on NPS came out to roughly 3 times the sampling error of a plain average, and 1.5 times the error of the straightforward top 2 box (percentage giving 9 or 10) metric.

Original chart based on Mercury Research’s simulation findings

We’re not the only ones who noticed. Colin Fraser, published a similar experiment on Medium showing, more explicitly than our experiment, just how much of NPS’s apparent movement is really just statistical noise – not a real shift in customer sentiment.

Most of the clients we work with come to us wanting an explanation for why their NPS went up, or (more often) down. The uncomfortable truth, more often than people want to hear: it didn’t really move at all. It’s noise, read through a gauge too coarse to tell the difference.

And if you’ve ever run an NPS survey with an open comment box, you already know the last frustration: “Why did you give this answer?” We see that question constantly in NPS surveys – and it exists precisely because the number on its own explains so little.

Still – recommendation clearly matters. So what’s actually going on?

Here’s where it gets interesting. NPS as a metric has real problems. But recommendation as a behavior is absolutely worth understanding – it just needs a different lens. So we ran focus groups to find out: what actually makes someone ask for a recommendation, trust one, give one, or ignore one entirely?

The essentials of asking for a recommendation

Three factors decide whether someone goes looking for advice before they buy:

• Information. When you have full, relevant information, you can decide confidently even under high risk and high stakes. When you don’t, you’re stuck – anxious, and searching.
• Risk. If any product on the market will basically do the job, there’s no real possibility of failure, and no reason to ask around. But once you know things can go wrong, the fear kicks in.
• Stake. Standing to lose a few pennies? You don’t care. Standing to lose something you can’t afford to redo? The anticipation of failure eats at you. No fear if you don’t care; real fear if you can’t afford to get it wrong.

Here is the theory above applied

Here’s what that looks like in respondents’ own words.

No personal experience to fall back on. A woman from Gen X, on choosing a hotel:

For women in our sample, this was often tied to a fear of ruining a vacation – frequently the one part of life they felt they were fully entitled to enjoy.

For men, a similar fear showed up differently – less about the experience itself, more about being mocked by friends for a bad choice. A Gen Y man, on choosing car speakers:

Specs that don’t connect to real usage. A Gen X woman, on choosing an oven:

Here, the fear wasn’t social embarrassment among friends – it was having your baking criticized by friends, coworkers, your mother-in-law, or worst of all, your own mother.

A previous failure raising the stakes. A Gen Y woman, on choosing a family doctor:

Big money on the line. A Gen X woman, on buying a durable good:

Four very different products – hotels, speakers, ovens, a doctor – and every single quote maps back to the same underlying drivers: missing information, missing relevant information, high stakes, or high risk.

The “plenty of information” trap

Take laptops – the most frequent example in our research of both offering and following recommendations. You’d assume, with endless spec sheets, benchmark sites, and reviews, that laptop buyers are drowning in information. And yet this is one of the categories most driven by recommendation. Because the information from the ads and from the product specifications is not relevant. Nobody tells you how frequently it crashes or how much the game lag is.

The IT industry lives on recommendations and wastes on advertising.

Which categories does recommendation actually move the needle for?

Not every category behaves the same way. In our research, recommendation clearly mattered for:

• Goods available online where buyer reviews are visible – epilators, stoves, TVs, phones, fridges, laptops, headphones
• Accommodation and vacations (booking sites, TripAdvisor)
• Restaurants
• Shoes and clothing stores
• Mobile operators – particularly for finding out about current special offers and available handsets
• Healthcare – spas, family doctors, clinic subscriptions
• Niche hobbies – stamps, coins, paintings, gyms, pools, show and theatre tickets
• Movies
• Car tires and car service
• Over-the-counter drugs

And it mattered much less, or not at all, for:

• Groceries – not worth the effort of searching for recommendations relative to the money spent
• Clothing, when personal style is strongly involved (more on this below)
• Anything where the person already has their own direct experience

The psychology of listening to a recommendation

Where recommendations come from. Friends and family top the list – but only when tastes actually align:

One respondent put the idea of taste-compatibility especially well, describing how a friend who loves visiting monasteries is a perfectly good source of advice for a monastery trip – but useless for advice on a mountain camping trip, because the two of them simply want different things out of a holiday.

Husbands stepping in for wives on technical purchases came up repeatedly:

So did coworkers:

Beyond personal networks: buyer reviews on ecommerce and travel sites, YouTube unboxing videos (seeing the actual person – and the actual product – builds more trust than text alone), established branding, Facebook friends sharing product links, call centers, and in-store consultants (with heavy reservations, discussed below).

What makes a recommendation trustworthy. Three things came up again and again: the recommender actually owns the product; it’s someone you know well who has your interest at heart; or it’s a specialist who has your interest at heart – which respondents rated as the single highest-impact source, and the clear opposite of a specialist who wants your money.

Where recommendation runs into trouble. The biggest one: the risk of hurting your own image by recommending something that clashes with someone else’s taste.

Strong branding can also simply override the need for a recommendation entirely:

And so can an abundance of visible, aggregated information:

Reciprocity is real

People give recommendations partly because they hope to get them back:

Looking for Recommendations – the essentials

The psychology of giving a recommendation

Recommending isn’t a neutral act, either. People do it because it feels good to have influence, and because helping someone creates a small sense that they now owe you one. Some people recommend even without being asked, when the product is simply that good:

But recommending also carries risk: you might disappoint the person you recommended to and become disliked for it. Or you might raise suspicion that you have some hidden interest in the recommendation and become disliked for that instead.

Here’s the asymmetry that matters most for your brand: the barriers fall away almost entirely when the recommendation is negative. That’s exactly why negative word of mouth travels faster and easier than positive word of mouth.

This isn’t a new observation, but it’s worth restating: for years, the industry rule of thumb was that a satisfied customer tells three friends, but a dissatisfied one tells twenty. Pete Blackshaw’s more recent book title updates the math for the social media era: “Satisfied Customers Tell Three Friends, Angry Customers Tell 3,000.”

Offering Recommendations – the essentials

People want to recommend if they had a good experience with the product and they are asked about it.

People tend to withhold recommendation if they fear there is a difference in taste and their recommendation may not prove as pleasant, if they think product variability is high and the friend they intend to help may get a worse product and if there is suspicion of a conflict of interest.

Does advertising affect recommendation?

In our groups, Gen X women were adamant: no way, advertising and recommendation are separate things entirely. Gen Y men were more nuanced – if an ad includes a specific price, and it’s for something they’d have recommended anyway, they might well pass along that special-offer detail as a form of recommendation. One respondent’s take on Coca-Cola sums up the more cynical end of this view:

Will recommendation actually move the needle for your business?

This is the practical question every marketer reading this cares about. Here’s a simple scoring framework straight from our research, using seven criteria: lack of information/branding, lack of relevant information, high stakes, high risk, importance of taste, personal experience, and fear of failure. Score each roughly + / 0 / – for your category, and add them up.

Example: laptop (Sum=+2-0=+2) is likely to be impacted by recommendations. Recommendation would be a good KPI

This lines up exactly with what we saw across categories: laptops work well on recommendation, accommodation works very well on recommendation, and banks simply do not work well on recommendation at all.

Example: accommodations (Sum=+3-0=+3) is likely to be impacted by recommendations. Recommendation would be a good KPI

Now let’s take one example that is different – banks

Example: bank (Sum=-2+1=-1) is not likely to be impacted by recommendations. Recommendation would not be a good KPI

So what should you actually do?

If your category scores well on this framework, here are the levers that came out of our research:

• Have a genuinely great product – better than any competitor. People will recommend a great product even if nobody asks them to. There is no lever more powerful than this one.
• Make sure specialists and influencers in your category are happy with your product – their recommendations carry the most weight of all.
• Make sure user reviews are visible online, wherever your buyers actually go looking.
• Make sure real customers are demonstrating your product on YouTube – seeing an actual person with the actual product builds trust faster than text alone.
• You can buy recommendations with money – discounts, vouchers – but only for well-known brands that can reliably deliver the same benefit to the next customer too. And even then, make sure it costs you less than the advertising it’s replacing.

Where does this leave your NPS number?

NPS isn’t going away, and it’s not entirely useless either – but treat the single score with real caution, especially quarter-to-quarter movement, which is very often just noise. The more useful question for a marketer isn’t “what’s our NPS?” It’s: does our category even run on recommendation in the first place – and if it does, are we actually giving people a reason, and the right information, to pass us along?

Score your own industry against the framework above. You might be surprised which side of the line you land on. And if recommendation is an important KPI for you, use average rather than other contraptions – it holds more information and it offers you all available statistical tools to analyze it!