OBSERVATIONS FROM THE FINTECH SNARK TANK
Companies bragging about their Net Promoter Score (NPS) isn’t new but seems to be on the rise—and it’s not a good thing.
A Wall Street Journal article titled The Dubious Management Fad Sweeping Corporate America found that “net promoter” or “NPS” was cited more than 150 times in earnings conference calls by 50 S&P 500 companies in 2018. Of all the mentions the Journal tracked, no company ever said its NPS declined.
Surprise, surprise.
In a Harvard Business Review article titled Net Promoter Score 3.0, the authors warned business executives that “self-reported scores and misinterpretations of the NPS framework have sown confusion and diminished its credibility.”
Don’t laugh too hard at that joke of a statement. The self-reporting of scores helped create the metric’s popularity, and it’s weaknesses in the score’s methodology that have rightfully diminished the metric’s credibility.
The Net Promoter Question Doesn’t Even Make Sense
I won’t belabor the fact that the NPS is the stupidest business metric on the planet. The question doesn’t even make sense. How “likely” are you to refer? You either plan to refer or you don’t—there’s no scale.
And then you either do or don’t refer, so “likelihood” is useless.
Furthermore, just because someone gives a low score doesn’t make them a detractor.
I’m a loyal JetBlue customer. The airline sends me a NPS survey after every flight and I give them a zero every time. I’m not dissatisfied—and I have no intention to bad mouth them—I just don’t talk to my family and friends (or business associates, for that matter) about airlines.
Should Laggards Brag About Net Promoter Scores?
The inconsistency across firms with which the metric is captured (i.e, timing, frequency, representative of the customer base) renders the score incomparable across companies.
That doesn’t stop firms from bragging about their internally-generated, unaudited NPS scores. Like one bank tech firm who has been struggling in the US for the past few years who recently touted in a press release that it achieved:
“…a market-leading NPS of 54 following a survey of over 900 customer contacts. The score is reflective of [the firm’s] status as the most trusted banking platform provider and leading position in the industry driving customer success and business growth.”
Market-leading defined by who?
This is a company, by the way, that has seen a rash of customer defections in the US over the past few years. It’s hard to imagine that those former customers were included in the NPS survey.
As a colleague of mine pointed out, “Look at who they claimed to have surveyed. No digital, lending, or credit union clients—only core, ops, and payments.”
NPS Statistical Nonsense
Some firms go through statistical gyrations when heralding their NPS score. Talroo, a “leading provider” (every firm is a “leading”provider, no?) of HR software claimed in a press release that its NPS was “67% higher than industry benchmarks.”
Apparently, its internal survey of customers produced a net promoter score of 77, while an independent source calculated an industry-wide average NPS of 46.
This begs the question, of course: What did the independent source find Talroo’s NPS to be and why didn’t Talroo report that score?
Vymo, which claims to be the “world’s leading sales engagement platform for financial institutions” (not just a leader, but the leading), reported a NPS of 54 in a press release which said that made the company “one of the highest-rated enterprise SaaS platforms in the world.”
Not surprisingly, there was no proof given of that claim as the press release made no mention of any other SaaS platforms and what those firms’ net promoter scores are.
The invalid comparisons of net promoter scores was highlighted (inadvertently) by the Talroo press release which said its NPS survey response rate of 54% was more than double the industry benchmark of 15% of 25%.
First off, I’m not sure where Talroo got “15% to 25% from—Customer Gauge says the response rate for most companies is about 5%.
Second—and more importantly—whether the response is 5% or 25%, the big question is: Was the respondent sample representative of the overall population of customers?
Not one of the NPS braggarts’ press releases addresses this question.
For B2B software companies like Talroo and Vymo another unanswered question is: Who, exactly, completed their NPS surveys? Was it the end users, the CEOs or CIOs of the client companies, or some other set of executives? Whose opinion of the “likelihood to refer” is really the most important one here?
Who Are the Braggarts Trying to Influence?
If there are equity analysts out there—people who influence stock prices—that are swayed by companies’ NPS pronouncements, they should be fired.
I can’t imagine that these professionals, who look at financial results governed by generally accepted account principles, would rely on the statistical nonsense being fed them by companies’ marketing and press people.
I also can’t imagine that these NPS press releases influence decision makers in any kind of company for that matter. References—real live actual references—matter, not “intention to refer.”
It’s all BS about a BS metric.
