Don’t Believe Everything You Read
“NPS is the most powerful tool we have ever deployed at GE.” This evaluation comes from Dan Henson, a 20-year GE veteran who is now the company’s chief marketing officer. His comments, based on several years of experience implementing NPS across a wide range of businesses, stand in stark contrast to a recent article in an academic market-research journal. That article, written by two professors, not only concluded that there is no connection between NPS and business growth or profits, but also cautioned that using NPS “is misguided and potentially harmful.” Meanwhile, NPS has become a centerpiece of GE CEO Jeff Immelt’s growth agenda, and his company is rolling it out to every one of its businesses around the world.
What could account for such a difference of viewpoint between the academics on the one hand and the real-world experience of GE and many other firms on the other? Those who take the time to read the original article will find that the authors never included a true NPS in their statistical analysis. Instead, they created another metric altogether, called it NPS, and proceeded to demonstrate that this pseudo-NPS had no relationship to firm growth or profits.
The problem arose because no question in the authors’ database remotely resembled the “ultimate question” that forms the basis for the NPS framework. (Remember that the ultimate question is clearly defined: “On a scale of 0-10, how likely is it that you would recommend (Company X) to a friend or colleague?”) Undeterred, the authors conjured up another statistic, this one based on survey questions that were contained in their database, and labeled this new statistic “NPS.” Then they proved, with impressive statistical sophistication, that the pseudo-NPS had no correlation with financial results.
[I should note that this is a rhetorical device that dates back to the time of Aristotle. If you want to criticize a thoroughbred horse, demonstrate that the horse is similar to a donkey, prove that the donkey cannot win the race, and then conclude that the horse must be a loser.]
What was the authors’ new statistic? They focused on two questions in their survey: Have you discussed your experiences with [Company X] (yes/no)? and , Have you complained to the company about your [product/experience] in the past three months (yes/no)? Since they had no way to quantify the percentage of promoters (defined in my book as the percentage of customers scoring 9 or 10 on the ultimate question), they assumed that a good estimate would be the percentage of customers who replied Yes, they did discuss their experiences (positive, neutral, or negative). To estimate the percentage of detractors, they took the percentage of customers who replied Yes to the question about whether they had complained.
So this statistic is based on a recipe with quite different ingredients than the promoters and detractors used to calculate NPS. And it is indeed novel, since to my knowledge it has never been utilized by any firm in the business world. Interestingly, it seems to focus on the number of people who like to share their experiences with others—not just positive experiences, but neutral or negative ones as well—and calls them promoters. But discussing an experience hardly makes a consumer into a promoter—that is, someone who would enthusiastically recommend the company to a friend. In my experience, discussions about experiences with products or services are just as likely to be negative as positive, and perhaps more so.
The professors’ estimate for detractors is equally flawed. Customers who complain to a company are not necessarily detractors. For example, I am a big fan of Four Seasons Hotels, but I recently complained to them about their phone pricing in London. (It is better than their competitors but still too high, in my opinion.) Similarly, I am a fan of Enterprise Rent –A- Car, but I recently complained that the branch I was renting from didn’t have the car I had reserved. Even though I am a promoter of both Enterprise and Four Seasons and recommend them to friends and colleagues, this new statistic would categorize me as a detractor. (By the way, I rarely complain to companies that have not earned my loyalty—it would be a waste of time.)
My advice? Ignore the pseudo-NPS and focus on the real one. Follow the example of practical business executives, like those at GE, American Express, Intuit, Allianz, Schwab, and the hundreds of other firms who are putting NPS to work productively. Their leaders recognize that the power of NPS is based on common sense: in order to grow, their companies must generate more customer promoters and fewer detractors. NPS offers them a powerful tool and management discipline for managing these numbers in the direction of profitable growth.
PS: I would like to thank Tim Keiningham for bringing the original article to my attention, as well as sharing a paper that he and his colleagues have submitted in rebuttal to Marketing Science Magazine. (The original article, authored by Morgan and Rego, was published in the September-October edition of that magazine.)
