As a product manager or business leader, you live and die (well, fail) by metrics. For every project, feature or change shipped, you want to measure the impact. “Did this new button have any effect? It’s the perfect shade of blue!”
The most popular product metric is the Net Promoter Score (NPS). It’s slapped on dashboards, slides and documents wherever possible. It’s become the default way of measuring product success. Do customers like your product? Just look at the NPS numbers like a magic crystal ball.
For many businesses, however — especially in software and technology — NPS is about as useful as measuring the average shoe size of your customers. It’s a number, but what does it actually tell you?
Related: I’ve Interviewed More Than 200 Product-Based Entrepreneurs. Here Are 5 Things the Successful Ones Have in Common
How it works
Here’s how NPS works — you ask customers, “On a scale of 0 to 10, how likely are you to recommend [organization/product/service] to a friend or colleague?” Then you bucket their answers into three groups:
-
9-10: “Promoters” (happy customers)
-
7-8: “Passives” (neutral customers)
-
0-6: “Detractors” (unhappy customers)
Your NPS is the percentage of promoters minus the percentage of detractors. The score ranges from -100 (everyone’s a detractor) to +100 (everyone’s a promoter).
The crazy thing about this formula is the “detractors” bucket. A customer rating a zero is placed in the same category as one rating a 6. This is like saying someone who absolutely hates your product is the same as someone who thinks it’s mediocre.
If you are running an A/B experiment and you move someone from a zero to a six, it’s a massive improvement — but your NPS metric will remain unchanged. That is a big blind spot.
The recommendation paradox
The even larger problem with NPS is inherent in the question itself. Imagine this conversation:
“Hi Alice, how was your weekend?”
“Pretty good! Have you heard about this amazing new tool for building dashboards from your business data? You should really check it out!”
This conversation would normally never happen. People don’t go around recommending business software and APIs the same way they do movies or restaurants. Even if a customer likes your product, they probably will not go around talking about it. Understandably, an NPS survey will get random answers from customers who want to quickly dismiss the weird question and carry on.
Related: More Data Doesn’t Mean Better Insight. Drive Product Growth With A Metric That Guides You to Success.
What you should measure instead
1. Usage: Are people using your product?
If you want to know if customers like your product and find it useful, the first metric to measure is whether they are actually using the product!
A lot of customers using your product gives you insights without any surveys. This means that customers either find your product useful or believe in the advertised value proposition.
Usage is insightful but not a silver bullet. Low usage alone doesn’t mean a bad product. It can also signal issues with marketing, onboarding or the market size.
2. Retention: Do people keep using your product?
Retention is the ultimate vote of confidence. When you have customers who use your product week after week, you don’t need to ask them to know that they value your product.
For any new product or feature, retention is a key North Star metric. If customers tend to stick around once they use your product, you have probably achieved the holy grail of product-market fit.
If retention is high and usage is low at the start, it’s easy to fix that with better marketing and onboarding. The reverse, however, is much harder to solve.
3. Qualitative insights: Pick up the phone and talk to them
Dashboards and numbers are easy and clean, but nothing beats actually talking to your customers. Frequent open-ended conversations with customers will tell you more about how your product is doing than any metric or dashboard.
Remember, a survey is not the same as talking to customers. Talking to customers will be messy. However good a script you write, the conversation will meander with the customer’s train of thought. Ask open-ended questions about how they use your product, what they love and what they hate. Watch out for unexpected use cases and pain points.
Picking up a phone and talking to someone in real life feels like an ancient ritual in this day and age, but you will understand which parts of the product actually work and which parts need change. It will reveal the story behind your dashboard of product metrics.
Related: 3 Methods to Help You Determine What Customers Really Want (and Really Don’t Want)
The path forward
All of this does not mean you should immediately abandon NPS. If your organization has been tracking it for years, it will have value as a windsock — signaling directional change. The key is to understand the limitations and use it accordingly.
The best product teams I have worked with use a balanced approach. They track usage and retention as key metrics, conduct regular customer interviews, and yes, occasionally they will look at NPS, too — but they recognize that NPS is just one imperfect windsock in a complicated world.
If you were trying to pack for a vacation and checking the weather, you would not look at just the temperature. You would also check the rainfall, humidity and UV Index. Maybe you would ask a friend who was there recently. Product metrics are similar — you need to consider a number of factors to get the full picture.
So, the next time you talk about the NPS of your product, pair it with the usage, retention and what customers are saying. That is where the real insights are, and that is how you can build products people truly love — whether they recommend them or not.
Read the full article here
