Part of my role at the conference, as well as blogging, included introducing some of my Satmetrix colleagues who were presenting on the Programme Design track. This kicked off with Henry Jones, our UK Managing Director who was focusing on who, how and what you should measure in your NPS program.
Now, I'm not going to spend a lot of words in describing the presentation itself, but instead focus on some key points that came out in questions from the audience.
First of all was a question around response rates. Getting your customers to respond to a survey request is pretty key to gathering feedback! Short surveys, however, are not in themselves an automatic guarantee of a fantastic response. The question from the audience focused on the difference between Web and telephone response rates. Well, in my experience (3 years at Satmetrix and 15 in total in the customer management sphere), Web survey response rates can range from under 10% to over 90%. For telephone -- the response rates that we've been getting for a customer recently have hit as high as 80%.
But a word of caution -- before measuring response rates, you need to take strike rate into consideration. The response rate is usually worked out as the number of customers who agree to take the survey as a percentage of those customers who have been been successfully reached. What we've found is, dependent on the quality of data etc., is that the strike rate -- the number of customers you successfully reach -- can be as low as 15%. The real key to response rates is having true customer engagement.
Another question from the audience asked about correlations. Popular ways of understanding the areas which impact most on loyalty are linear correlations (based on Pearson's R) and regression analysis (dependencies of variables). The point is that if you are adopting the pure two question survey, you can't build correlations or use regression. You need to have asked specific scale questions about functional areas to be able to determine correlation/regression drivers. What you can do is use your verbatim comment analysis to determine the volume of comments to highlight what appear to be key issues for your customers. But be aware that volume analysis does not always throw up the same results as correlation. It may look, from the volume of comments, that issue A is the key area for action focus -- but correlation techniques may show that issue A is only the 3rd highest driver of NPS.
Finally, following the discussion of whether or not the 2-question approach or multiple-question approach is the right questionnaire to use, there was a query about multiple question surveys -- in that surely it dilutes the strength of of Fred's pure approach? NPS in it's purest form is the two questions: "How likely are you to recommend Company X to your friends and family?"; and "Why?". My take on this is that pure is great…but, a lot of my customers are based in the B2B world and they have very complex business relationships and deliver highly complex solutions and sometimes the two question approach does not give them the clear information and correlation data they need to build a successful, actionable program.
In those cases, we tend to use a mixed approach of short surveys about overall relationship and transactional surveys (also short) for key touch points. What we keep in mind is Fred's comment about being respectful of people's time -- and not asking them to answer a survey version of War and Peace.
Now there's a thought…War and Peace style surveys…how likely are you to recommend those?
Comments