Voice of customer analysis (Part 2)

3

voice of the customer analysisThis is my second article about voice of customer analysis; you can find the first here. The first time we discussed that a simple sentiment polarity score was a rather a narrow view. This time we will examine a more insightful approach, using voice of customer analysis to monitor customers’ opinions and understand the issues they are raising. It helps if you are clear on your brand’s priorities from a customer experience perspective. For example, if we compare priorities for two online retail brands:

  • For Brand A, it might be customer service, followed by product quality and then price.
  • For Brand B, it might be price, followed by product range and the ease of ordering/returns.

Many of our customers who have embraced an Net Promotor Score (NPS) program have already identified the relative importance of the different drivers of customer satisfaction. For voice of customer analysis, these can be calculated from NPS survey results, using SAS/IML® (interactive matrix programming language) to implement an approach like a Shapley Value regression, to understand the relative importance to customers of different product and service features. Our customers can then use a weighted scorecard, based upon these relative importance feature dimensions, to assess feature mentions in customer communications.

SAS® Text Analytics is then used to consistently track these opinion/feature mentions. The customers’ opinions can be thought of as having five dimensions.

The first three can be found in the text:

  1. Product/brand.screenshot 2
  2. Feature/attribute.
  3. Sentiment positive, negative  or neutral.

The other two elements are typically available as structured information relating to the source document:

  1. Opinion holder ID (ideally a known customer, but if not, a social media source/ID or unique survey ID).
  2. Date and time of the opinion (for example, the date the survey was completed, or for a complaint, the date of the complaint and the incident itself).
Learn how to deliver exceptional customer experiences with voice of the customer analysis.

 

To achieve the best accuracy with voice of customer analysis, the brand/feature/ sentiment mentions need to be assessed in context, rather than just looking at isolated words or phrases. Consider some of the more sophisticated ways your customers use language when expressing their sentiment:

  • Negated sentiment, for example, “the movie wasn’t good .”
  • Submodifiers – this is where adverbs are used in front of adjectives (or other adverbs). Sometimes these increase the polarity of a positive/negative (e.g., “the dress was very pretty” or the “the food was really salty”). At other times, the adjective may be neutral, but the submodifier makes it negative, for example “the dress was too purple” or “the food was incredibly spicy.”
  • Whether words are positive or negative will sometimes be specific to the product feature. “Fast” might be considered a positive word in the context of a “fast hotel check-in,” but negative if the comment is “it was a very fast spa session.”
  • With web chat or call center interactions where there is a conversation with the customer, you might consider customers’ opinions at the end of the conversation. They might start with several negative utterances, but if at the end they are positive, you might wish to also consider this in the weighing. This decision depends on your objective. The negative expressions about service features might be more useful from a root cause and resolution perspective.
  • Consider how to handle irrelevant positive and negative opinions. For example:

I love my Apple iPhone; however, I'm angry that ABC Bank doesn't offer Apple Pay yet, and then Airline XYZ lost my phone when I left it at the gate in the airport.

This is positive if you are Apple and negative if you are ABC Bank, but what about if you are the XYZ airline, which isn't actually responsible for the gate at the airport, but the customer thinks it is?

  • Sarcasm can be handled. However, there is a trade-off between accuracy and the effort required to ensure sarcasm is correctly handled. It’s often best to focus on the most common examples, such as positives relating to negative concepts (e.g., “I was delighted to spend my lunch break in the queue in your bank”) or wordplay on brand names.

SAS Text Analytics allows you to consider the language in context. It is worth highlighting, however, that analyzing text to assess sentiment is often subjective, even for a human with considerable knowledge of the subject area. Automated voice of customer analysis using SAS Text Analytics can achieve high levels of accuracy – in excess of 90 percent – but there is a trade-off to be considered between accuracy, the time to conduct the analysis and the value of the improved accuracy of the score.

The customer experience opinions weighted by our brand’s priorities are now available as an output from SAS Text Analytics and can be benchmarked, visualized and explored within SAS® Visual Analytics. Christina Engelhardt showed a number of good examples of this within her recent Come chat with us! post.

screenshot 3Ultimately, the consistent measurement of the voice of customer to understand customer opinions can have significant benefits. Monitoring customer satisfaction can improve both business performance (through a weighted sentiment scorecard) and the understanding of customer opinions at a product and feature level. This can be used to:

  • Improve handling of customer interactions (e.g., web chat responses), to train staff on customer preferences and improve resolution/call times.
  • Continually improve through understanding problems and the root causes of service and product quality issues.
  • Provide feedback to product and service design.

We would love to hear from you about your experiences with voice of customer applications in your organization.

Share

About Author

Matthew Stainer

Sr Systems Engineer

Matthew Stainer is a business analytics consultant in the SAS UK Analytics team, he supports customers enhance their businesses decision making. He works across the analytical technologies including data mining and forecasting. His particular areas of interest and expertise is text analysis, identifying the impact of analytical solutions on business processes and communicating the results of analysis to a business audience. He has specific experience in Customer Management solutions, including Complaints Analysis, NPS, Customer Analysis, Optimisation, Decision Management, Recommendation Engines, Campaign Management, Debt Management and Web Analytics. He has been at SAS almost 10 years and the software industry for over 25, working across a variety of vertical sectors including retail banking, insurance, healthcare, retail, media and telecommunications.

3 Comments

  1. Hi Matthew,
    Thanks for shedding light on the complexities we encounter in real world comments. Do you feel it helps if we use concepts and facts to break a comment into "sub-documents" , and then run sentiment analysis on them? For example, your airport related comment could be separated into 3 phrases - "I love my Apple i-phone" , "I am angry that ABC Bank ..." and "XYZ airline..." . You then have isolated facts surrounding a single entity or subject. It might be challenging to link them back to the parent document though.

    Another aspect I feel we could consider would be "hooks" or conjunctions - words like "however", "therefore" etc. might work to provide either supporting or contrasting arguments to a sentiment. In the online world where people might be less careful about punctuation (me included), we could use these hooks to separate phrases in the absence of proper commas and semicolons.

    • Matthew Stainer
      Matthew Stainer on

      Sundaresh,

      Typically I find its enough to focus on the relevant sentiment you are interested in and ignore non relevant mentions / sentiment. You are absolutely right about conjunctions and more generally transition phrases indicating different clauses or writers ideas. I'll often incorporate these into my analysis. You can look for different types of transition phrases, indicating different types of transitions e.g. "but" or "however" for contrasting idea, "as well as" or ""likewise" showing addition, "in conclusion" or "for now" showing summary or sequence, etc.

  2. Hi Matthew,

    Very interesting article and voice of customer analysis using text mining is something we are getting involved in. As you may recall, one of the interesting finding is also when sentiments from open ended question about an aspect of service don't necessarily match up scores given by customers for that aspect of service.

    I would also add that "fast hotel check-in" could also be viewed negatively if fast refers to customer interaction (I.e. they were rushed at check-in) but fast could be viewed positively if it refers to queue length at check-in. Often this is hard to understand if the customer only mentions "fast hotel check-in" without elaborating on it.

    I liked your explanation of submodifiers and looking forward to sharing our work on this specific area for your expert views/suggestions 🙂

Leave A Reply

Back to Top