Text mining, voice mining and unstructured data analysis

Language is an interesting and innately human thing.  Language transcends the spoken word and involves vocal inflection, facial expressions and other forms of body language.  Growing up, my mother could give me “the look” and I knew exactly what she was trying to say, even without words.  “The look” told me to straighten up and fly right or face the consequences!

Given all of those elements, it becomes less challenging to figure out what the people around us are trying to say, but we, as people, still don’t always get it right.  How often have you been talking to a spouse or loved one and had your words or intentions completely misunderstood?  It happens all the time, even with the visual and auditory cues.

The challenges grow when the visual and auditory cues are taken out of the mix and language is converted to text.  Unless you are the person who wrote the review, filled out the survey or tweeted the Tweet, no one can really say with 100% accuracy what the true sentiment or intention is.

Consider these examples with inserted auditory and visual cues:

PFFT!  I love that product. <with an eye roll>

Or

Woah!  I love that product!  <with an open armed gesture and quick raise of the eyebrows>

The first example is probably sarcastic, and the person probably doesn’t really love the product.  The second might be genuine with the person being a fan of the product.

Now consider the written sentence:

I love that product.

Does the person really love the product?  With no other context it can be hard to tell.  Maybe if there were an exclamation point at the end, we could make the leap that the person was being sincere, but in reality, without the context, we don’t really know.

As organizations take steps to fully analyze the voice of the customer using sentiment analysis tools, the question on accuracy always comes up.  Everyone wants to talk about the numbers.  Here’s one take on the numbers.

It’s estimated that a professionally trained linguist can accurately assess sentiment on unstructured text 80% of the time.  Although there are incredible benefits to automating text analysis, an automated process without human intervention, even with stellar natural language processing, is not going to do better than a professionally trained human.

With 80% accuracy as our baseline, let’s take a look at numbers that I often see bandied about—85%, 95%, 99%.  But are they really getting higher accuracy rates than a linguist?  They might be overtraining their models.  They may be talking about 85%, 95% or 99% of the 80%.  I hate to say it, but like language, accuracy numbers are also open to interpretation!

A hybrid approach to sentiment analysis unites the power of statistical learning with advanced linguistic methods to home in on the sentiment buried within textual data.  This is proven to provide more accurate sentiment analysis results.  You have the best of both worlds--the unbiased statistics and the human validation.

The initial steps in beginning sentiment analysis can be daunting.  There’s so much data available (survey, call center, social media, etc.).  Where do you start?

Methodologies need to be explored.  Should we try to assess sentiment on individual documents?  Does that give us valuable information?  Should we look at larger collections of documents?

If we place our focus on every individual survey or tweet in an effort to assess sentiment, the results could be overwhelming. There are as many opinions and ideas as there are individuals.  A more widely accepted approach is to look at an overall collection of documents, for trends.  There might be certain thought leaders or influencers you will want to analyze on an individual basis, but from a voice of the customer standpoint, looking at the bigger picture can provide more insights.

Software automates a good portion of sentiment analysis

The value software brings to the table with regards to sentiment analysis is the ability to automate a good portion of the work.  Just having people (in most cases, not linguists) reading through verbatims can be fraught with challenges.  Because we all have our own experiences and biases, what may seem positive to me, may not seem positive to you.  The interpretations of a team of readers can introduce additional error and confusion.  The volume is so great that one person can’t read it all, so automated methods of assessing sentiment truly are necessary.

Software approaches still require initial human involvement in order to ensure accuracy and get the most appropriate results.  Demographics and subject matter expertise often come into play.  My dad is a baby boomer who restores classic Corvettes.  When he uses the word “boss”, he uses it as an adjective to mean that something is really good.  When I use the word “boss”, more often than not, I’m using it as a noun referencing my manager.  If I know my data is related to baby boomer car guys, I can adjust my models appropriately to capture “boss” with positive tonality rather than a neutral noun.  Human intervention is also required to validate and refine the models.  Over time, the amount of human intervention required decreases.  It’s an iterative process that becomes much less manual as it matures.

There’s no easy button approach to assessing sentiment, and often there are no concrete “right” answers.  In order to best gauge how your customers feel about products or services, it becomes necessary to spend some time understanding how your customers communicate—not only looking at the different “voice of the customer” sources, but also looking at their colloquialisms or jargon.  The work involved in gaining better insights into who your customers are, can pay off in dividends—better, more self-sufficient models over time.  Better models will yield answers that are more “right”.  Those answers can provide your organization with information that people in your organization can trust and act upon.

Richard Foley, the product manager for SAS Text Analytics has written a blog regarding precision versus accuracy:  http://blogs.sas.com/content/text-mining/2011/01/21/precision-over-accuracy/

Write and let me know your experiences with sentiment analysis precision versus accuracy.

One final note, sentiment analysis is a key topic at Text Analytics World in San Francisco.  http://www.textanalyticsworld.com/

Kathy Lange of SAS is presenting the latest in high-performance analytics on March 7.

Information has always been valued and tightly protected. But now, more than ever, information can be captured in undetectable ways in the form webpages, forums, blogs, chat, text messages, call center notes and emails to be analyzed.

According to “Vision 2015,” a report created by the U.S. Office of the Director of National Intelligence, “Information overload already presents a profound challenge to our business model. Given these challenges, the analytic community has no choice but to pursue major breakthroughs in capability.” The September/October 2011 issue of Analytics Magazine discusses how technology is facilitating critical business decisions in a fraction of the time, including stopping fraud before it happens.

The interesting and challenging aspect of analyzing text for fraud-related business problems is that often times a single bit of information can open up the case, creating a trail leading back to the criminal. Clearly, the challenge is extracting this information. Another challenge is the fact that criminals use everyday words or made up words to foil investigators. With over 500,000 new emails and documents being generated daily, the criminal may feel a bit more relaxed knowing that their actions are buried in the volume of information.  To zero in on clues, it is essential to have the tools and analytics to explore, filter, categorize, and zero in on relevant results. This intricate process takes much more than keyword matching, but rather a combination of natural language processing and pattern matching to extract entities and key facts.

On April 22^nd, I will co-present an Enron case study that explores the potential opportunity that is created for investigators when text analytics is paired with case management. The study targets investigators and fraud units interested in automating “reading” through large volumes of information in order to find the clues that solve the crime.

The goal is to find a pattern in criminal activity and text analytics can handle task more effectively and efficiently than was ever possible before.

Join us on April 22^nd in Orlando Florida at SAS Global Forum to learn more. Visit http://support.sas.com/events/sasglobalforum/2012/register.html to register.

In my work at SAS, I meet many people intent on using text analytics to help their organization achieve the next big breakthrough in competitive advantage. The most successful of those do three things well.

#1 – Listen with a business goal in mind

Leading organizations understand the value of tapping into unstructured data sources. They’re monitoring social media, analyzing customer reviews, collecting surveys, capturing call center notes, and much more. If you’re not doing this, you should. And if you are already analyzing the voice of your customers, consider how your text analytics projects are aligned with your business goals.

Text analytics empowers businesses to achieve key goals. Here are some areas where I see text analytics applied most often:

• Understand and identify topics within thousands of comments, tweets, blog posts
• Model and segment customers by integrating unstructured and structured data
• Pinpoint how your customers feel about your brand, products, and services (and how those compare to your competitor’s brands)
• Automatically categorize content, extract entities, and disambiguate text
• Manage and link jargon across business units, companies, or industries

To stay competitive, it's essential for businesses to leverage this new technology. There’s no time like the present to plan for improvement. What are your critical business goals for 2012 and what are your customers saying about your organization that affects those goals?

#2 – Capture high quality conversation

As we all know, there’s high quality conversation and there’s casual chat. Typically, the latter adds no real business value and is a source of “noise.” This probably isn’t a big deal in your social life but, in business, such noise is a distraction and a waste of time and money. The same holds true when you’re capturing online conversations, news, and information.

Natural language processing is making it easier to filter and disambiguate relevant content by identifying the part-of-speech (ie. noun, verb, adjective) and by stemming terms (ie. sell, sells, selling, sold). But language is too complex to simply filter on a single term or phrase, rather it’s becoming necessary to filter and analyze the context of words to gain a deeper understanding in order to extract high quality content from conversations.

Having access to a wealth of information is great, but the foremost challenge facing businesses is knowing what information to capture, where to find it, and how to filter it for pertinent information. The goal is to focus on relevant, high-quality content that informs and enables organizations to take strategic action.

#3 – Turn intelligence into action

Intelligence is worthless if it doesn’t enable organizations to take action. The volume of information available to the average businesses can be overwhelming. My team and I work with leading organizations to extract high-quality conversations and transform the resulting insights into action. Goals include:

• Identify and respond to unhappy customers on social media
• Trace potential fraudulent activity within survey comments, insurance claims, and social media
• Gain a competitive advantage by monitoring the online reputation and that of competitors
• Automatically cluster and categorize call center logs to identify high-volume issues for employee training
• Monitor and forecast sentiment prior to and during a product launch
• Identify emerging issues before they become costly problems

Software technology does the heavy lifting.  Text analytics software delivers the key intelligence that organizations need to gain a competitive advantage. How individuals leverage the resulting insights, and the actions they take, help them determine their goals for future text analysis, the types of conversations they’ll examine next time, and which actions are likely to effect the change they desire.

Will that improve your text analysis results?  I continually see SAS users in organizations drawing on that cycle of continuous learning to expand competitive advantage.

Comment or email me with your recent challenges and successes.

I love fantasy football. It has nothing to do with liking or understanding football (still baffled by what a tight end does after three years), but fantasy football is awesome. I can download data and write models for it (as I did in one league). Or I can just leave it to auto-draft (as I did in the other two).

Text Miner concept links

Models, however, don't predict spinal injuries well. As my opponent put it over this weekend, "I want to beat you fair and square, not because you've got half your team out with ACL injuries!" That team had Jamaal Charles, and in another I managed to pull Peyton Manning as my QB. Because of these injuries I've been struggling to find replacement players for all but the kicker position.

One of the things that's popping up this year is the use of text to enhance the models. I noticed in one of my leagues that there's an option to look up the most recent Twitter posts about players. In another I can read the blog posts on the individual games and players. My backup for Manning, for example, Kyle Orton, at least seemed to have some positive Twitter sentiment. This makes him seem worth keeping when my other option is Rex Grossman (full disclaimer: I am a Redskins fan, but I'm also a realist).

Text Miner results for NFL tight ends

 What's the value of this text-based data? More information faster about injuries and other potential factors that aren't accounted for in the rushing and passing stats. While it's easy enough to find data on football players, it usually doesn't come with a note on whether they'll be benched for being in jail during the season opener (drafted that guy, too, but since getting out he's been OK).

For example, in one league I need a new tight end – Aaron Hernandez is out and Brent Celek is underperforming by any estimate, especially compared to when I had him two years ago. I used some SAS code to scan the tweets for "tight end" over a set of days, then extracted the list of player names through our entity extraction tool. From there I was able to look at which players were mentioned the most, then see if they were available for me to pick up or trade for in my league. I could have used SAS Sentiment Analysis to further see which comments were good or bad, but since I've only got a few TEs to look over I can do that myself in the interface.

 So who will help lead the Fightin' Midgets to victory this week? Probably not Chris Cooley (I love my Redskins, but no). I'll keep monitoring Twitter all week to see who should be picked up. Maybe I can get Jason Witten (probably not). Jerome Shockey’s at least available, if not the best option.