Did you check your email and/or favourite social media site before leaving for work this morning? Did you do it before getting out of bed? Don’t worry, you’re not alone. I do it every morning as a little treat while I “wake up” and whether I realize it or not, sometimes it sets the tone for the rest of the day.

The other day I was looking at my Facebook news feed and a couple of things drew my attention. One of them was an abridged transcript of an interview on Conan of the writer of the A Song of Ice and Fire books, which the TV series Game of Thrones is based on, George R. R. Martin. Because I don’t have much time in the mornings, if the article was much longer I would probably have stopped partway through. If I had, I would have missed this quote right at the end: “I hate some of these modern systems where you type a lowercase letter and it becomes a capital. I don’t want a capital. If I’d wanted a capital, I would’ve typed a capital. I know how to work the shift key.”. This put a smile on my face and in a good mood for the rest of the day.

This made me think.

What if I had woken up just that 5 minutes too late to have read the whole thing or the article at all? How many other interesting things could I have missed because I didn’t have time or had a “squirrel” moment? And as everybody with a social media account knows, there is too much to delve into everything.

With all that data, how can I be sure that I’m exposing myself to the most interesting information? Maybe it’s just a matter of looking at how many people have viewed something. But then I like to think I’m unique so that doesn’t really work. Maybe I can only look at things my closest friends recommend. But that’s more about being a good friend than being interested. Sorry friends.

Let’s take this situation to your organisation. How do you know what information is relevant to your business? There are a myriad of techniques to analyse the structured data that the data warehouse folks have invested a large amount of time designing and the business folks have spent vetting. But how about the unstructured data – the text-based data like survey responses, call centre feedback and social media posts? This data accounts for up to 90% of all information available for analysis.

How can we make use of this text-based data? Should you have people read through all the documents and give their opinions on themes and trends? But there is an inherent bias and unreliability as people have different backgrounds and perspectives. And it’s unlikely that 1 person would be able to read everything in a timely manner. On top of all this, what we need is more than just a word search. It’s more than word counts or word clouds. It’s more than just discovering topics. In fact, what we really need is to attach a measure of prediction to words, phrases and topics.

• Escalate an incoming customer service call to the client relations team because the caller has used 3 key “future churn” phrases in the right order.
• Redesign a product because very negative feedback always contain words that are classified under “aesthetic features”.
• Discover the true customer service differentiators which give a positive Net Promoter Score (NPS).
• The areas law enforcement should increase its presence to protect the public from activities being promoted on social media that are likely to have dangerous outcomes.
• In a B2B scenario, understand and communicate the gaps in knowledge of the client organisation based on the volume, topics and severity of support calls they put through to the service organisation.
• Determine the root cause of employee concerns and the best methods to manage them.

We need Text Analytics to structure the unstructured text-based data in an objective way.

There are 2 sides to Text Analytics:

• A statistical approach where text data is converted into numerical information for analysis, and words and phrases are grouped by their common pattern across documents. The converted data and groupings can then be used alone or combined with structured data in a statistical model to predict outcomes.
• A linguistic approach or Natural Language Processing (NLP) where logical text-based rules are created to classify and measure the polarity (e.g. of sentiment) of documents.

Both sides are equally important because despite how far advanced computing algorithms have gotten, there is still a lot of nuance in the way people speak, like sarcasm and colloquialism. By using techniques from both sides in an integrated environment, we can create a whole brained analysis which includes clustering of speech behavior, prediction of speech and other behavior against topics, and prediction of the severity of sentiment towards a product, person or organisation.

One organisation which has been using Text Analytics from SAS for a number of years to provide pro-active services to their clients is the Hong Kong Efficiency Unit. This organisation is the central point of contact for handling public inquiries and complaints on behalf of many government departments.  With this comes the responsibility of managing 2.65 million calls and 98,000 e-mails a year.

"Having received so many calls and e-mails, we gather substantial volumes of data. The next step is to make sense of the data. Now, with SAS^®, we can obtain deep insights through uncovering the hidden relationship between words and sentences of complaints information, spot emerging trends and public concerns, and produce high-quality, visual interactive intelligence about complaints for the departments we serve." Efficiency Unit's Assistant Director, W. F. Yuk.

Whatever size your organisation is, and whatever purpose your organisation has, there are many sources of text-based data that is readily available and may already be amongst the structured data in your data warehouse, Hadoop or on a network drive. By using this data to supplement the structured data many people are already analyzing, we can better pinpoint not only what is driving behavior but how we can better serve our customers and our employees. Wouldn’t it be great to know what is relevant to people in and out of your organisation without having to manually read thousands of documents?

Applying Text Analytics to your documents is treating yourself with Text because amongst the masses of words you will find nuggets which will brighten up your (and your company’s) day.

If you’re interested in treating yourself with Text and are in the Sydney region this July, sign up for the SAS Business Knowledge Series course Text Analytics and Sentiment Mining Using SAS taught by renowned expert in the field Dr Goutam Chakraborty from Oklahoma State University. Dr Chakraborty will also be speaking of his experiences in the field at the next Institute of Analytics Professionals of Australia (IAPA) NSW Chapter meeting.

Happy Texting!