It is estimated that that many companies now hold over 80% of their data in an unstructured form. In other words, not as numbers or code, but free text. This textual data arises from social media, customer comments, call center notes, books, emails, messages and the like, and holds enormous value for the company that can decipher it, discover new information, topics and term relationships and generate insights. Enter text mining, a way of exploring this huge resource. In my conversations with SMEs across EMEA, I’ve received six common questions that might be of interest to a broader audience.
1.What is the difference between text mining and enterprise search
Alison Bolen’s article is a useful introduction to the differences between text mining and enterprise search. Many providers are offering solutions involving search to help companies manage unstructured data. This, however, is unlikely to be enough to do more than identify documents, or at best headlines. For deeper insights, text mining is essential. Alison concludes with a couple of examples, including one of a personal story about how one mathematician used text mining to bring together information about dosage and use of a particular antibiotic, and save her husband from a leg amputation.
Alison Bolen’s article is a useful introduction to the differences between text mining and enterprise search.
2. What are some practical examples of business impact?
One of the best ways to understand how a technique is being used, and to get a grasp of its potential, is to read about examples. Alison’s article above provides some of these, and this article adds another. The article explains how Alberta Parks Department is using text mining to understand the free text information from the visitors’ surveys that the Parks Department has been using since 2002. Automatic coding of data has hugely speeded up the process of analysis, and using text mining allow the Parks Department’s analyst to generate better insights from the data, get a deeper understanding of customer issues and what park visitors really want.
3. Is text mining only for commercial businesses?
No. Healthcare is an area with a particularly strong tradition of unstructured data, particularly from patients’ medical notes. This review by Christian Hardahl describes how Hospital Lillebaelt, in Denmark, decided to use text mining to ensure that doctors were able to access all relevant information about patients without having to read every last word of the notes. The article goes into detail about some of the techniques used, and how they were chosen to aid translation to other settings and countries.
4. Does text mining need ‘big data’?
It is always fun to read about personal trials of particular software. This one, by author Gerhard Svolba, describes using text mining software to explore the 59 chapters in the two books that he has written for SAS Press. He admits that this is not exactly a massive ‘big data’ problem, but it does show the potential of the software to cluster and group information, and enable the analyst to focus on the areas of interest. This article shows very effectively that text mining is not just for huge companies, and huge problems, but can be used on a much smaller scale.
5. Is text mining always about efficiency?
A slightly more philosophical ‘take’ on text mining, this piece by Lonnie Miller discusses the fact that textual analysis can help you to see things that you would otherwise have missed. The information is there, but you have not previously seen it, a bit like the iceberg that sank the Titanic. In particular, Lonnie notes that textual analysis can add predictive value, for example, by helping companies to identify customers who may go elsewhere from messaging data. He also shows how it can help different company functions, including marketing and HR.
Lonnie Miller discusses the fact that textual analysis can help you to see things that you would otherwise have missed.
6. Will text analytics benefit from real-time approaches?
My final pick discusses how real-time analytics, and particularly textual analytics, is changing the way that companies can respond to customers, manage security, and check compliance. It describes the use of event stream processing techniques, used to analyse text and other inputs in real time, and then provides some examples of how companies are using these techniques. From real-time analysis of customer calls to improve complaint management, through to prevention of fraud, collusion and insider trading, organisations large and small are using these techniques to improve insights into customers, staff, and supplier behavior, and therefore their bottom line.
These are by no means all the possible queries you might have about text mining, but they are the ones I hear most often. I’d love to hear about your adventures in booting up or advancing your organisations text analytics capabilities.
