sascom voices - The sascom magazine blog

Last week I saw a slide from a customer presentation that said 85 percent of that company's data is unstructured data. This isn't usual, either. But what does it mean? First and foremost, it means the data we're all collecting at unprecedented rates is no longer stored strictly in spreadsheets and tidy numeric formats. Instead, it's in field service notes, tech support memos, customer comments, e-mails, blogs, wikis, news feeds, transcripts, medical records, case histories and hundreds of other text-based formats.

It also means that it's getting harder and harder to find the information you need from the hundreds and thousands of text-based documents within an organization. Companies like Google, Microsoft and others are hoping to solve this problem with enterprise search technologies that scour unstructured data sources throughout the enterprise and feed users results in the same user-friendly format they've become accustomed to seeing in Internet search results.

But is search enough? When you're looking for answers in your company's growing collection of text-based documents, are you going to find the answers you need with a basic search algorithm? If you're looking for a single report or piece of background information, yes. But if you're looking for deeper answers, the answer is no.

Here's an example. Enterprise search can quickly reveal how many customers are complaining about your company's newest cellular calling plan for college students. But can it tell you why they're complaining? Or what other factors are affecting their complaints? No. But text mining can.

Text mining transforms text into a structured format by automatically classifying documents and finding key relationships that deliver instant insights. So, now customer letters and call center notes provide valuable messages for WHY some customers may be unsatisfied.

It might turn out, in our example above, that students in different geographic regions are complaining for different reasons. And parents' complaints might differ drastically from students' complaints. Those relationships would take much longer to ascertain with enterprise search. With text mining, the insight is automatic.

Need another example? Read how text mining helped prevent a leg amputation, from Computerworld.

What could you learn with the ability apply text mining to your company's unstructured data sources?