The Text Frontier - Text mining, voice mining and unstructured data analysis

Last Month at the SIGIR meeting in Boston , one of the presentations given by a Teragram customer attracted notice in a twitter post.

The NY Times automated the tagging of topics for their online website by their implementation of software to automatically build their indexes. However, as the tweet points out - the machine has NOT replaced Man because the newspaper continues to rely on MANUAL entries by people who maintain and build the New York Times Index, a more traditional index.



Stephen Arnold wondered in his blog why an organization might continue to require human labor on a task machine can now perform? Could be political resistance to change? or perhaps the machine fails sometimes? Perhaps the employees without skills to be reassigned are in fact prime for the next round of employees to see a "pink slip" as budgets get cut?

Mr. Arnold's ideas are all valid possibilities and I've seen cases of each in my experiences transferring technology from the research lab into business production environments. Those who put a stake in the ground and step forward to be the first to serve as role models for how text analytics can carry their business forward - ought to pause and consider their own culture.

Since the original question was about a Teragram customer implementation, I asked Saratendu Sethi , the director of Engineering at Teragram to share what he's pbserved in his consulting engagements. Here is his response.

First of all, even if automatic categorization guarantees >99% accuracy, for a News company, it is absolutely critical to not portray any wrong information for even 1%. This can only be verified by having humans validate the categorization results. They are doing that on a subset of articles, e.g. front-page articles.

Secondly, new topics constantly emerge in the coverage of current events. Even the best text mining algorithms can’t achieve perfection in spotting emerging topics because these algorithms are usually based on processing past content. Also, the

definition of emerging topics is based on human perception

which is affected by time, location and the type of entities involved in the event. Therefore, these topics have to be manually spotted and added to documents/taxonomy while they are emerging.

Having said that, the following are four benefits that Teragram categorization achieves for New York Times:
(1) If two people are asked to suggest categories on the same document on their own, they are always going to come up with different categories. Automatic categorization enforces consistency and removes human subjectivity by automatically suggesting them categories.
(2) Automatic categorization saves time because it is easier to ask editors to select appropriate categories from an automatically generated list rather than having them to think about them. With automatic categorization, I can just spend few seconds but with manual categorization I have to use few minutes to read the content and decide the appropriate topics
(3) Entity extraction (e.g. identifying person, locations, etc), which doesn’t require much human input, is automated.
(4) Automatic categorization enables New York Times to process all their past archives. Currently New York Times re-processes all their past 25years of content with updated taxonomies every few months.

4a. The human editors are only reviewing articles for current day (~500-1000 articles/day) whereas the past archives might include 100K articles/year.

4b. If “swine flu” was only identified as a News topic in 2009, then automatic categorization allows NYT to find out what other news appeared in past.

So what do you conclude from this post? How would YOU answer the question posed in the title of this entry ? It is cost effective to apply MAN "and" Machine together -- or has the science progressed enought to replace MAN ? Is it time to choose and go with Man "or" Machine approach when deciding about becoming more efficient?

Saratendu answers with the "AND" operator -- and thats the answer I prefer too --cause i'm not comfortable letting those sci fi robots and machines take over my world.

How about you?

SAS is sending data & text mining experts (including Teragram employees) over the ocean to Europe for two different events this week.
We'll have a booth in the exhibit hall at KDD09 Sunday through Wednesday. If you are one of the lucky ones attending KDD, mark your program to attend the panel discussion to listen to Dr Wayne Thompson from SAS talk about Emerging Trends in Open Standards and Cloud Computing for Data Mining .

Even if you don't make it to the KDD conference to personally pick up the new book authored by the conference chair John Elder , you can experience our Software-on-Demand version of data mining by buying his book, "Handbook of Statistical Analysis and Data Mining Applications."


The second event where you can find us is at the SAS conference devoted to ANALYTICS called A2009 in Denmark July 1,2. The program is online. There you can read the abstract about the a Swedish Insurance firm that studied hand written notes collected by police officers and security guards during 2004-2007.

At both shows, you'll be able to see live demos of our software and pick up a hard copy of the most recent fact sheet, highlighting the enhancements that are now available with the TEXT MINER 4.1 version that was made available to customers 5 weeks ago. Those of you reading this blog that haven't yet seen it may want to read the fact sheet on our SAS 9.2 release of Text Miner on the SAS website.

What does your summer hold for you? Do you have travel plans to shows or conferences with text analytics tracks or sessions included? Please add a comment to this blog and do share!