Maybe you’ve heard of text analytics (or natural language processing) as a way to analyze consumer sentiment. Businesses often use these techniques to analyze customer complaints or comments on social media, to identify when a response is needed.

But text analytics has far more to offer than examining posts on social media to check when someone is making a complaint, and about what. I’ve been involved in a lot of exciting text analytics projects since I started working in this field in 2006. Let me tell you about five that still inspire me – and might inspire you too.

1. Understanding public opinion without polls.

Text analytics offers huge potential for governments to understand the needs and wants of citizens, without the need for expensive and often inaccurate opinion polls or focus groups. Although, text analytics works on surveys, opinion polls and the results from focus groups too!

2. Discovering new opportunities for aerospace research.

Text analytics can examine trends in research, and this remains a popular use of analytics today. Employing automated analysis of textual sources helps to prevent research organizations from reinventing the wheel and ensures that new work builds on previous studies. Research is generally a series of incremental steps, and text analytics helps to identify the turns in the staircase, as it were, showing researchers potential avenues for new studies.

3. Reforming unfair lending practices.

Text analytics has been used as a support for policymaking. For example, we used it to analyze publicly-available Consumer Financial Protection Bureau data to assess consumer financial complaints and gather information about trends and areas of concern. Our results suggest that text analytics could help to protect consumers from unfair financial practices. In one case, the results even led to reform of those practices through legislation. Through efforts of the Consumer Financial Protection Bureau (CFPB), congress replaced the Good Faith Estimate with the much easier to understand Loan Estimate.

4. Saving lives during disaster recovery.

We’ve also used text analytics to better assess recovery needs during and after disasters. This has huge practical implications for readiness to respond, including having the right support available on hand, and ensuring that those responsible for planning disaster relief know what is likely to be needed. There is similar potential for using text analytics in epidemiology to respond more quickly to outbreaks of disease, or to protect the food supply from problematic chemicals.

5. Prevent human trafficking.

Looking for patterns in crime reports from state to state and country to country can be a challenge. We used text analytics to comb through 600 international reports about human trafficking and surface useful data for advocacy groups and overseeing agencies. Clustering common words and locations can identify trafficking routes and help agencies create programs to prevent these crimes.

Why text analytics works for all these areas

So why has text analytics made such a big impact in these and other areas? There are three main benefits:

1. Reading reports, research papers or complaints takes time. Until the arrival of text analytics, there were few, if any, shortcuts to sitting down and reading through a large volume of information. More people reading meant less time, but led to potential issues of consistency between individuals. There was no guarantee, even, that the same person would consider each report in exactly the same way. The sheer volume of previous research and reports meant that few individuals had time to read every last one. This kind of analysis was therefore a long process, and the quality of the results could not be guaranteed.
2. Accuracy or consistency. Automation through text analytics also increases consistency and reliability of results. Algorithms can be programmed to identify patterns that might not even be visible to the human eye, not least because of the volume of information to be considered.
3. It’s true. We’re swimming in data volumes that unimaginable in the past. Text analytics lets us take advantage of all the text based data that’s available online in emails through phone calls and more.

Supplementing human abilities

Text analytics has been enormously helpful as a way of supplementing and extending human abilities, by adding speed and accuracy for a quantitative, rather than a qualitative assessment of text data. As AI is used more and more, this will only increase. Text analytics, beyond chatbots, forms the core of any artificial intelligence (AI) system that relies on human interaction through language. It can also be used as a core component of AI systems for productivity enhancement, to extend and learn from analysts’ reviews of textual information in a number of different domains. I hope to explore this, and other applications of text analytics in segments to come.