This post, co-authored with my colleague Sophia Rowland, is the first of two focused on operationalizing text analytics with SAS.

As businesses undergo digital transformation, a common priority is to discover new insights through harnessing text, the largest human-generated data source. SAS has been helping companies analyze their unstructured data for various industries and across a multitude of use cases for many years. Analysts tend to agree with our success as SAS was recently named a Leader in the Forrester Wave for AI-Based Text Analytics Platforms. However, as we often preach here at SAS, it's not just the modeling that's important. The most important aspect is how you operationalize and gain value from those results.

Challenges with Text

Analyzing unstructured data can be much more challenging than structured or tabular data. Through discussions with customers and our own experiences, we identified three types of problems commonly encountered when working with text data: the breadth of techniques, the subjectivity of text data, and the building of a robust text analytics process.

Breadth

First, there is the challenge of breadth as it relates to techniques and maturity. There are a multitude of techniques for analyzing text and we often identify use cases for all the techniques SAS has to offer. Levels of analytical maturity and the speed to value are often factors for customers. Some use cases require quick wins with a rules-based approach while others have a team of data scientists eager to use our more sophisticated deep learning techniques. With SAS Visual Text Analytics, we provide a tool for natural language processing (NLP) that has a wide range of techniques that supports users of all types, including programmatic and visual user interface.

Subjectivity

With unstructured data, the analysis or output of a model is rarely the end result. Unlike structured data where outcomes are generally black and white, there is an aspect of subjectivity that is needed when analyzing text. If you stop making a payment, the system knows you are in a state of delinquency. However, sentiment, text topics, and categorization generally require context and human auditing to fully understand what a model might suggest. To solve this challenge, we've developed an integrated solution to help our customers understand the context and provide relevant feedback to drive better results.

Process

Unstructured data presents another challenge over that of structured data in that often, the people generating the model and the ones consuming the results are very different. For example, a data scientist will generally develop a model off reviews from their company’s mobile app that will want to be consumed by UX developers for design feedback, product owners for operational feedback, and even legal and compliance teams for regulatory risk. Each stakeholder has different needs as it relates to when and how they consume that output, as well as what type of feedback they present back. Our solution provides a customizable investigative framework complete with alert generation and workflow to orchestrate across all these stakeholders.

What's the Answer? Text Investigation Framework

The Text Investigation Framework utilizes several technologies built on SAS Viya, including SAS Visual Text Analytics, SAS Visual Data Mining and Machine Learning, and SAS Visual Investigator. SAS Visual Investigator acts as the orchestrator to surface the results. With its broad set of capabilities, SAS Visual Investigator can perform scenario authoring, alert generation and disposition, and comprehensive workflow to gather vital outcomes and feedback.

The Text Investigation Framework is a flexible solution for addressing text challenges across several domains. As with all things, our framework starts with the data and our ability to ingest data from CSVs, databases, cloud, and even documents. Next, we build text models in SAS Visual Text Analytics, incorporating a broad set of techniques like text extraction, topics, and categorization. From there, SAS Visual Investigator takes over to build rules, understand context, and keep the human in the loop via triage and disposition. This portion of the framework is critical as it will act as the feedback loop, allowing the human input to improve the text models moving forward.

Applications

We believe that this framework can be utilized across several domains and use cases. One example is the ongoing need for contract analytics. With LIBOR rates being phased out, an investigative process for finding contracts relying on LIBOR and resolving contract changes will be necessary. In compliance with various privacy regulations, this framework can help companies navigate multiple data sources to find all mentions of a specific individual. From an insurance perspective, adjuster notes can be analyzed and investigated for efficient claims processing. Our Text Investigation Framework can be applied to many more uses cases, wherever an organization needs to operationalize text analytics.

In the next post of this series, my colleague Sophia Rowland will dive deeper into an application of this framework.