SAS' Julia Florou-Moreno shows you how to use generative AI to build a digital assistant that interacts with a model using natural language conversation.
SAS' Ali Dixon and Mary Osborne reveal why a BERT-based classifier is now part of our natural language processing capabilities of SAS Viya.
To find exact duplicates, matching all string pairs is the simplest approach, but it is not a very efficient or sufficient technique. Using the MD5 or SHA-1 hash algorithms can get us a correct outcome with a faster speed, yet near-duplicates would still not be on the radar. Text similarity is useful for finding files that look alike. There are various approaches to this and each of them has its own way to define documents that are considered duplicates. Furthermore, the definition of duplicate documents has implications for the type of processing and the results produced. Below are some of the options. Using SAS Visual Text Analytics, you can customize and accomplish this task during your corpus analysis journey either with Python SWAT package or with PROC SQL in SAS.
Word embeddings are the learned representations of words within a set of documents. Each word or term is represented as a real-valued vector within a vector space. Terms or words that reside closer to each other within that vector space are expected to share similar meanings. Thus, embeddings try to capture the meaning of each word or term through its relationships with the other words in the corpus.
With the release of SAS Viya 2020.1.4, text categories and concept models can now be deployed into production with just a few clicks and used to score data in-batch and via API! You can also now use these models in decision flows.
SAS Conversation Designer is available with every offering that also includes SAS Visual Analytics. Users can easily access Visual Text Analytics capabilities from SAS Conversation Designer with minimum additional configuration.
