In today's world, data-driven systems make significant decisions across industries. While these systems can bring many benefits, they can also foster distrust by obscuring how decisions are made. Therefore, transparency within data driven systems is critical to responsible innovation. Transparency requires clear, explainable communication. Since transparency helps people understand how
Tag: natural language processing
SAS' Ali Dixon and Mary Osborne reveal why a BERT-based classifier is now part of our natural language processing capabilities of SAS Viya.
Editor's note: This article follows Curious about ChatGPT: Exploring the origins of generative AI and natural language processing. As ChatGPT has entered the scene, many fear and uncertainty have been expressed by those working in education at all levels. Educators worry about cheating and rightly so. ChatGPT can do everything
How did we get to a place where a conversational chatbot can quickly create a personalized letter? Join us as we explore some of the key innovations over the past 50 years that help inform us about how to respond and what the future might hold.
SAS' Kirk Swilley and Tom Sabo showcase how you can use perform text analysis on minimal structured narrative data to spot patterns of possible human trafficking.
ChatGPT from OpenAI has changed how the general public thinks about AI. What does this mean for analytics practitioners?
Imagine trying to dig a useful bit of information out of 50,000 lines of a chat log? Now, imagine if that needle in the haystack was the difference in a criminal being arrested or staying at-large? Thousands of lines of confusing and unreadable chat text are more and more frequently
Using such features and Natural Language Processing capabilities like text parsing and information extraction in SAS Visual Text Analytics (VTA) helps us uncover emerging trends and unlock the value of unstructured text data.
To find exact duplicates, matching all string pairs is the simplest approach, but it is not a very efficient or sufficient technique. Using the MD5 or SHA-1 hash algorithms can get us a correct outcome with a faster speed, yet near-duplicates would still not be on the radar. Text similarity is useful for finding files that look alike. There are various approaches to this and each of them has its own way to define documents that are considered duplicates. Furthermore, the definition of duplicate documents has implications for the type of processing and the results produced. Below are some of the options. Using SAS Visual Text Analytics, you can customize and accomplish this task during your corpus analysis journey either with Python SWAT package or with PROC SQL in SAS.
In the face of rapid digitalization and modernization, data scientists in Cameroon joined the SAS Hackathon seeking a way to preserve indigenous African languages.
A cancer journey affects both physical and mental health. This often results in feelings of social isolation, loss of identity, clinical depression and even PTSD. This often goes unrecognized and undiagnosed due in part to lack of resources, tools and time. Swedish startup War On Cancer wondered whether they could
Corpus analysis is a technique widely used by data scientists because it provides an understanding of a document collection and provides insights into the text.
The 2021 SAS Hackathon was a major success and teams are now signing up for the 2022 hackathon. We are inviting you to join us. The world has lots of problems in search of answers and it’s your chance to contribute some creative solutions. Here’s what the team from KPMG
Technological advancements in connectivity and global positioning systems (GPS) have led to increased data tracking and related business use cases to analyze such movements. Whether analyzing a vehicle, an animal or a population's movements - each use case requires analyzing underlying spatial information. Global challenges such as virus outbreaks, deforestation
El texto no estructurado es la mayor fuente de datos generada por el ser humano y crece exponencialmente cada minuto. No hay que olvidar que la tecnología está ya presente en todos los aspectos de nuestras vidas, tanto profesionales como personales, y nos permite conversar rápidamente a través de textos,
Conversational AI can offer a way to provide that always-on 24/7, fast, convenient experience that can go anywhere (phone, computer smart speakers, even your car). It can provide a human-like experience through real-time, personalized interaction with AI running in the background. This technology is being applied across many industries for a variety of use cases (both customer-facing and for internal use).
Gemeenten krijgen nogal wat kritiek te verduren. Er is geen twijfel dat sommige beter kunnen, maar vele zijn sterk gericht op het leveren van hoogwaardige diensten voor hun burgers. Echter, deze 'goed nieuws' verhalen komen zelden in de pers - zelfs niet in de lokale kranten in rustige weken. Zoals
Discovery is an important part of setting up your analysis for success – essentially it prevents you from plunging into a haystack to try to find that elusive needle, and rather, helps you organize the haystack into neater, compact organized bales that you can navigate with ease. Proper discovery can help you more efficiently find patterns in your data set.
Unlocking the potential of your unstructured text data can lead to great business outcomes but the prospect of starting a new or enhancing your existing Natural Language Processing (NLP) program can feel overwhelming because of the inherently unique (and sometimes messy) nature of human language. Text data doesn’t fit neatly into rows or columns the way that structured data does, which can make it seem more complex to work with. Conversations and written language range from objective statements to subjective perspectives and opinions. The same sentence, depending on its intent and the nuances in how it's said, can have a positive, negative, or neutral sentiment. To get us started, we'll share different types of NLP models used to analyze unstructured data with a focus on the hybrid approach.
Local government gets some bad press. There is no doubt that some could be better, but many are strongly focused on delivering high-quality services for their citizens. However, these "good news" stories seldom make the press – even in local newspapers in slow weeks. Like most public sector organisations around
I think that this pandemic has put digital transformation at the top of every executive agenda.
Natural Language Processing can offer invaluable benefits to councils and increase resident satisfaction.
Which measures financial services can take to keep their customers complaints at a minimum.
Interestingly enough, paperclips have their own day of honor. On May 29th we celebrate #NationalPaperclipDay! That well-known piece of curved wire deserves attention for keeping our papers together and helping us stay organized. Do you remember who else deserved the same attention? Clippit – the infamous Microsoft Office assistant, popularly known as ‘Clippy’.
As we honor Mental Health Month, there are many calls to reduce suffering. Seems reasonable, right? It’s even in California’s Mental Health Services Act (MHSA), where public systems are called to “reduce subjective suffering.” And as we broadly focus more on outcomes in health, measuring suffering (and hopefully its reduction)
This blog shows how the automatically generated concepts and categories in Visual Text Analytics (VTA) can be refined using LITI and Boolean rules. I will use a data set that contains information on 1527 randomly selected movies: their titles, reviews, MPAA Ratings, Main Genre classifications and Viewer Ratings.
In this blog, I use a Recurrent Neural Network (RNN) to predict whether opinions for a given review will be positive or negative. This prediction is treated as a text classification example. The Sentiment Classification Model is trained using deepRNN algorithms and the resulting model is used to predict if new reviews are positive or negative.
Imagine a world where satisfying human-computer dialogues exist. With the resurgence of interest in natural language processing (NLP) and understanding (NLU) – that day may not be far off. In order to provide more satisfying interactions with machines, researchers are designing smart systems that use artificial intelligence (AI) to develop
Et si, en dehors de la nouvelle organisation des moyens de production, la 4ème révolution industrielle induisait également une évolution significative dans la gestion de la connaissance intrinsèque à chaque domaine ? Et si les nouvelles technologies numériques permettaient aux acteurs opérationnels d’accéder simplement à cette connaissance, le plus souvent fruit de méthodes
Natural language understanding (NLU) is a subfield of natural language processing (NLP) that enables machine reading comprehension. While both understand human language, NLU goes beyond the structural understanding of language to interpret intent, resolve context and word ambiguity, and even generate human language on its own. NLU is designed for