Tag: natural language processing
Interestingly enough, paperclips have their own day of honor. On May 29th we celebrate #NationalPaperclipDay! That well-known piece of curved wire deserves attention for keeping our papers together and helping us stay organized. Do you remember who else deserved the same attention? Clippit – the infamous Microsoft Office assistant, popularly known as ‘Clippy’.
As we honor Mental Health Month, there are many calls to reduce suffering. Seems reasonable, right? It’s even in California’s Mental Health Services Act (MHSA), where public systems are called to “reduce subjective suffering.” And as we broadly focus more on outcomes in health, measuring suffering (and hopefully its reduction)
This blog shows how the automatically generated concepts and categories in Visual Text Analytics (VTA) can be refined using LITI and Boolean rules. I will use a data set that contains information on 1527 randomly selected movies: their titles, reviews, MPAA Ratings, Main Genre classifications and Viewer Ratings.
In this blog, I use a Recurrent Neural Network (RNN) to predict whether opinions for a given review will be positive or negative. This prediction is treated as a text classification example. The Sentiment Classification Model is trained using deepRNN algorithms and the resulting model is used to predict if new reviews are positive or negative.
I just spent four inspiring days talking to customers about the many ways they are putting analytics into action in their organizations. From computer vision models that interpret medical images to natural language processing models that analyze supply chain records, SAS users are doing ground-breaking work with analytics and AI.
Imagine a world where satisfying human-computer dialogues exist. With the resurgence of interest in natural language processing (NLP) and understanding (NLU) – that day may not be far off. In order to provide more satisfying interactions with machines, researchers are designing smart systems that use artificial intelligence (AI) to develop
Et si, en dehors de la nouvelle organisation des moyens de production, la 4ème révolution industrielle induisait également une évolution significative dans la gestion de la connaissance intrinsèque à chaque domaine ? Et si les nouvelles technologies numériques permettaient aux acteurs opérationnels d’accéder simplement à cette connaissance, le plus souvent fruit de méthodes
Natural language understanding (NLU) is a subfield of natural language processing (NLP) that enables machine reading comprehension. While both understand human language, NLU goes beyond the structural understanding of language to interpret intent, resolve context and word ambiguity, and even generate human language on its own. NLU is designed for
Recently, the North Carolina Human Trafficking Commission hosted a regional symposium to help strengthen North Carolina’s multidisciplinary response to human trafficking. One of the speakers shared an anecdote from a busy young woman with kids. She had returned home from work and was preparing for dinner; her young son wanted
Structuring a highly unstructured data source Human language is astoundingly complex and diverse. We express ourselves in infinite ways. It can be very difficult to model and extract meaning from both written and spoken language. Usually the most meaningful analysis uses a number of techniques. While supervised and unsupervised learning,
Artificial intelligence and machine learning are words that evoke a sense of inhumanity, of cold and mechanical functions. In reality, AI is coming closer and closer to reaching human understanding through capabilities like deep learning and natural language processing (NLP). With early chatbots, you had to ask exactly the right
Artificial intelligence (AI) is a natural evolution of analytics. Over the years, we have seen AI add learning and automation capabilities to the predictive and prescriptive jobs of analytics. We have been building AI systems for decades, but a few things have changed to make today’s AI systems more powerful.
I look forward to Pi Day every year at SAS because it's a day of celebration including yummy pies and challenging games that challenge you to recall the digits after the decimal point in Pi. Plus you get to wear Pi t-shirts (I have about seven). Today, though we are going to talk about DLPy which sounds like Pi and
The Special Olympics is part of the inclusion movement for people with intellectual disabilities. The organisation provides year-round sports training and competitions for adults and children with intellectual disabilities. In March 2019 the Special Olympics World Games will be held in Abu Dhabi, United Arab Emirates. SAS is an official
There is tremendous value buried text sources such as call center and chat dialogues, survey comments, product reviews, technical notes, legal contracts... How can we extract the signal we want amidst all the noise?
When it comes to forecasting new product launches, executives say that it's a frustrating, almost futile, effort. The reason? Minimal data, limited analytic capabilities and a general uncertainty surrounding a new product launch. Not to mention the ever-changing marketplace. Nevertheless, companies cannot disregard the need for a new product forecast
Every day, military intelligence analysts sit behind computers reading a never-ending stream of reports, updating presentation templates and writing assessments. But intelligence is more than documenting events and sharing breaking news. It involves understanding and predicting complexities in human behavior across various organizational constructs and using facets of information to
Regular expressions are a powerful method for finding specific patterns in text. The syntax of regular expressions is intimidating, but once you've solved a few pattern-recognition problems with regex, you'll never go back to your old methods.
Amidst the growing popularity of modern machine learning and deep learning techniques, one of the biggest challenges is the ability to obtain large amounts of training data suitable for your use case. This post discusses how the analytical approach for Named Entity Recognition (NER) can help.
In my first post I looked at the role of analytics in policing and how analytics could and should be used to benefit modern policing. However, a key point that can be forgotten is analytics is only as good as the data it is based on. It’s vital to have
Deep learning has taken off because organizations of all sizes are capturing a greater variety of data and can mine bigger data, including unstructured data. It’s not just large companies like Amazon, SAS and Google that have access to big data. It’s everywhere. Deep learning needs big data, and now
As someone who worked in the police force for many years, I know just how important each small piece of evidence can be in securing a conviction. So in times of tightening budgets, imagine how useful it could be to have technology that helps predict when or where crimes will
Word Mover's Distance (WMD) is a distance metric used to measure the dissimilarity between two documents, and its application in text analytics was introduced by a research group from Washington University in 2015. The group's paper, From Word Embeddings To Document Distances, was published on the 32nd International Conference on Machine
My local middle school publishes a weekly paper. Very recently, I noted an article in that paper regarding an expose on human trafficking overseas, "World Slavery: The Terrors Our World Tries to Forget." The eloquent article in part highlighted how children have been exploited in the fishing industry in Ghana
Maybe you’ve heard of text analytics (or natural language processing) as a way to analyze consumer sentiment. Businesses often use these techniques to analyze customer complaints or comments on social media, to identify when a response is needed. But text analytics has far more to offer than examining posts on
Like most men in the world, shopping (especially shopping for clothes) is an unpleasant experience for me, as I really feel like my time could be much better utilised. I make it a personal mission to get in and out in minimal time. So, I wonder to myself, “Could AI
It is estimated that that many companies now hold over 80% of their data in an unstructured form. In other words, not as numbers or code, but free text. This textual data arises from social media, customer comments, call center notes, books, emails, messages and the like, and holds enormous
As a former intelligence analyst, I can't help but breathe a huge sigh of frustration. The special AI "task forces" and their massive budgets are great, but it's time to get honest about the rest of the military. Ask any every day soldier, sailor, airman or Marine their opinion of
A chatbot is a computer program that uses natural language processing (NLP) and artificial intelligence to simulate human conversation and derive a response. Essentially, it’s a machine that can chat with you or respond to your chatter. Chatbots can save time and money when used to handle simple, automated tasks.