How to measure data quality
Learn the nuts and bolts of how to measure data quality from expert Jim Harris.
English
Going beyond the box score: Text analysis in sports
Critics of sports analytics (and there are some entertaining ones) love to point out that analytics isn’t capable of capturing the things that don’t show up on a box score. A player who dives on the floor to save a loose ball, a quarterback strategically misleading a defender to free
Doing good and giving back
Written by Allison Hines and Kara Roberts Over the last few weeks, we've shared stories of how SAS employees are banding together while staying apart during the COVID-19 crisis. Our people have donated their time and skills to the cause, including making masks for healthcare workers, leading virtual dance classes for students with special needs and hosting trivia games for charity via Zoom – among many other efforts. Keep reading
Interactions with spline effects in regression models
A SAS customer asked how to specify interaction effects between a classification variable and a spline effect in a SAS regression procedure. There are at least two ways to do this. If the SAS procedure supports the EFFECT statement, you can build the interaction term in the MODEL statement. For
How to explain your complex models in SAS® Viya®
In the second of a three-part series of posts, SAS' Funda Gunes and her colleague Ricky Tharrington summarize model-agnostic model interpretability in SAS Viya.
Not on the high street?
Intelligent decisioning can help telcos survive the retail shutdown by accelerating the shift to digital channels
In Response to Recent Events: A message from the Work/Life Center
A message from the Work/Life Center: It is not surprising that the SAS community is deeply moved and troubled by the recent examples of racism that reflect larger systemic problems in our nation. Work/Life wants to respond to your requests for ideas about supporting colleagues and taking action in your community. We understand the urge to act and will provide suggestions in our
Should we really be afraid of artificial intelligence?
AI has huge potential to help us understand and manage our world, and especially to get value from the enormous amount of data that we now generate.
Monotonic Constraints with SAS
A monotonic relationship exists when a model’s output increases or stays constant in step with an increase in your model’s inputs. Relationships can be monotonically increasing or decreasing with the distinction based on which direction the input and output travel. A common example is in credit risk where you would expect someone’s risk score to increase with the amount of debt they have relative to their income.
How to save Jupyter notebooks in assorted formats
Learning never stops. When SAS had to change this year’s SAS Global Forum (SASGF) to a virtual event, everyone was disappointed. I am, however, super excited about all of the papers and stream of video releases over the last month (and I encourage you to register for the upcoming live
The chasm between a clever AI model and insight for your organisation
According to Gartner only 50% of clever AI models ever get deployed, put into practise and used to make decisions.
How to estimate the difference between percentiles
I recently read an article that describes ways to compute confidence intervals for the difference in a percentile between two groups. In Eaton, Moore, and MacKenzie (2019), the authors describe a problem in hydrology. The data are the sizes of pebbles (grains) in rivers at two different sites. The authors
Transform word vector tables from long to wide
In natural language processing, word vectors play a key role in making technologies such as machine translation and speech recognition possible. Let's talk about how to transform word vector tables from long to wide in SAS, so we can potentially get sentence vectors to process further.
How do retailers understand the new demand? An interview with Antonio Calvo
In many countries, retail has been right at the centre of the coronavirus storm. Some retailers – those selling essential goods – have been unable or barely able to keep up with demand. Other sectors, notably clothes and luxury goods, have seen a complete drop-off in demand. Some retailers have
Summer Planning for Teens
I wish I had a magic wand! The pandemic would disappear. I’d be back on the SAS campus meeting with folks face-to-face [with a fresh haircut & color] and enjoying the great café food. And I would have plenty of ideas and suggestions to point your teens to for their summer
Operationalising analytics skills development: How ABSA and NWU lead by example
When the technology is deeply embedded into the curriculum, we sometimes describe the resulting programmes as creating an ‘analytics skills factory’.
Data for good: the home edition
“He spends a lot of time wandering around in circles in the backyard,” my wife said to someone on the telephone. That’s true. Our backyard is only about 1/8th of an acre and I have taken to wandering outside and walking around the fence line. Ostensibly, I am checking to
The Kullback–Leibler divergence between continuous probability distributions
In a previous article, I discussed the definition of the Kullback-Leibler (K-L) divergence between two discrete probability distributions. For completeness, this article shows how to compute the Kullback-Leibler divergence between two continuous distributions. When f and g are discrete distributions, the K-L divergence is the sum of f(x)*log(f(x)/g(x)) over all
Using SAS to estimate the link between ozone and asthma (and a neat trick)
While working at the Rutgers Robert Wood Johnson Medical School, I had access to data on over ten million visits to emergency departments in central New Jersey, including ICD-9 (International Classification of Disease – 9th edition) codes along with some patient demographic data. I also had the ozone level from
Combining business expertise and tech advancements to fight fraud
Fraud prevention: Collaboration is one of the traits we have utilized to maximum effect in pretty much everything.
From 9 cows to the future of agtech
The COVID-19 pandemic challenged agriculture and supply chains, but the overarching resilience of agriculture around the world speaks to the industry's efficiency, built-in redundancy and indispensability. In the US, flourishing interactions between government, industry and academic stakeholders underscore how ag represents unity and consilience. And there may be no better
Minimizing the Kullback–Leibler divergence
The Kullback–Leibler divergence is a measure of dissimilarity between two probability distributions. An application in machine learning is to measure how distributions in a parametric family differ from a data distribution. This article shows that if you minimize the Kullback–Leibler divergence over a set of parameters, you can find a
Multi-purpose macro function for getting information about data sets
SAS' Leonid Batkhan describes programming technique for creating multi-purpose SAS® Macro functions.
A closer look at chocolate, coffee and tea
Coffee and tea are the most popular drinks around the world, and of course we all love chocolate. However, special attention should be paid to these crops not only because of their environmental impact but also because of the social injustice issues that are pervasive among these industries.
Using what you have, right where you are
Written by Allison Hines and Kara Roberts To encourage and inspire others to think of ways to give back during this pandemic, we’re sharing stories of how SAS employees are spreading compassion in their local communities. They’ve done everything from making masks and preparing meals for their communities to leading virtual dance classes and collecting supplies for hospitals. Keep reading to learn how our employees are lifting spirits
Digital transformation: What can banks learn from other sectors?
The banking sector sets a benchmark for other industries in many areas. When businesses need to process large volumes of transactions reliably, maintain 24/7 availability, meet complex regulatory requirements, analyse risks or make financial plans, they often aim to follow the same practices and adopt the same technologies that banks
Imposter Syndrome
It can be nice to take a break from COVID-19 topics. In this blog I am writing about something else I find myself discussing often: imposter syndrome. I would say I discuss it most often with young professionals, but I’ve heard from people of all ages that they experience it,
Whole person education has never been more important
You may not have heard the term "whole person education" before but it recognizes the critical intersection of schools and a young person's behavioral health. When I was running youth behavioral health treatment programs, schools were a major part of raising awareness and advocating for help for many kids. I
Forecasting with SAS (free eBook)
Forecasting with SAS®: Special Collection SAS Press has added to its selection of free downloadable eBooks with the new Forecasting with SAS®: Special Collection. From the description: Want to get the most insight out of your data and improve the quality of your forecasts? SAS offers many different solutions to
The Kullback–Leibler divergence between discrete probability distributions
If you have been learning about machine learning or mathematical statistics, you might have heard about the Kullback–Leibler divergence. The Kullback–Leibler divergence is a measure of dissimilarity between two probability distributions. It measures how much one distribution differs from a reference distribution. This article explains the Kullback–Leibler divergence and shows