Recently, I interviewed three SAS customers to understand firsthand how each is using data visualization and analytics in education. In this education analytics series of blog posts, I’ll take you on a journey to learn how each of these customers are turning their data into insights to be a more
The State of Illinois faces an unprecedented budget crisis, with more than $15 billion in unpaid bills. While experts will argue over the exact causes of states' financial struggles, many are pointing to the problem of state leaders avoiding long-term budgetary problems for short-term fixes. Illinois is not alone in
With all the recent talk about some people wanting to move from the US to Canada, I got to wondering how cold, and how far north Canada is. And after a few Google searches, I was surprised to learn that 27 US states are actually farther north than the southernmost point
In this post I describe the important tasks of data preparation, exploration and binning.These three steps enable you to know your data well and build accurate predictive models. First you need to clean your data. Cleaning includes eliminating variables which have uneven spread across the target variable. I give an
This is the fifth post in my series of machine learning best practices. Hyperparameters are the algorithm options one "turns and tunes" when building a learning model. Hyperparameters cannot be learned using that algorithm. So, these parameters need to be assigned before training of the model. A lot of manual
1980년대 후반에만 해도 머신러닝(machine learning)이나 데이터 과학자와 같은 개념은 없었습니다. 대신 통계, 분석, 데이터 마이닝, 데이터 모델링과 같은 단어가 사용됐는데요. 이후 글로벌 기업들은 30년 이상 머신러닝 모델을 연구해 왔으며, 페이스북의 이미지 인식 소프트웨어, 아마존의 음성 비서 알렉사, KT의 인공지능 서비스 기가 지니(GiGA Genie)까지 그 결과들이 연이어 쏟아지고 있죠! 이러한 결실
Recently, I was asked whether SAS can perform a principal component analysis (PCA) that is robust to the presence of outliers in the data. A PCA requires a data matrix, an estimate for the center of the data, and an estimate for the variance/covariance of the variables. Classically, these estimates
If I’m having a tough week, I might feel sorry for myself for a minute or two. But to snap myself out of it, all I have to do is think of the data managers. Those folks have to contend with duplicate records, bad information, data proliferation, accountability, permissions, privacy,
Joyce Norris-Montanari says focus on data quality and governance, privacy and security when providing data on demand.
A SAS customer asked, "I computed the eigenvectors of a matrix in SAS and in another software package. I got different answers? How do I know which answer is correct?" I've been asked variations of this question dozens of times. The answer is usually "both answers are correct." The mathematical
Many years ago I shared a custom task that allows you to view and manage SAS catalogs within SAS Enterprise Guide. As a reminder, a SAS catalog is a special type of SAS file that acts as a container, like a folder, for a variety of content items called catalog
Yesterday I recommended Steve Morlidge’s The Little Book of Beyond Budgeting, for helping to illuminate the troubling usage of business forecasts in the traditional management / budgeting process. Steve reached out to me overnight with some additional points that he shares in this guest blogger post. Steve Morlidge on Forecasts
The dramatic drop in crude oil prices from the highs above $100USD per barrel down to levels around $50 per barrel is one of the drivers behind the industry’s march towards improved operating efficiency. Along the corridors of Independent Oil Companies (IOCs) and National Oil Companies (NOCs), machine learning and
In the spirit of my Forecasting Sharknadoes blog post, I now bring you Sunsquatch! In this blog post, I create a map that helps you find a location where you can see the total eclipse *and* have a chance of seeing Bigfoot (aka, Sasquatch)! But before we get into the nitty-gritty
Is There Something Beyond Budgeting? Forecasting is an integral part of the business planning and budgeting process. Presumably the forecast (which should be an "unbiased best guess" at what is really going to happen in the future) can provide a reasonable foundation upon which the annual budget and operating plans
When I first started teaching JMP classes for SAS Education 10 years ago, I sat in to observe a colleague teaching our introductory JMP data exploration course. The students had lots of questions about using JMP for their own projects, and were excited to learn about all the capabilities of
This is the fourth post in my series of 10 machine learning best practices. It’s common to build models on historical training data and then apply the model to new data to make decisions. This process is called model deployment or scoring. I often hear data scientists say, “It took
There will be a total solar eclipse on August 21, 2017, and the umbra (total shadow) will pass right across the United States! As a data guy, a map guy, and an astronomy fan, this is an opportunity I just couldn't pass up! Follow along as I apply my computer skills
August’s move of the month is the Overhead Squat. It is a multi-joint movement that incorporates a squat as well as significant shoulder mobility and core strength. Although it seems quite simple, the technique takes time to learn, so start with something like a broomstick or PVC pipe to familiarize
Last week I blogged about the broken-stick problem in probability, which reminded me that the broken-stick model is one of the many techniques that have been proposed for choosing the number of principal components to retain during a principal component analysis. Recall that for a principal component analysis (PCA) of
Did you pay your taxes? From stiff penalties to even jail time, the federal government provides plenty of incentive for citizens to pay, but each year nearly one in five Americans do not pay on time. This leads to a more than $450 billion gap in unpaid taxes, creating a
David Loshin recommends enforcing governed standards to help avoid conflicting analytical results.
A lot of tourists flock to North Carolina. We have beaches, wreck diving, and lighthouses. We have the Great Smoky Mountains with whitewater kayaking, colorful fall leaves, and snow skiing. We have hot air balloon festivals & Scottish highland games. Oh, and some of the best barbecue you've ever tasted! But
A SAS user needed to convert a program from MATLAB into the SAS/IML matrix language and asked whether there is a SAS/IML equivalent to the fliplr and flipud functions in MATLAB. These functions flip the columns or rows (respectively) of a matrix; "LR" stands for "left-right" and "UD" stands for
When we talk about consent management for the EU’s General Data Protection Regulation (GDPR), one of the key considerations is “consent for a purpose.” It might have been sufficient in the past to provide a form with a single generic consent check box and store the fact that consent was
IIF Tao Hong Award Dr. Tao Hong is a friend, former SAS colleague, former SAS basketball league opponent, a major contributor to SAS Energy Forecasting software, and now Associate Professor at UNC Charlotte. When he isn't raining three-pointers on me on the basketball court, Tao is Director of the Big
I hate my title, but I’m going to stick with it. In spite of it, I hope I can encourage you to pay attention to the current automation revolution and actively contribute to augmention-fueled innovation. If everyone understands those terms and actively tries to stay on top of how they
I recently saw an interesting PEW study showing the percent of each state's revenue that came from federal funds. They had some pretty nice graphs ... but just like jell-o, there's always room for more graphs, eh! Let's start with the map. Their map had an informative title, a reasonable gradient
I use SAS Enterprise Guide every day, and for a wide variety of tasks. As a result, I have a huge collection of project files (EGP files) and SAS program files. I have always relied on the "recently used" list in the File menu to provide me with quick access
“I do not like this modern technology,” said my father-in-law. “It is making people too lazy. Things are too easy now.” He was referring to my grocery order. I was sitting in his kitchen in Reykjavik, Iceland, the day before my return to the United States. I had just explained
