When you implement a statistical algorithm in a vector-matrix language such as SAS/IML, R, or MATLAB, you should measure the performance of your implementation, which means that you should time how long a program takes to analyze data of varying sizes and characteristics. There are some general tips that can
SAS programmers have high expectations for their coding environment, and why shouldn't they? Companies have a huge investment in their SAS code base, and it's important to have tools that help you understand that code and track changes over time. Few things are more satisfying as a SAS program that
The revised EU Payment Services Directive (PSD2) has been a focal point for the financial services industry over the last couple of years and its adoption is set to revolutionise the payment ecosystem in Europe. New entrants, innovative technologies and increased regulation are already posing major challenges to traditional banks
Robots - everyone has probably been fascinated by the idea of robots at one time or another. From the early science fiction robots (such as Klaatu's robot Gort) to the mid-1980s movie robots (like Johnny 5), they have been portrayed in many different ways in fiction. These days, with the
Analytics-driven forecasting means more than measuring trend and seasonality. It includes all categories of methods (e.g. exponential smoothing, dynamic regression, ARIMA, ARIMA(X), unobserved component models, and more), including artificial intelligence, but not necessarily deep learning algorithms. That said, deep learning algorithms like neural networks can also be used for demand forecasting,
This question has been asked repeatedly for decades by anyone facing a new system. That system might be a new product, a new piece of equipment, a new process, or anything really that is new and not well understood. Ultimately, you might need to change this system but first you
I frequently get asked about my favorite book on a particular topic, how to find free SAS learning materials online, how to get help with SAS issues, etc. So I thought I'd share it with you here! Last updated 2/25/2020 Getting Started Resources New to SAS? Here's your SAS Starter
Visualizing the correlations between variables often provides insight into the relationships between variables. I've previously written about how to use a heat map to visualize a correlation matrix in SAS/IML, and Chris Hemedinger showed how to use Base SAS to visualize correlations between variables. Recently a SAS programmer asked how
If you know me, you probably know that I spend a lot of time on the water. I like speed paddling (dragon boat, outrigger canoe, surfski, and racing SUP), and I also have a big pontoon boat at Jordan Lake where I try to go fishing at least once a
Veterans transitioning back to civilian life face a number of challenges, many of which involve working in a non-military environment. The Honor Foundation helps Navy SEALs and other current and transitioning members of the U.S. Special Operations community enter civilian life. Last week, a group of former special operators visited several
In an increasingly complicated operating environment, the global oil and gas industry struggles to optimize their asset portfolio. Producers combat severe challenges inherent to aging facilities in the context of pricing volatility. Change is the new constant, as the cyclical nature of the oil and gas industry means that producers
Phil Simon shares his thoughts on this simple yet often-overlooked question.
With North Korea's growing missile capabilities in the news lately, I thought it would be interesting to create a map showing how far (or close) they are from other parts of the world. I first did a few searches on the Web, to see what maps are already out there.
When someone refers to the correlation between two variables, they are probably referring to the Pearson correlation, which is the standard statistic that is taught in elementary statistics courses. Elementary courses do not usually mention that there are other measures of correlation. Why would anyone want a different estimate of
Recently, I interviewed three SAS customers to understand firsthand how each is using data visualization and analytics in education. In this education analytics series of blog posts, I’ll take you on a journey to learn how each of these customers are turning their data into insights to be a more
The State of Illinois faces an unprecedented budget crisis, with more than $15 billion in unpaid bills. While experts will argue over the exact causes of states' financial struggles, many are pointing to the problem of state leaders avoiding long-term budgetary problems for short-term fixes. Illinois is not alone in
With all the recent talk about some people wanting to move from the US to Canada, I got to wondering how cold, and how far north Canada is. And after a few Google searches, I was surprised to learn that 27 US states are actually farther north than the southernmost point
In this post I describe the important tasks of data preparation, exploration and binning.These three steps enable you to know your data well and build accurate predictive models. First you need to clean your data. Cleaning includes eliminating variables which have uneven spread across the target variable. I give an
This is the fifth post in my series of machine learning best practices. Hyperparameters are the algorithm options one "turns and tunes" when building a learning model. Hyperparameters cannot be learned using that algorithm. So, these parameters need to be assigned before training of the model. A lot of manual
1980년대 후반에만 해도 머신러닝(machine learning)이나 데이터 과학자와 같은 개념은 없었습니다. 대신 통계, 분석, 데이터 마이닝, 데이터 모델링과 같은 단어가 사용됐는데요. 이후 글로벌 기업들은 30년 이상 머신러닝 모델을 연구해 왔으며, 페이스북의 이미지 인식 소프트웨어, 아마존의 음성 비서 알렉사, KT의 인공지능 서비스 기가 지니(GiGA Genie)까지 그 결과들이 연이어 쏟아지고 있죠! 이러한 결실
Recently, I was asked whether SAS can perform a principal component analysis (PCA) that is robust to the presence of outliers in the data. A PCA requires a data matrix, an estimate for the center of the data, and an estimate for the variance/covariance of the variables. Classically, these estimates
If I’m having a tough week, I might feel sorry for myself for a minute or two. But to snap myself out of it, all I have to do is think of the data managers. Those folks have to contend with duplicate records, bad information, data proliferation, accountability, permissions, privacy,
Joyce Norris-Montanari says focus on data quality and governance, privacy and security when providing data on demand.
A SAS customer asked, "I computed the eigenvectors of a matrix in SAS and in another software package. I got different answers? How do I know which answer is correct?" I've been asked variations of this question dozens of times. The answer is usually "both answers are correct." The mathematical
Many years ago I shared a custom task that allows you to view and manage SAS catalogs within SAS Enterprise Guide. As a reminder, a SAS catalog is a special type of SAS file that acts as a container, like a folder, for a variety of content items called catalog
Yesterday I recommended Steve Morlidge’s The Little Book of Beyond Budgeting, for helping to illuminate the troubling usage of business forecasts in the traditional management / budgeting process. Steve reached out to me overnight with some additional points that he shares in this guest blogger post. Steve Morlidge on Forecasts
The dramatic drop in crude oil prices from the highs above $100USD per barrel down to levels around $50 per barrel is one of the drivers behind the industry’s march towards improved operating efficiency. Along the corridors of Independent Oil Companies (IOCs) and National Oil Companies (NOCs), machine learning and
In the spirit of my Forecasting Sharknadoes blog post, I now bring you Sunsquatch! In this blog post, I create a map that helps you find a location where you can see the total eclipse *and* have a chance of seeing Bigfoot (aka, Sasquatch)! But before we get into the nitty-gritty
Is There Something Beyond Budgeting? Forecasting is an integral part of the business planning and budgeting process. Presumably the forecast (which should be an "unbiased best guess" at what is really going to happen in the future) can provide a reasonable foundation upon which the annual budget and operating plans
When I first started teaching JMP classes for SAS Education 10 years ago, I sat in to observe a colleague teaching our introductory JMP data exploration course. The students had lots of questions about using JMP for their own projects, and were excited to learn about all the capabilities of
