The DO Loop

Advanced Analytics

Move beyond spreadsheets to data mining, forecasting, optimization – and more

Rick WicklinJune 12, 2019 5

Leave-one-out statistics and a formula to update a matrix inverse

For linear regression models, there is a class of statistics that I call deletion diagnostics or leave-one-out statistics. These observation-wise statistics address the question, "If I delete the i_th observation and refit the model, what happens to the statistics for the model?" For example: The PRESS statistic is similar to

English

Advanced Analytics | Machine Learning

Rick WicklinFebruary 6, 2019 0

Feature generation and correlations among features in machine learning

Feature generation (also known as feature creation) is the process of creating new features to use for training machine learning models. This article focuses on regression models. The new features (which statisticians call variables) are typically nonlinear transformations of existing variables or combinations of two or more existing variables. This

English

Advanced Analytics | Machine Learning

Rick WicklinFebruary 4, 2019 5

Model selection with PROC GLMSELECT

I previously discussed how you can use validation data to choose between a set of competing regression models. In that article, I manually evaluated seven models for a continuous response on the training data and manually chose the model that gave the best predictions for the validation data. Fortunately, SAS

English

Advanced Analytics | Machine Learning

Rick WicklinJanuary 30, 2019 3

Model assessment and selection in machine learning

Machine learning differs from classical statistics in the way it assesses and compares competing models. In classical statistics, you use all the data to fit each model. You choose between models by using a statistic (such as AIC, AICC, SBC, ...) that measures both the goodness of fit and the

English

Advanced Analytics

Sample from mixture distribution showing sample median

Rick WicklinFebruary 21, 2018 2

A Monte Carlo algorithm to estimate a median

This article describes and implements a fast algorithm that estimates a median for very large samples. The traditional median estimate sorts a sample of size N and returns the middle value (when N is odd). The algorithm in this article uses Monte Carlo techniques to estimate the median much faster.

English

Advanced Analytics | Learn SAS | Programming Tips

Simulate clustered data from a Gaussian mixture distribution

Rick WicklinSeptember 13, 2017 7

Simulate multivariate clusters in SAS

This article shows how to simulate data from a mixture of multivariate normal distributions, which is also called a Gaussian mixture. You can use this simulation to generate clustered data. The adjacent graph shows three clusters, each simulated from a four-dimensional normal distribution. Each cluster has its own within-cluster covariance,

English

Advanced Analytics

Rick WicklinAugust 30, 2017 1

The singular value decomposition and low-rank approximations

A previous article discussed the mathematical properties of the singular value decomposition (SVD) and showed how to use the SVD subroutine in SAS/IML software. This article uses the SVD to construct a low-rank approximation to an image. Applications include image compression and denoising an image. Construct a grayscale image The

English

Advanced Analytics

Geometric interpretation of the singular value decomposition (SVD) as the product of a rotation/reflection, followed by a scaling, followed by another rotation/reflection.

Rick WicklinAugust 28, 2017 8

The singular value decomposition: A fundamental technique in multivariate data analysis

The singular value decomposition (SVD) could be called the "billion-dollar algorithm" since it provides the mathematical basis for many modern algorithms in data science, including text mining, recommender systems (think Netflix and Amazon), image processing, and classification problems. Although the SVD was mathematically discovered in the late 1800s, computers have

Advanced Analytics

Classical and robust principal component scores for crime data, computed in SAS

Rick WicklinAugust 9, 2017 11

Robust principal component analysis in SAS

Recently, I was asked whether SAS can perform a principal component analysis (PCA) that is robust to the presence of outliers in the data. A PCA requires a data matrix, an estimate for the center of the data, and an estimate for the variance/covariance of the variables. Classically, these estimates

English

Advanced Analytics | Data Visualization

Prediction regions for a classification problem with two outcomes. Graph created in SAS.

Rick WicklinJuly 17, 2017 0

3 ways to visualize prediction regions for classification problems

An important problem in machine learning is the "classification problem." In this supervised learning problem, you build a statistical model that predicts a set of categorical outcomes (responses) based on a set of input features (explanatory variables). You do this by training the model on data for which the outcomes

English

Blogs

Blogs

Advanced Analytics