The DO Loop

Analytics

Find out how analytics, from data mining to cognitive computing, is changing the way we do business

Analytics

Rick WicklinJuly 22, 2019 8

Extreme values: What is an extreme value for normally distributed data?

Is 4 an extreme value for the standard normal distribution? In high school, students learn the famous 68-95-99.7 rule, which is a way to remember that 99.7 percent of random observation from a normal distribution are within three standard deviations from the mean. For the standard normal distribution, the probability

English

Analytics | Learn SAS | Programming Tips

Rick WicklinJune 26, 2019 1

Jump-start PROC LOGISTIC by using parameter estimates from PROC HPLOGISTIC

SAS/STAT software contains a number of so-called HP procedures for training and evaluating predictive models. ("HP" stands for "high performance.") A popular HP procedure is HPLOGISTIC, which enables you to fit logistic models on Big Data. A goal of the HP procedures is to fit models quickly. Inferential statistics such

English

Analytics | Data Visualization | Learn SAS

Rick WicklinJune 24, 2019 0

Add loess smoothers to residual plots

When fitting a least squares regression model to data, it is often useful to create diagnostic plots of the residuals versus the explanatory variables. If the model fits the data well, the plots of the residuals should not display any patterns. Systematic patterns can indicate that you need to include

English

Analytics | Data Visualization | Learn SAS

Rick WicklinJune 19, 2019 1

Influential observations in a linear regression model: The DFFITS and Cook's D statistics

A previous article describes the DFBETAS statistics for detecting influential observations, where "influential" means that if you delete the observation and refit the model, the estimates for the regression coefficients change substantially. Of course, there are other statistics that you could use to measure influence. Two popular ones are the

English

Analytics | Data Visualization | Learn SAS

Rick WicklinJune 17, 2019 11

Influential observations in a linear regression model: The DFBETAS statistics

My article about deletion diagnostics investigated how influential an observation is to a least squares regression model. In other words, if you delete the i_th observation and refit the model, what happens to the statistics for the model? SAS regression procedures provide many tables and graphs that enable you to

English

Analytics | Data Visualization | Learn SAS

Rick WicklinMay 30, 2019 10

Visualize interaction effects in regression models

Knowing how to visualize a regression model is a valuable skill. A good visualization can help you to interpret a model and understand how its predictions depend on explanatory factors in the model. Visualization is especially important in understanding interactions between factors. Recently I read about work by Jacob A.

English

Analytics | Programming Tips

Rick WicklinMay 28, 2019 2

The Theil-Sen robust estimator for simple linear regression

Modern statistical software provides many options for computing robust statistics. For example, SAS can compute robust univariate statistics by using PROC UNIVARIATE, robust linear regression by using PROC ROBUSTREG, and robust multivariate statistics such as robust principal component analysis. Much of the research on robust regression was conducted in the

English

Analytics | Data Visualization

Rick WicklinMay 22, 2019 1

Gershgorin discs and the location of eigenvalues

The eigenvalues of a matrix are not easy to compute. It is remarkable, therefore, that with relatively simple mental arithmetic, you can obtain bounds for the eigenvalues of a matrix of any size. The bounds are provided by using a marvelous mathematical result known as Gershgorin's Disc Theorem. For certain

English

Analytics | Programming Tips

Rick WicklinMay 20, 2019 4

Critical values of the Kolmogorov-Smirnov test

Recently I wrote about how to compute the Kolmogorov D statistic, which is used to determine whether a sample has a particular distribution. One of the beautiful facts about modern computational statistics is that if you can compute a statistic, you can use simulation to estimate the sampling distribution of

English

Analytics | Learn SAS

Rick WicklinMay 15, 2019 2

What is Kolmogorov's D statistic?

Have you ever run a statistical test to determine whether data are normally distributed? If so, you have probably used Kolmogorov's D statistic. Kolmogorov's D statistic (also called the Kolmogorov-Smirnov statistic) enables you to test whether the empirical distribution of data is different than a reference distribution. The reference distribution

English

Blogs

Blogs

Analytics