The DO Loop

Analytics

Find out how analytics, from data mining to cognitive computing, is changing the way we do business

Analytics | Learn SAS | Programming Tips

Rick WicklinDecember 9, 2020 0

Horn's method: A simulation-based method for retaining principal components

One purpose of principal component analysis (PCA) is to reduce the number of important variables in a data analysis. Thus, PCA is known as a dimension-reduction algorithm. I have written about four simple rules for deciding how many principal components (PCs) to keep. There are other methods for deciding how

English

Analytics | Data Visualization

Rick WicklinDecember 7, 2020 3

Can you transplant an indoor Christmas tree?

"O Christmas tree, O Christmas tree, how lovely are your branches!" The idealized image of a Christmas tree is a perfectly straight conical tree with lush branches and no bare spots. Although this ideal exists only on Christmas cards, forest researchers are always trying to develop trees that approach the

English

Analytics | Data Visualization | Programming Tips

Rick WicklinDecember 2, 2020 4

How to score a logistic regression model that was not fit by PROC LOGISTIC

A SAS customer asked a great question: "I have parameter estimates for a logistic regression model that I computed by using multiple imputations. How do I use these parameter estimates to score new observations and to visualize the model? PROC LOGISTIC can do the computation I want, but how do

English

Analytics

Rick WicklinNovember 30, 2020 1

Intransitive dice

Most games of skill are transitive. If Player A wins against Player B and Player B wins against Player C, then you expect Player A to win against Player C, should they play. Because of this, you can rank the players: A > B > C Interestingly, not all games

English

Analytics | Data Visualization

Rick WicklinNovember 23, 2020 4

Decile plots in SAS

I previously showed how to create a decile calibration plot for a logistic regression model in SAS. A decile calibration plot (or "decile plot," for short) is used in some fields to visualize agreement between the data and a regression model. It can be used to diagnose an incorrectly specified

English

Analytics | Data Visualization

Predicted probabilities for a logistic regression model

Rick WicklinNovember 18, 2020 0

Create scoring data when regressors are correlated

To help visualize regression models, SAS provides the EFFECTPLOT statement in several regression procedures and in PROC PLM, which is a general-purpose procedure for post-fitting analysis of linear models. When scoring and visualizing a model, it is important to use reasonable combinations of the explanatory variables for the visualization. When

English

Analytics

Monte Carlo distribution of skewness statistic (B=10000, N=100)

Rick WicklinOctober 28, 2020 4

The sample skewness is a biased statistic

The skewness of a distribution indicates whether a distribution is symmetric or not. The Wikipedia article about skewness discusses two common definitions for the sample skewness, including the definition used by SAS. In the middle of the article, you will discover the following sentence: In general, the [estimators]are both biased

English

Analytics | Programming Tips

Graphical comparison of two methods for estimating confidence intervals of eigenvalues of a correlation matrix

Rick WicklinOctober 26, 2020 3

Confidence intervals for eigenvalues of a correlation matrix

A fundamental principle of data analysis is that a statistic is an estimate of a parameter for the population. A statistic is calculated from a random sample. This leads to uncertainty in the estimate: a different random sample would have produced a different statistic. To quantify the uncertainty, SAS procedures

English

Analytics | Programming Tips

Rick WicklinOctober 19, 2020 5

Generate random points in a triangle

How can you efficiently generate N random uniform points in a triangular region of the plane? There is a very cool algorithm (which I call the reflection method) that makes the process easy. I no longer remember where I saw this algorithm, but it is different from the "weighted average"

English

Analytics | Data Visualization

Rick WicklinOctober 14, 2020 1

A continuous band plot for visualizing uncertainty in regression predictions

A previous article discusses the confidence band for the mean predicted value in a regression model. The article shows a "graded confidence band plot," which I saw in Claus O. Wilke's online book, Fundamentals of Data Visualization (Section 16.3). It communicates uncertainty in the predictions. A graded band plot is

English

Blogs

Blogs

Analytics