The DO Loop

Analytics

Find out how analytics, from data mining to cognitive computing, is changing the way we do business

Analytics

Rick WicklinNovember 30, 2020 1

Intransitive dice

Most games of skill are transitive. If Player A wins against Player B and Player B wins against Player C, then you expect Player A to win against Player C, should they play. Because of this, you can rank the players: A > B > C Interestingly, not all games

English

Analytics | Data Visualization

Rick WicklinNovember 23, 2020 4

Decile plots in SAS

I previously showed how to create a decile calibration plot for a logistic regression model in SAS. A decile calibration plot (or "decile plot," for short) is used in some fields to visualize agreement between the data and a regression model. It can be used to diagnose an incorrectly specified

English

Analytics | Data Visualization

Predicted probabilities for a logistic regression model

Rick WicklinNovember 18, 2020 0

Create scoring data when regressors are correlated

To help visualize regression models, SAS provides the EFFECTPLOT statement in several regression procedures and in PROC PLM, which is a general-purpose procedure for post-fitting analysis of linear models. When scoring and visualizing a model, it is important to use reasonable combinations of the explanatory variables for the visualization. When

English

Analytics

Monte Carlo distribution of skewness statistic (B=10000, N=100)

Rick WicklinOctober 28, 2020 4

The sample skewness is a biased statistic

The skewness of a distribution indicates whether a distribution is symmetric or not. The Wikipedia article about skewness discusses two common definitions for the sample skewness, including the definition used by SAS. In the middle of the article, you will discover the following sentence: In general, the [estimators]are both biased

English

Analytics | Programming Tips

Graphical comparison of two methods for estimating confidence intervals of eigenvalues of a correlation matrix

Rick WicklinOctober 26, 2020 3

Confidence intervals for eigenvalues of a correlation matrix

A fundamental principle of data analysis is that a statistic is an estimate of a parameter for the population. A statistic is calculated from a random sample. This leads to uncertainty in the estimate: a different random sample would have produced a different statistic. To quantify the uncertainty, SAS procedures

English

Analytics | Programming Tips

Rick WicklinOctober 19, 2020 5

Generate random points in a triangle

How can you efficiently generate N random uniform points in a triangular region of the plane? There is a very cool algorithm (which I call the reflection method) that makes the process easy. I no longer remember where I saw this algorithm, but it is different from the "weighted average"

English

Analytics | Data Visualization

Rick WicklinOctober 14, 2020 1

A continuous band plot for visualizing uncertainty in regression predictions

A previous article discusses the confidence band for the mean predicted value in a regression model. The article shows a "graded confidence band plot," which I saw in Claus O. Wilke's online book, Fundamentals of Data Visualization (Section 16.3). It communicates uncertainty in the predictions. A graded band plot is

English

Analytics | Data Visualization

Rick WicklinOctober 12, 2020 1

Visualize uncertainty in regression predictions

You've probably seen many graphs that are similar to the one at the right. This plot shows a regression line overlaid on a scatter plot of some data. Given a value for the independent variable (x), the regression line gives the best prediction for the mean of the response variable

English

Analytics | Programming Tips

Rick WicklinOctober 7, 2020 0

The Poisson-binomial distribution for hundreds of parameters

A previous article shows how to use a recursive formula to compute exact probabilities for the Poisson-binomial distribution. The recursive formula is an O(N2) computation, where N is the number of parameters for the Poisson-binomial (PB) distribution. If you have a distribution that has hundreds (or even thousands) of parameters,

English

Analytics | Programming Tips

Rick WicklinSeptember 28, 2020 0

The Poisson-binomial distribution

The Poisson-binomial distribution is a generalization of the binomial distribution. For the binomial distribution, you carry out N independent and identical Bernoulli trials. Each trial has a probability, p, of success. The total number of successes, which can be between 0 and N, is a binomial random variable. The distribution

English

Blogs

Blogs

Analytics