The DO Loop

Analytics

Find out how analytics, from data mining to cognitive computing, is changing the way we do business

Analytics | Learn SAS

Rick WicklinNovember 18, 2024 0

The correlation between two sets of variables

In a correlation analysis, it is common to consider the correlations between all pairs of numerical variables. That is, if there are k numerical variables, most people examine the complete k x k matrix of correlations. This matrix is symmetric and has 1s on the diagonal, so more than half of the

English

Analytics | Programming Tips

Rick WicklinOctober 21, 2024 1

The correlogram: Visualize correlations by fitting angles

A common way to visualize the sample correlations between many numeric variables is to display a heat map that shows the Pearson correlation for each pair of variables, as shown in the image to the right. The correlation is a number in the range [-1, 1], where -1 indicated perfect

English

Analytics | Learn SAS | Programming Tips

Rick WicklinOctober 7, 2024 0

The three-sigma rule

A remarkable result in probability theory is the "three-sigma rule," which is a generic name for theorems that bound the probability that a univariate random variable will appear near the center of its distribution. This article discusses the familiar three-sigma rule for the normal distribution, a less-familiar rule for unimodal

English

Analytics | Learn SAS

Rick WicklinAugust 7, 2024 0

Simulate data from a Poisson regression model

This article shows how to simulate data from a Poisson regression model, including how to account for an offset variable. If you are not familiar with how to run a Poisson regression in SAS, see the article "Poisson regression in SAS." A Poisson regression model is a specific type of

English

Analytics | Learn SAS

Rick WicklinAugust 5, 2024 5

Poisson regression in SAS

This article demonstrates how to use PROC GENMOD to perform a Poisson regression in SAS. There are different examples in the SAS documentation and in conference papers, but I chose this example because it uses two categorical explanatory variables. Therefore, the Poisson regression can be visualized by using a contingency

English

Analytics | Machine Learning

Rick WicklinJuly 31, 2024 0

Fit, simulate, fit: How models can collapse after generations of recursive fitting

An article published in Nature has the intriguing title, "AI models collapse when trained on recursively generated data." (Shumailov, et al., 2024). The article is quite readable, but I also recommend a less technical overview of the result: "AI models fed AI-generated data quickly spew nonsense" (Gibney, 2024). The Gibney

English

Analytics | Programming Tips

Rick WicklinJuly 29, 2024 0

A geometric solution to isotonic regression

A previous article shows that you can run a simple (one-variable) isotonic regression by using a quadratic programming (QP) formulation. While I was reading a book about computational geometry, I learned that there is a connection between isotonic regression and the convex hull of a certain set of points. Whaaaaat?

English

Analytics | Learn SAS | Programming Tips

Rick WicklinJuly 15, 2024 7

Isotonic regression: An application of quadratic optimization

Isotonic regression (also called monotonic regression) is a type of regression model that assumes that the response variable is a monotonic function of the explanatory variable(s). The model can be nondecreasing or nonincreasing. Certain physical and biological processes can be analyzed by using an isotonic regression model. For example, a

English

Analytics | Data Visualization

Rick WicklinJune 19, 2024 0

Scale a density curve to match a histogram

This article discusses how to scale a probability density curve so that it fits appropriately on a histogram, as shown in the graph to the right. By definition, a probability density curve is scaled so that the area under the curve equals 1. However, a histogram might show counts or

English

Analytics | Data Visualization | Learn SAS

Rick WicklinJune 17, 2024 5

A bootstrap confidence interval for an R-square statistic

A previous article discusses a formula for a confidence interval for R-square in a linear regression model (Olkin and Finn (1995) "Correlations redux", Psychological Bulletin) The formula is useful for large data sets, but should be used with caution for small samples. At the end of the previous article, I

English

Blogs

Blogs

Analytics