Advanced Analytics | Analytics | Programming Tips

Kevin ScottMay 22, 2025 0

Detecting lags in nonlinear time series with PROC TSSELECTLAG in SAS Viya

Accurately identifying lag structures between related time series is essential in public health forecasting, particularly during epidemics where delays between infections and hospitalizations affect planning. Using a simulated SEIR model and SAS Viya’s PROC TSSELECTLAG, distance correlation is shown to outperform Pearson correlation by correctly identifying nonlinear lag relationships—such as the true seven-day lag between new infections and hospital admissions.

Read More

English

Advanced Analytics | Machine Learning

Rick WicklinMay 5, 2025 0

Implement a SMOTE simulation algorithm in SAS

A recent article describes the main features of simulation by using the Synthetic Minority Over-sampling Technique (SMOTE). SMOTE was created to oversample from a set of rare events prior to running a machine learning classification algorithm. However, at its heart, the SMOTE algorithm (Chawla et al., 2002) provides a way

Read More

English

Learn SAS | Machine Learning | Programming Tips

Rick WicklinApril 28, 2025 0

The SMOTE method for generating synthetic data

The Synthetic Minority Over-sampling Technique (SMOTE) was created to address class-imbalance problems in machine learning algorithms. The idea is to oversample from the rare events prior to running a machine learning classification algorithm. However, at its heart, the SMOTE algorithm (Chawla et al., 2002) is essentially a way to simulate

Read More

English

Analytics | Programming Tips

Rick WicklinDecember 18, 2024 0

Generate correlation matrices with specified eigenvalues

A previous article discusses how to generate a random covariance matrix with a specified set of (positive) eigenvalues. A SAS programmer asked whether it is possible to produce a correlation matrix that has a specified set of eigenvalues. After discussing the problem with a friend, I am happy to report

Read More

English

Advanced Analytics | Programming Tips

Rick WicklinDecember 9, 2024 0

Latin hypercube sampling in SAS

While researching the topic of Latin hypercube sampling (LHS), I read an article by Emily Gao (2019) that shows how to use PROC IML in SAS to perform the algorithm. It is possible to simplify Gao's implementation of Latin hypercube sampling in SAS while also making the computation more efficient.

Read More

English

Analytics | Programming Tips

Rick WicklinDecember 2, 2024 0

A historical method of generating random normal variates

Decades ago, it was a challenge to generate (pseudo-) random numbers that had good statistical properties. The proliferation of desktop computers in the 1980s and '90s led to many advances in computational mathematics, including better ways to generate pseudorandom variates from a wide range of probability distributions. (For brevity, I

Read More

English

Advanced Analytics | Learn SAS

Rick WicklinNovember 11, 2024 0

Introducing PROC SIMSYSTEM in SAS Viya

When the SAS Global Forum 2020 conference was cancelled by the global COVID-19 pandemic, I felt sorry for the customers and colleagues who had spent months preparing their presentations. One presentation I especially wanted to attend was by Bucky Ransdell and Randy Tobias: "Introducing PROC SIMSYSTEM for Systematic Nonnormal Simulation".

Read More

English

Learn SAS | Programming Tips

Rick WicklinNovember 4, 2024 0

Levy flight and vectorizing a simulation in SAS

A previous article shows a simulation of two different models of a foraging animal. The first model is a random walk, which assumes that the animal chooses a random direction, then takes a step that is distributed according to a Gaussian random variable. In the second model, the animal again

Read More

English

Analytics | Learn SAS | Programming Tips

Rick WicklinOctober 7, 2024 0

The three-sigma rule

A remarkable result in probability theory is the "three-sigma rule," which is a generic name for theorems that bound the probability that a univariate random variable will appear near the center of its distribution. This article discusses the familiar three-sigma rule for the normal distribution, a less-familiar rule for unimodal

Read More

English

Analytics | Learn SAS

Rick WicklinAugust 7, 2024 0

Simulate data from a Poisson regression model

This article shows how to simulate data from a Poisson regression model, including how to account for an offset variable. If you are not familiar with how to run a Poisson regression in SAS, see the article "Poisson regression in SAS." A Poisson regression model is a specific type of

Read More

English

Analytics | Machine Learning

Rick WicklinJuly 31, 2024 0

Fit, simulate, fit: How models can collapse after generations of recursive fitting

An article published in Nature has the intriguing title, "AI models collapse when trained on recursively generated data." (Shumailov, et al., 2024). The article is quite readable, but I also recommend a less technical overview of the result: "AI models fed AI-generated data quickly spew nonsense" (Gibney, 2024). The Gibney

Read More

English

Analytics | Data Visualization | Programming Tips

Rick WicklinJune 10, 2024 0

The distribution of the R-square statistic

A SAS analyst ran a linear regression model and obtained an R-square statistic for the fit. However, he wanted a confidence interval, so he posted a question to a discussion forum asking how to obtain a confidence interval for the R-square parameter. Someone suggested a formula from a textbook (Cohen,

Read More

English

Analytics | Learn SAS

Rick WicklinMay 20, 2024 0

On the correctness of a discrete simulation

After writing a program that simulates data, it is important to check that the statistical properties of the simulated (synthetic) data match the properties of the model. As a first step, you can generate a large random sample from the model distribution and compare the sample statistics to the expected

Read More

English

Analytics | Programming Tips

Rick WicklinMay 13, 2024 0

The distribution of p-values under the null hypothesis

A SAS statistical programmer recently asked a theoretical question about statistics. "I've read that 'p-values are uniformly distributed under the null hypothesis,'" he began, "but what does that mean in practice? Is it important?" I think data simulation is a great way to discuss the conditions for which p-values are

Read More

English

Learn SAS | Programming Tips

Rick WicklinFebruary 19, 2024 0

The linear distribution on an interval

In a recent Monte Carlo project, I needed to simulate numbers on an interval by using a continuous linear probability density function (PDF). An example is shown to the right. In this example, the linear density function is decreasing on the interval, but the function could also be constant or

Read More

English

Analytics | Learn SAS

Rick WicklinJanuary 22, 2024 0

Angles vs slopes: The statistics of steepness

There are two popular ways to express the steepness of a line or ray. The most-often used mathematical definition is from high-school math where the slope is defined as "rise over run." A second way is to report the angle of inclination to the horizontal, as introduced in basic trigonometry.

Read More

English

Learn SAS | Programming Tips

Rick WicklinJanuary 15, 2024 0

Simulate correlated continuous and discrete variables

Statistical software provides methods to simulate independent random variates from continuous and discrete distributions. For example, in the SAS DATA step, you can use the RAND function to simulate variates from continuous distributions (such as the normal or lognormal distributions) or from discrete distributions (such as the Bernoulli or Poisson).

Read More

English

Advanced Analytics

Harry SnartOctober 13, 2023 0

Simulating a rugby tournament with SAS® Viya®

The world’s largest rugby tournament returns for the knockout stages. This blog post explores how probability and simulation can be used to predict likely winners in each of the knockout stages. Team sports are dynamic, time-varying and complex topics to model. When modeling regular competitions, such as domestic leagues, it

Read More

English

Sports & Entertainment

Analytics | Learn SAS | Programming Tips

Rick WicklinSeptember 6, 2023 0

Model data from published summary statistics

There are many ways to model a set of raw data by using a continuous probability distribution. It can be challenging, however, to choose the distribution that best models the data. Are the data normal? Lognormal? Is there a theoretical reason to prefer one distribution over another? The SAS has

Read More

English

Learn SAS | Programming Tips

Rick WicklinAugust 30, 2023 0

Simulate the use of personal checks in the US

Does anyone write paper checks anymore? According to researchers at the Federal Reserve Bank of Atlanta (Greene, et al., 2020), the use of paper checks has declined 63% among US consumers since the year 2000. The researchers surveyed more than 3,000 consumers in 2017-2018 and discovered that only 7% of

Read More

English

Analytics | Data Visualization | Programming Tips

Rick WicklinAugust 28, 2023 0

Generate random uniform points in an ellipse

I have previously written about how to efficiently generate points uniformly at random inside a sphere (often called a ball by mathematicians). The method uses a mathematical fact from multivariate statistics: If X is drawn from the uncorrelated multivariate normal distribution in dimensiond, then S = r*X / ||X|| has

Read More

English

Learn SAS | Programming Tips

Rick WicklinAugust 7, 2023 0

Construct an envelope function for the acceptance-rejection method

The acceptance-rejection method (sometimes called rejection sampling) is a method that enables you to generate a random sample from an arbitrary distribution by using only the probability density function (PDF). This is in contrast to the inverse CDF method, which uses the cumulative distribution function (CDF) to generate a random

Read More

English

Learn SAS | Programming Tips

Rick WicklinJuly 10, 2023 0

Simulate from a Markov model

A previous article shows an example of a Markov chain model and computes the probability that the system ends up in a terminal state (called an absorbing state). As explained previously, you can often compute exact probabilities for questions about Markov chains. Nevertheless, it can be useful to know how

Read More

English

Analytics | Learn SAS | Programming Tips

Rick WicklinApril 17, 2023 0

Should you use the Wald confidence interval for a binomial proportion?

The "Teacher’s Corner" of The American Statistician enables statisticians to discuss topics that are relevant to teaching and learning statistics. Sometimes, the articles have practical relevance, too. Andersson (2023) "The Wald Confidence Interval for a Binomial p as an Illuminating 'Bad' Example," is intended for professors and masters-level students in

Read More

English

Analytics | Learn SAS | Programming Tips

Rick WicklinMarch 13, 2023 0

Use the metalog distribution in SAS

A previous article describes the metalog distribution (Keelin, 2016). The metalog distribution is a flexible family of distributions that can model a wide range of shapes for data distributions. The metalog system can model bounded, semibounded, and unbounded continuous distributions. This article shows how to use the metalog distribution in

Read More

English

Analytics | Learn SAS | Programming Tips

Rick WicklinFebruary 20, 2023 0

Simulate from a bounded distribution that has a specified mean

A SAS programmer asked for help to simulate data from a distribution that has certain properties. The distribution must be supported on the interval [a, b] and have a specified mean, μ, where a < μ < b. It turns out that there are infinitely many distributions that satisfy these

Read More

English

Data Visualization | Learn SAS | Programming Tips

Rick WicklinFebruary 8, 2023 0

A random walk inside a heart

SAS programmers love to make special graphs for Valentine's Day. In fact, there is a long history of heart-shaped graphs and love-inspired programs written in SAS! Last year, I added to the collection by showing how a ball bounces on a heart-shaped billiards table. This year, I create a similar

Read More

English

Analytics | Data Visualization | Learn SAS

Rick WicklinJanuary 18, 2023 0

Visualize how parameters in a binary logistic regression model affect the probability of the event

A previous article shows that you can use the Intercept parameter to control the ratio of events to nonevents in a simulation of data from a logistic regression model. If you decrease the intercept parameter, the probability of the event decreases; if you increase the intercept parameter, the probability of

Read More

English

Analytics | Programming Tips

Rick WicklinJanuary 16, 2023 0

Simulate data from a logistic regression model: How the intercept parameter affects the probability of the event

This article shows that you can use the intercept parameter to control the probability of the event in a simulation study that involves a binary logistic regression model. For simplicity, I will simulate data from a logistic regression model that involves only one explanatory variable, but the main idea applies

Read More

English

Advanced Analytics

Bahar BillerDecember 14, 2022 0

Accelerating what-if analysis with machine learning

SAS' Bahar Biller expounds on the idea that stochastic simulations are large-data generation programs for highly complex and dynamic stochastic systems.

Read More

English