Rick Wicklin, PhD, is a distinguished researcher in computational statistics at SAS and is a principal developer of SAS/IML software. His areas of expertise include computational statistics, simulation, statistical graphics, and modern methods in statistical data analysis. Rick is author of the books Statistical Programming with SAS/IML Software and Simulating Data with SAS.
Recently, a colleague struggled to find the source of a run-time error happening somewhere within a very large library of SAS IML function modules. Since the error happens at run time, I told my colleague about how to find the location of a run time error by reading the traceback
In some applications, it is useful to permute the rows or the columns of a matrix. A previous article discusses how random permutation of columns (within each row) are useful in constructing permutation tests. This article shows a simpler situation: Permuting the rows of a matrix to change their order.
It is difficult to evaluate high-dimensional integrals. One numerical technique that can be useful is quasi-Monte Carlo integration. In this article, I show how you can generate quasirandom points in SAS and use them to evaluate a definite integral on a compact region. For simplicity, the example in this article
A previous article shows how to convert a positive integer from base 10 to any other arbitrary base. For example, 15 (base 10) = 120 (base 3) because 15 = 1*32 + 2*31 + 0*30. Representing integers is probably familiar to many readers. But did you know that you can
While many applications of Monte Carlo techniques use pseudorandom numbers, some applications that involve integrals are more accurate when you use quasirandom numbers, which, despite their names, are not random but are deterministic sequences of numbers. Many of these sequences are constructed by representing base-10 numbers in a different base.
Many data analysts are familiar with logistic regression, where the response variable, Y, has two observed values, often represented as Y=0 and Y=1. The case Y=0 encodes that an event did not happen. For example, a patient did not experience some disease or did not die. The opposite case (Y=1)
A previous article discusses various ways to overlay a density curve on a histogram in SAS. SAS provides several procedures that handle this task for common univariate probability distributions such as normal, lognormal, and gamma. If you define and use a less common distribution, you can write a GTL template
SAS has several procedures that can fit a probability distribution to data, plot a histogram, and overlay one or more density estimates: PROC UNIVARIATE in Base SAS enables you to overlay parametric density curves from about 20 common continuous probability distributions, such as normal, lognormal, and gamma. It also enables
The power method is a well-known iterative scheme to approximate the largest eigenvalue (in absolute value) of a symmetric matrix. It is useful in practice when you need only the largest eigenvalue and eigenvector of a large matrix. The method requires only matrix-vector multiplication and vector scaling. There is a
When I encounter a new function, I usually graph it to gain intuition about how the function transforms its inputs. Recently, I needed to use the Rayleigh quotient function, which is connected to the estimation of eigenvalues and eigenvectors for symmetric matrices. It has been several years since I last
The new school year had barely started when I got a call from a friend who is an elementary school principal. She told me that every morning she announces the names of students who are celebrating a birthday. "One student noticed that we've already had two days on which no
In probability and statistics, special numbers are used to compute probabilities by counting the number of ways certain events can occur. The most famous are combinations and permutations. Both are used to count the ways to arrange or select items from a set. If a set contains n elements: A
I've often wondered about the logic that the SGPLOT procedure in SAS uses to determine whether a set of graphical overlays will receive identical attributes or different attributes. (Recall that color, size, line style, and marker symbol are all examples of attributes.) I know that when you plot grouped data
In data analysis, sometimes we need to perform a preliminary task before we can analyze data. Often the task needs to be performed only once per session. For example, you might need to download or merge data prior to your analysis. Or you might need to define or load a
SAS supports more than 25 common probability distributions for the PDF, CDF, QUANTILE, and RAND functions. If you need a less-common distribution, you can implement new distributions by using Base SAS (specifically, PROC FCMP) or the SAS/IML language. On the SAS Support Communities, a SAS programmer asked how to implement
A previous article discusses Cohen's d statistic and how to compute it in SAS. For a two-sample independent design, Cohen's d estimates the standardized mean difference (SMD). Because Cohen's d is a biased statistic, the previous article also computes Hedges' g, which is an unbiased estimate of the SMD. Lastly,
What is Cohen's d statistic and how is it used? How can you compute it in SAS? This article gives some history behind Cohen's d statistic. It shows how to compute the statistic in SAS for the difference in means between two independent samples. It shows how to estimate a
SAS provides procedures to fit common probability distributions to sample data. You can use PROC UNIVARIATE in Base SAS or PROC SEVERITY in SAS/ETS software to estimate the distribution parameters for approximately 20 common distributions, including normal, lognormal, beta, gamma, and Weibull. Since there are infinitely many distributions, you may
Many common probability distributions contain terms that increase or decrease quickly, such as the exponential function and factorials. The numerical evaluation of these quantities can result in numerical overflow (or underflow). This is why we often work on the logarithmic scale: on the log-scale, the numerical computations for equations such
Suppose you measure data weekly. According to the ISO standard, weeks are measured in the range 1-53, where most years have 52 weeks, but occasionally there is a "leap week." (The WEEK function in SAS implements the ISO standard to find the week-of-the-year for any date.) The heat map to
Dating can be a challenge. No, I'm not talking about the process of finding a soulmate. I'm talking about managing days, weeks, months, and years in statistical analyses and reports! One challenge is how to number the weeks of the year. Because there are seven days in a week, 52
I follow several data visualization experts on social media. Sometimes, I see a graph that I struggle to interpret. When that happens, I ask myself whether there is a simpler and more effective way to visualize the data. Recently, I saw an example of using a "horizon plot" to visualize
While researching a topic on effect sizes, I learned about a SAS function that is related to noncentrality parameters. I previously wrote an article about the noncentral t distribution, which is one of several well-known distributions that contains an optional noncentrality parameter. I mentioned that the PDF, CDF, and QUANTILE
A previous article discusses a "Catch-22" paradox for fitting nonlinear regression models: You can't estimate the parameters until you fit the model, but you can't fit the model until you provide an initial guess for the parameters! If your initial guess for the parameters is not good enough, the nonlinear
I have previously written about the moment-ratio diagram as a graphical tool for modeling univariate distributions and also as a tool for examining the distribution of the skewness and kurtosis statistics for distributions. The simple moment-ratio (M-R) diagram from my book (Wicklin, 2013, Simulating Data with SAS, Chapter 16) is
This article shows how to classify a set of high-dimensional data into orthants. An orthant is the d-dimensional generalization of a quadrant. For 2-D Euclidean space, there are four quadrants, often labeled by Roman numerals I-IV. The quadrants are open sets that are defined by the signs of each coordinate
This article shows how to use SAS to implement the ISO algorithm to generate (and validate) a Universal Loan Identifier (ULI). A ULI is a long string of numbers and letters that serves as a unique identifier that is used for certain financial transactions. The ISO standard ensures that banks
On social media, a SAS user reported that SAS could not compute the modulo of an extremely large integer. In SAS, the modulo operation is usually performed by using the MOD function, which computes the remainder of dividing an integer, N, by another integer, d. (In symbols, the remainder is
A previous article discusses the Gini-Simpson diversity index and how to compute it in SAS. Suppose you have a sample that contains R classes. (Classes are also called groups or categories.) Intuitively, the sample exhibits "high diversity" if the class sizes are approximately equal. The sample shows "low diversity" if
An article by David Corliss in Amstat News (Corliss D. (2025) "Quantifying Diversity: Calculating the Gini-Simpson Diversity Index") discusses a new statistical measure of diversity that was adopted by the US Census Bureau. The statistic is called the Gini-Simpson diversity index. The Census Bureau has published an article about how
