The DO Loop

Analytics

Find out how analytics, from data mining to cognitive computing, is changing the way we do business

Rick WicklinJune 14, 2021 5

Simulate correlated variables by using the Iman-Conover transformation

Simulating univariate data is relatively easy. Simulating multivariate data is much harder. The main difficulty is to generate variables that have given univariate distributions but also are correlated with each other according to a specified correlation matrix. However, Iman and Conover (1982, "A distribution-free approach to inducing rank correlation among

English

Analytics | Learn SAS

Rick WicklinJune 9, 2021 0

Rank-based scores and tied values

Many nonparametric statistical methods use the ranks of observations to compute distribution-free statistics. In SAS, two procedures that use ranks are PROC NPAR1WAY and PROC CORR. Whereas the SPEARMAN option in PROC CORR (which computes rank correlation) uses only the "raw" tied ranks, PROC NPAR1WAY uses transformations of the ranks,

English

Analytics | Programming Tips

Rick WicklinJune 7, 2021 1

Permutation tests and independent sorting of data

For many univariate statistics (mean, median, standard deviation, etc.), the order of the data is unimportant. If you sort univariate data, the mean and standard deviation do not change. However, you cannot sort an individual variable (independently) if you want to preserve its relationship with other variables. This statement is

English

Analytics | Programming Tips

Rick WicklinJune 1, 2021 10

The Hampel identifier: Robust outlier detection in a time series

It is well known that classical estimates of location and scale (for example, the mean and standard deviation) are influenced by outliers. In the 1960s, '70s, and '80s, researchers such as Tukey, Huber, Hampel, and Rousseeuw advocated analyzing data by using robust statistical estimates such as the median and the

English

Analytics | Programming Tips

Rick WicklinMay 26, 2021 1

The running median as a time series smoother

When data contain outliers, medians estimate the center of the data better than means do. In general, robust estimates of location and sale are preferred over classical moment-based estimates when the data contain outliers or are from a heavy-tailed distribution. Thus, instead of using the mean and standard deviation of

English

Analytics | Learn SAS

Rick WicklinMay 17, 2021 0

Standardized regression coefficients in PROC GLIMMIX

I previously wrote about how to understand standardized regression coefficients in PROC REG in SAS. You can obtain the standardized estimates by using the STB option on the MODEL statement in PROC REG. Several readers have written to ask whether I could write a similar article about the STDCOEF option

English

Analytics | Learn SAS | Programming Tips

Rick WicklinMay 12, 2021 2

Nonstandard ways to standardize variables

You can standardize a numerical variable by subtracting a location parameter from each observation and then dividing by a scale parameter. Often, the parameters depend on the data that you are standardizing. For example, the most common way to standardize a variable is to subtract the sample mean and divide

English

Analytics | Learn SAS

Rick WicklinMay 3, 2021 0

Examples of using the Hoeffding D statistic

A previous article discusses the definition of the Hoeffding D statistic and how to compute it in SAS. The letter D stands for "dependence." Unlike the Pearson correlation, which measures linear relationships, the Hoeffding D statistic tests whether two random variables are independent. Dependent variables have a Hoeffding D statistic

English

Analytics | Programming Tips

Rick WicklinApril 28, 2021 1

Compute the Hoeffding D statistic in SAS

There are many statistics that measure whether two continuous random variables are independent or whether they are related to each other in some way. The most well-known statistic is Pearson's correlation, which is a parametric measure of the linear relationship between two variables. A related measure is Spearman's rank correlation,

English

Analytics | Learn SAS

Rick WicklinApril 21, 2021 0

Compute bivariate ranks

Ranking is a fundamental concept in statistics. Ranks of univariate data are used by statisticians to estimate statistics such as percentiles (quantiles) and empirical distributions. A more advanced use is to compute various rank-based measures of correlation or association between pairs of variables. For example, ranks are used to compute

English

Blogs

Blogs

Analytics