Advanced Analytics Archives

Advanced Analytics

Move beyond spreadsheets to data mining, forecasting, optimization – and more

Advanced Analytics | Data Visualization | Programming Tips

Rick WicklinMay 17, 2023 1

A previous article defines the silhouette statistic (Rousseeuw, 1987) and shows how to use it to identify observations in a cluster analysis that are potentially misclassified. The article provides many graphs, including the silhouette plot, which is a bar chart or histogram that displays the distribution of the silhouette statistic

English

Advanced Analytics

Rick WicklinJuly 5, 2021 18

An introduction to simulating correlated data by using copulas

Do you know what a copula is? It is a popular way to simulate multivariate correlated data. The literature for copulas is mathematically formidable, but this article provides an intuitive introduction to copulas by describing the geometry of the transformations that are involved in the simulation process. Although there are

Advanced Analytics | Learn SAS

Rick WicklinSeptember 14, 2020 0

Video: How to Write a Custom Parallel Program in SAS Viya

My 2020 SAS Global Forum paper was about how to write custom parallel programs by using the iml action in SAS Viya 3.5. My conference presentation was canceled because of the coronavirus pandemic, but I recently recorded a 15-minute video that summarizes the main ideas in the paper. One of

English

Advanced Analytics | Learn SAS | Programming Tips

Rick WicklinAugust 17, 2020 2

Estimate a power curve in parallel in SAS Viya

I recently showed how to use simulation to estimate the power of a statistical hypothesis test. The example (a two-sample t test for the difference of means) is a simple SAS/IML module that is very fast. Fast is good because often you want to perform a sequence of simulations over

English

Advanced Analytics | Machine Learning | Programming Tips

Rick WicklinJuly 23, 2020 11

Fit a multivariate Gaussian mixture model by using the expectation-maximization (EM) algorithm

Last month a SAS programmer asked how to fit a multivariate Gaussian mixture model in SAS. For univariate data, you can use the FMM Procedure, which fits a large variety of finite mixture models. If your company is using SAS Viya, you can use the MBC or GMM procedures, which

English

Advanced Analytics | Programming Tips

Rick WicklinJuly 15, 2020 1

How to evaluate the multivariate normal log likelihood

The multivariate normal distribution is used frequently in multivariate statistics and machine learning. In many applications, you need to evaluate the log-likelihood function in order to compare how well different models fit the data. The log-likelihood for a vector x is the natural logarithm of the multivariate normal (MVN) density

English

Advanced Analytics | Learn SAS | Programming Tips

Architecture of an MPP session in SAS Viya. The client calls an action, which can use multiple nodes and threads.

Rick WicklinJune 17, 2020 0

Write a CAS data table by using the iml action

A previous article shows how to use the iml action to read a CAS data table into an IML matrix. This article shows how to write a CAS table from data in an IML matrix. You can read an overview of the iml action, which was introduced in SAS Viya

English

Advanced Analytics | Learn SAS | Programming Tips

Rick WicklinJune 15, 2020 0

Read a CAS data table by using the iml action

A previous article compares a SAS/IML program that runs in PROC IML to the same program that runs in the iml action. (You can read an overview of the iml action.) The example in the previous article was very simple and did not read or write data. This article compares

English

Advanced Analytics | Machine Learning

Rick WicklinJune 1, 2020 0

The Kullback–Leibler divergence between continuous probability distributions

In a previous article, I discussed the definition of the Kullback-Leibler (K-L) divergence between two discrete probability distributions. For completeness, this article shows how to compute the Kullback-Leibler divergence between two continuous distributions. When f and g are discrete distributions, the K-L divergence is the sum of f(x)*log(f(x)/g(x)) over all

English

Advanced Analytics | Machine Learning

Rick WicklinMay 28, 2020 1

Minimizing the Kullback–Leibler divergence

The Kullback–Leibler divergence is a measure of dissimilarity between two probability distributions. An application in machine learning is to measure how distributions in a parametric family differ from a data distribution. This article shows that if you minimize the Kullback–Leibler divergence over a set of parameters, you can find a

English

Blogs

Blogs

Advanced Analytics