Rick Wicklin, PhD, is a distinguished researcher in computational statistics at SAS and is a principal developer of SAS/IML software. His areas of expertise include computational statistics, simulation, statistical graphics, and modern methods in statistical data analysis. Rick is author of the books Statistical Programming with SAS/IML Software and Simulating Data with SAS.
Vector languages such as SAS/IML, MATLAB, and R are powerful because they enable you to use high-level matrix operations (matrix multiplication, dot products, etc) rather than loops that perform scalar operations. In general, vectorized programs are more efficient (and therefore run faster) than programs that contain loops. For an example
In 2013 I published 110 blog posts. Some of these articles were more popular than others, often because they were linked to from a SAS newsletter such as the SAS Statistics and Operations Research News. In no particular order, here are some of my most popular posts from 2013, organized
O Christmas tree, O Christmas tree, Last year a fractal made thee! O Christmas tree, O Christmas tree, A heat map can display thee! O tree of green, adorned with lights! A trunk of brown, the rest is white. O Christmas tree, O Christmas tree, A heat map can display
Recently a SAS/IML programmer asked a question regarding how to perform matrix arithmetic when some of the data are in vectors and other are in matrices. The programmer wanted to add the following matrices: The problem was that the numbers in the first two matrices were stored in vectors. The
A heat map is a graphical representation of a matrix that uses colors to represent values in the matrix cells. Heat maps often reveal the structure of a matrix. There are three common applications of visualizing matrices with heat maps: Visualizing a correlation or covariance matrix reveals relationships between variables.
When learning a new language, it is important to learn to interpret error messages that come from the language's parser or compiler. Three years ago I blogged about how to interpret SAS/IML error messages. However, many questions have been posted to the SAS/IML Support Community that indicate that some people
As the International Year of Statistics comes to a close, I've been reflecting on the role statistics plays in our modern society. Of course, statistics provides estimates, forecasts, and the like, but to me the great contribution of statistics is that it enables us to deal with uncertainty in a
Each year my siblings choose names for a Christmas gift exchange. It is not unusual for a sibling to pick her own name, whereupon the name is replaced into the hat and a new name is drawn. In fact, that "glitch" in the drawing process was a motivation for me
If you write an n x p matrix from PROC IML to a SAS data set, you'll get a data set with n rows and p columns. For some applications, it is more convenient to write the matrix in a "long format" with np observations and three columns. The first
For several years, there has been interest in calling R from SAS software, primarily because of the large number of special-purpose R packages. The ability to call R from SAS has been available in SAS/IML since 2009. Previous blog posts about R include a video on how to call R
When I call R from within the SAS/IML language, I often pass parameters from SAS into R. This feature enables me to write general-purpose, reusable, modules that can analyze data from many different data sets. I've previously blogged about how to pass values to SAS procedures from PROC IML by
In using a vector-matrix language such as SAS/IML, MATLAB, or R, one of the challenges for programmers is learning how to vectorize computations. Often it is not intuitive how to program a computation so that you avoid looping over the rows and columns of a matrix. However, there are a
While sorting through an old pile of papers, I discovered notes from a 2012 SAS conference that I had attended. Next to the abstract for one presentation, I had scrawled a note to myself that read "BLOG about the incomplete beta function!" Okay, Rick, whatever you say! In statistics, the
My daughter's middle school math class recently reviewed how to compute the greatest common factor (GCF) and the least common multiple (LCM) of a set of integers. (The GCF is sometimes called the greatest common divisor, or GCD.) Both algorithms require factoring integers into a product of primes. While helping
The mosaic plot is a graphical visualization of a frequency table. In a previous post, I showed how to use the FREQ procedure to create a mosaic plot. This article shows how to create a mosaic plot by using the MOSAICPARM statement in the graph template language (GTL). (The MOSAICPARM
Mosaic plots (Hartigan and Kleiner, 1981; Friendly, 1994, JASA) are used for exploratory data analysis of categorical data. Mosaic plots have been available for decades in SAS products such as JMP, SAS/INSIGHT, and SAS/IML Studio. However, not all SAS customers have access to these specialized products, so I am pleased
I was looking at someone else's SAS/IML program when I saw this line of code: y = sqrt(x<>0); The statement uses the element maximum operator (<>) in the SAS/IML language to make sure that negative value are never passed to the square root function. This little trick is a real
If you've ever tried to use PROC FREQ to create a frequency table of two character variables, you know that by default the categories for each variable are displayed in alphabetical order. A different order is sometimes more useful. For example, consider the following two-way table for the smoking status
A challenge for statistical programmers is getting data into the right form for analysis. For graphing or analyzing data, sometimes the "wide format" (each subject is represented by one row and many variables) is required, but other times the "long format" (observations for each subject span multiple rows) is more
SAS/IML programmers know that the VECDIAG matrix can be used to extract the diagonal elements of a matrix. For example, the following statements extract the diagonal of a 3 x 3 matrix: proc iml; m = {1 2 3, 4 5 6, 7 8 9}; v = vecdiag(m); /* v = {1,5,9}
This article is about rotating matrices. No, I don't mean "rotation matrices," I mean rotating matrices. As in turning a matrix 90 degrees in a clockwise or counterclockwise direction. I was reading a program written in MATLAB in which the programmer used a MATLAB function called ROT90, which rotates a
On Kaiser Fung's Junk Charts blog, he showed a bar chart that was "published by Teach for America, touting its diversity." Kaiser objected to the chart because the bar lengths did not accurately depict the proportions of the Teach for America corps members. The chart bothers me for another reason:
Should you ever guess on the SAT® or PSAT standardized tests? My son is getting ready to take the preliminary SAT (PSAT), which is a practice test for the SAT. A teacher gave his class this advice regarding guessing: For a multiple-choice questions, if you can eliminate one or two
In my last blog post I described how to implement a "runs test" in the SAS/IML language. The runs test determines whether a sequence of two values (for example, heads and tails) is likely to have been generated by random chance. This article describes two applications of the runs test.
While walking in the woods, a statistician named Goldilocks wanders into a cottage and discovers three bears. The bears, being hungry, threaten to eat the young lady, but Goldilocks begs them to give her a chance to win her freedom. The bears agree. While Mama Bear and Papa Bear block
What is the best way to share SAS/IML functions with your colleagues? Give them the source code? Create a function library that they can use? This article describes three techniques that make your SAS/IML functions accessible to others. As background, remember that you can define new functions and subroutines in
Massive open online courses (MOOCs) are all the rage today. Some people see free online courses as a convenient way to introduce statistical concepts to tens of thousands of students who would not otherwise have an opportunity to learn about data analysis. Whereas 2013 is the International Year of Statistics,
Last week I described how to generate permutations in SAS. A related concept is the "combination." In probability and statistics, a combination is a subset of k items chosen from a set that contains N items. Order does not matter, so although the ordered triplets (B, A, C) and (C,
This is the last post in my recent series of articles on computing contours in SAS. Last month a SAS customer asked how to compute the contours of the bivariate normal cumulative distribution function (CDF). Answering that question in a single blog post would have resulted in a long article,
I've written several articles that show how to generate permutations in SAS. In the SAS DATA step, you can use the ALLPEM subroutine to generate all permutations of a DATA step array that contain a small number (18 or fewer) elements. In addition, the PLAN procedure enables you to generate
