"What is the chance that two people in a room of 20 share initials?" This was the question posed to me by a colleague who had been taking notes at a meeting with 20 people. He recorded each person's initials next to their comments and, upon editing the notes, was
Computing probabilities can be tricky. And if you are a statistician and you get them wrong, you feel pretty foolish. That's why I like to run a quick simulation just to make sure that the numbers that I think are correct are, in fact, correct. My last post of 2010
Over at the SAS/IML Discussion Forum, someone posted an interesting question about how to create a special matrix that contains all combinations of zeros and ones for a given size. Specifically, the problem is as follows. Given an integer n ≥ 1, produce a matrix with 2n rows and n
It's a New Year and I'm ready to make some resolutions. Last year I launched this blog with my Hello, World post in which I said: In this blog I intend to discuss, describe, and disseminate ideas related to statistical programming with the SAS/IML language.... I will present tips and
In many families, siblings draw names so that each family member and spouse gives and receives exactly one present. This year there was a little bit of controversy when a family member noticed that once again she was assigned to give presents to me. This post includes my response to
When I wake up early to write my blog, I often wonder, "Is anyone going to read this?" Apparently so. I started writing The DO Loop in September, 2010. Since then, I've posted about 60 entries about statistical programming with SAS/IML software. Since this is a statistical blog, it is
Recently, I needed to detect whether a matrix consists entirely of missing values. I wrote the following module: proc iml; /** Module to detect whether all elements of a matrix are missing values. Works for both numeric and character matrices. Version 1 (not optimal) **/ start isMissing(x); if type(x)='C' then
NOTE: SAS stopped shipping the SAS/IML Studio interface in 2018. The references in this article to IMLPlus and SAS/IML Studio are no longer relevant. There are three kinds of programming errors: parse-time errors, run-time errors, and logical errors. It doesn't matter what language you are using (SAS/IML, MATLAB, R, C/C++,
Both covariance matrices and correlation matrices are used frequently in multivariate statistics. You can easily compute covariance and correlation matrices from data by using SAS software. However, sometimes you are given a covariance matrix, but your numerical technique requires a correlation matrix. Other times you are given a correlation matrix,
Sample covariance matrices and correlation matrices are used frequently in multivariate statistics. This post shows how to compute these matrices in SAS and use them in a SAS/IML program. There are two ways to compute these matrices: Compute the covariance and correlation with PROC CORR and read the results into
I enjoy reading about the Le Monde puzzles (and other topics!) at Christian Robert's blog. Recently he asked how to convert a number with s digits into a numerical vector where each element of the vector contains the corresponding digit (by place value). For example, if the number is 4321,
I am thankful to be a statistical programmer. When I wake up in the morning, I am eager to start my day. I love statistics, programming, and working at SAS, and I write my blog to share that joy. This a Golden Age for statistical programmers because theoretical ideas and
In a previous post, I discussed computing regression coefficients in different polynomial bases and showed how the coefficients change when you change the basis functions. In particular, I showed how to convert the coefficients computed in one basis to coefficients computed with respect to a different basis. It turns out
I am pleased to announce that the fine folks at SAS Press have made Chapter 2 of my book, Statistical Programming with SAS/IML Software available as a free PDF document. The chapter is titled "Getting Started with the SAS/IML Matrix Programming Language," and it features More than 60 fully functional
Chris started a tradition for SAS Press authors to post a photo of themselves with their new book. Thanks to everyone who helped with the production of Statistical Programming with SAS/IML Software.
Suppose that you compute the coefficients of a polynomial regression by using a certain set of polynomial effects and that I compute coefficients for a different set of polynomial effects. Can I use my coefficients to find your coefficients? The answer is yes, and this article explains how. Standard Polynomial
Sampling with replacement is a useful technique for simulations and for resampling from data. Over at the SAS/IML Discussion Forum, there was a recent question about how to use SAS/IML software to sample with replacement from a set of events. I have previously blogged about efficient sampling, but this topic
The SAS/IML language provides the QUAD function for evaluating one-dimensional integrals. You can also use the QUAD function to compute a double integral as an iterated integral. A One-Dimensional Integration Suppose you want to evaluate the following integral: To evaluate this integral in the SAS/IML language: Define a function module
I was recently asked how to create a tridiagonal matrix in SAS/IML software. For example, how can you easily specify the following symmetric tridiagonal matrix without typing all of the zeros? proc iml; m = {1 6 0 0 0, 6 2 7 0 0, 0 7 3 8 0,
In a previous post, I discussed how to use the LOC function to eliminate loops over observations. Dale McLerran chimed in to remind me that another way to improve efficiency is to use subscript reduction operators. I ended my previous post by issuing a challenge: can you write an efficient
Have you ever been stuck while trying to solve a scrambled-word puzzle? You stare and stare at the letters, but no word reveals itself? You are stumped. Stymied. I hope you didn't get stumped on the word puzzle I posted as an anniversary present for my wife. She breezed through
In a previous post, I discussed how to generate random permutations of N elements. But what if you want to systematically iterate through a list of ALL permutations of N elements? In the SAS DATA step you can use the ALLPERM subroutine in the SAS DATA step. For example, the
My previous post on creating a random permutation started me thinking about word games. My wife loves to solve the daily Jumble® puzzle that runs in our local paper. The puzzle displays a string of letters like MLYBOS, and you attempt to unscramble the letters to make an ordinary word.
I recently read a paper that described a SAS macro to carry out a permutation test. The permutations were generated by PROC IML. (In fact, an internet search for the terms "SAS/IML" and "permutation test" gives dozens of papers in recent years.) The PROC IML code was not as efficient
The SAS/IML language is a vector language, so statements that operate on a few long vectors run much faster than equivalent statements that involve many scalar quantities. For example, in a previous post, I asserted that the LOC function is much faster than writing a loop, for finding observations that
The SAS/IML run-time library contains hundreds of functions and subroutines that you can call to perform statistical analysis. There are also many functions in Base SAS software that you can call from SAS/IML programs. However, one day you might need to compute some quantity for which there is no prewritten
Today is the birthday of Bernhard Riemann, a German mathematician who made fundamental contributions to the fields of geometry, analysis, and number theory. Riemann is definitely on my list of the greatest mathematicians of all time, and his conjecture about the distribution of prime numbers is one of the great
Missing values are a fact of life. Many statistical analyses, such as regression, exclude observations that contain missing values prior to forming matrix equations that are used in the analysis. This post shows how to find rows of a data matrix that contain missing values and how to remove those
A frequently performed task in data analysis is identifying all the observations in a data set that satisfy certain conditions. For example, you might want to identify all of the female patients in your study or to identify all patients whose systolic blood pressure is greater than 140 mm Hg.
The R You Ready blog posed an interesting problem. Essentially, you have a vector that contains n(n+1)/2 elements, and you want to pack those elements into the upper left triangular portion of a matrix. For example, if your data are proc iml; /** vector v is given: ncol(v) = n(n+1)/2 for
