A variance-covariance matrix expresses linear relationships between variables. Given the covariances between variables, did you know that you can write down an invertible linear transformation that "uncorrelates" the variables? Conversely, you can transform a set of uncorrelated variables into variables with given covariances. The transformation that works this magic is

## Tag: **Tips and Techniques**

Have you ever wanted to run a sample program from the SAS documentation or wanted to use a data set that appears in the SAS documentation? You can: all programs and data sets in the documentation are distributed with SAS, you just have to know where to look! Sample data

Statistical programmers often need mathematical constants such as π (3.14159...) and e (2.71828...). Programmers of numerical algorithms often need to know machine-specific constants such as the machine precision constant (2.22E-16 on my Windows PC) or the largest representable double-precision value (1.798E308 on my Windows PC). Some computer languages build these

In my recent article on simulating Buffon's needle experiment, I computed the "running mean" of a series of values by using a single call to the CUSUM function in the SAS/IML language. For example, the following SAS/IML statements define a RunningMean function, generate 1,000 random normal values, and compute the

Once again I rediscovered something that I once knew, but had forgotten. Fortunately, this blog is a good place to share little code snippets that I don't want to forget. I needed to compute the diagonal elements of a product of two matrices. In symbols, I have an nxp matrix,

In simulation studies, the response variable is often a binary (or Bernoulli) variable. Often 1 is used to indicate "success" (or the occurrence of an event) whereas 0 indicates "failure" (or the absence of an event). For example, the following SAS/IML statements define a vector x of zeros and ones:

The other day someone posted the following question to the SAS-L discussion list: Is there a SAS PROC out there that takes a multi-category discrete variable with character categories and converts it to a single numeric coded variable (not a set of dummy variables) with the character categories assigned as

I got an email asking the following question: In the following program, I don't know how many variables are in the data set A. However, I do know that the variable names are X1–Xk for some value of k. How can I read them all into a SAS/IML matrix when

Here's a quick tip to keep in mind when you write SAS/IML programs: although the SAS/IML documentation lists about 300 functions that are built into the SAS/IML language, you can also call hundreds of functions in Base SAS. Furthermore, you can pass in SAS/IML vectors for arguments to the functions.

Did you know that you can define "abbreviations" in the SAS enhanced editor? These handy little shortcuts can save you a lot of typing. For example, I have an abbreviation for the string _iml. Whenever I type _iml, the editor prompts me to replace those four characters with the following

Here is a little trick to file away. Given a row vector of zeros and ones, thought of as representing a number in base 2, the following SAS/IML statements compute the decimal value of that vector. proc iml; x = {1 0 0 1 1 1}; /* number in base

In SAS/IML 9.22 and beyond, you can call the R statistical programming language from within a SAS/IML program. The syntax is similar to the syntax for calling SAS from SAS/IML: You use a SUBMIT statement, but add the R option: SUBMIT / R. All statements in the program between the

In SAS/IML 9.22 and beyond, you can call any SAS procedure, DATA step, or macro from within a SAS/IML program. The syntax is simple: place a SUBMIT statement prior to the SAS statements and place an ENDSUBMIT statement after the SAS statements. This enables you to call any SAS procedure

I received the following email: Dear Dr. Wicklin, Why doesn't SYMPUT work in IML? In the DATA step, I can say CALL SYMPUT("MyMacro", 5) but this doesn't work in IML! Frustrated Dear Frustrated, The SYMPUT subroutine does work in SAS/IML software! However, the second argument to SYMPUT must be a

I previously wrote about using SAS/IML for nonlinear optimization, and demonstrated optimization by maximizing a likelihood function. Many well-known optimization algorithms require derivative information during the optimization, including the conjugate gradient method (implemented in the NLPCG subroutine) and the Newton-Raphson method (implemented in the NLPNRA method). You should specify analytic

To celebrate the first anniversary of Statistical Programming with SAS/IML Software, you can now download the SAS/IML tip sheets (also called "cheat sheets") that I created for the book. At conferences, SAS Press displays these tip sheets next to my book. They have been very popular. Download these SAS/IML cheat

I showed a SAS/IML customer a debugging tip, and she said that I should blog about it because she had never seen it before. The tip is very simple: inside of a DO loop, use the MOD function to selectively print the values of variables. Recall that the expression MOD(a,b)

The other day I needed to compute the signum function for each element of a matrix. If x is a real number, then the sgn(x) is -1 when x<0, 1 when x>0, and 0 when x=0. I wrote a SAS/IML module that contains a compact little expression: proc iml; start

Welcome, SAS 9.3! I've already blogged about some interface and graphical changes that everyone should know about. Now I'll put on my statistical hat and mention a few 9.3 features that excite me, personally, as a data analyst and a statistical programmer: As a statistician, I am keen to try

Here are a few new interface and graphics changes that every SAS programmer should know about SAS 9.3: HTML is now the default output destination when you run the SAS windowing environment. This means that tables and graphs appear in an HTML document instead of the classic LISTING destination. Of

In my statistical analysis of coupons article, I presented a scatter plot that includes the identity line, y=x. This post describes how to write a general program that uses the SGPLOT procedure in SAS 9.2. By a "general program," I mean that the program produces the result based on the

Andrew Ratcliffe posted a fine article titled "Inadequate Mends" in which he extols the benefits of including the name of a macro on the %MEND statement. That is, if you create a macro function named foo, he recommends that you include the name in two places: %macro foo(x); /** define

A fundamental operation in data analysis is finding data that satisfy some criterion. How many people are older than 85? What are the phone numbers of the voters who are registered Democrats? These questions are examples of locating data with certain properties or characteristics. The SAS DATA step has a

The most common way to read observations from a SAS data set into SAS/IML matrices is to read all of the data at once by using the ALL clause in the READ statement. However, the READ statement also has options that do not require holding all of the observations in

In my article on computing confidence intervals for rankings, I had to generate p random vectors that each contained N random numbers. Each vector was generated from normal distribution with different parameters. This post compares two different ways to generate p vectors that are sampled from independent normal distributions. Sampling

I recently blogged about how to eliminate a macro loop in favor of using SAS/IML language statements. The purpose of the program was to extract N 3x3 matrices from a big 3Nx3 matrix. The main portion of my PROC IML program looked something like this: proc iml; ... do i=0

String comparisons in SAS software are case-sensitive. For example, the uppercase letter "F" and lowercase letter "f" are treated as unique characters. When these two letters represent the same condition (for example, a female patient), the strings need to be handled in a case-insensitive manner, and a SAS programmer might

If you haven't signed up for SAS Global Forum 2011 in Las Vegas, you'd better get moving: February 28 is the last day for early registration and the discounted hotel prices. You should also sign up for the pre-conference statistical tutorials, which are filling up fast! I was tempted to

The other day I needed to check that a sequence of numerical values was in strictly increasing order. My first thought was to sort the values and compare the sorted and original values, but I quickly discarded that approach because it does not detect duplicate values in a montonic (nondecreasing)

When I wake up early to write my blog, I often wonder, "Is anyone going to read this?" Apparently so. I started writing The DO Loop in September, 2010. Since then, I've posted about 60 entries about statistical programming with SAS/IML software. Since this is a statistical blog, it is