Rick Wicklin, PhD, is a distinguished researcher in computational statistics at SAS and is a principal developer of SAS/IML software. His areas of expertise include computational statistics, simulation, statistical graphics, and modern methods in statistical data analysis. Rick is author of the books Statistical Programming with SAS/IML Software and Simulating Data with SAS.
The SAS Output Delivery System (ODS) enables you to manage and customize tables (and graphics!) that are created by SAS procedures. I like to use the ODS SELECT statement to display only part of the output of a SAS procedure. For example, the UNIVARIATE procedure produces five tables by default,
Some SAS products such as SAS/IML Studio (which is included FREE as part of SAS/IML software) have interactive graphics. This makes it easy to interrogate a graph to determine values of "hidden" variables that might not appear in the graph. For example, in a scatter plot in SAS/IML Studio, you
Yesterday, December 7, 1941, a date which will live in infamy... - Franklin D. Roosevelt Today is the 70th anniversary of the Japanese attack on Pearl Harbor. The very next day, America declared war. During a visit to the Smithsonian National Museum of American History, I discovered the results of
In simulation studies, the response variable is often a binary (or Bernoulli) variable. Often 1 is used to indicate "success" (or the occurrence of an event) whereas 0 indicates "failure" (or the absence of an event). For example, the following SAS/IML statements define a vector x of zeros and ones:
Recently the "SAS Sample of the Day" was a Knowledge Base article with an impressively long title: Sample 42165: Using a stored process to eliminate duplicate values caused by multiple group memberships when creating a group-based, identity-driven filter in SAS® Information Map Studio "Wow," I thought. "This is the longest
The other day someone posted the following question to the SAS-L discussion list: Is there a SAS PROC out there that takes a multi-category discrete variable with character categories and converts it to a single numeric coded variable (not a set of dummy variables) with the character categories assigned as
I got an email asking the following question: In the following program, I don't know how many variables are in the data set A. However, I do know that the variable names are X1–Xk for some value of k. How can I read them all into a SAS/IML matrix when
I have previously written about how to create funnel plots in SAS software. A funnel plot is a way to compare the aggregated performance of many groups without ranking them. The groups can be states, counties, schools, hospitals, doctors, airlines, and so forth. A funnel plot graphs a performance metric
Here's a quick tip to keep in mind when you write SAS/IML programs: although the SAS/IML documentation lists about 300 functions that are built into the SAS/IML language, you can also call hundreds of functions in Base SAS. Furthermore, you can pass in SAS/IML vectors for arguments to the functions.
Halloween night was rainy, so many fewer kids knocked on the door than usual. Consequently, I'm left with a big bucket of undistributed candy. One evening as I treated myself to a mouthful of tooth decay, I chanced to open a package of Wonka® Bottle Caps. The package contained three
Did you know that you can define "abbreviations" in the SAS enhanced editor? These handy little shortcuts can save you a lot of typing. For example, I have an abbreviation for the string _iml. Whenever I type _iml, the editor prompts me to replace those four characters with the following
Here is a little trick to file away. Given a row vector of zeros and ones, thought of as representing a number in base 2, the following SAS/IML statements compute the decimal value of that vector. proc iml; x = {1 0 0 1 1 1}; /* number in base
One aspect of blogging that I enjoy is getting feedback from readers. Usually I get statistical or programming questions, but every so often I receive a comment from someone who stumbled across a blog post by way of an internet search. This morning I received the following delightful comment on
If you want to extract values from a SAS/IML vector, use the subscripting operation, such as in the following example: proc iml; x = {A B C D E}; y = x[{1 2 3}]; /* {A,B,C} */ The vector y contains the first three elements of x. However, did you
Sometimes you want to label only certain observations in a plot. This is useful in many ways, but one use is to label outliers on a scatter plot. In the SGPLOT procedure, the DATALABEL= option enables you to specify the name of a variable that is used to label observations.
I was at the Wikipedia site the other day, looking up properties of the Chi-square distribution. I noticed that the formula for the median of the chi-square distribution with d degrees of freedom is given as ≈ d(1-2/(9d))3. However, there is no mention of how well this formula approximates the
What do you call an interview on Twitter? A Tw-interview? A Twitter-view? Regardless of what you call it, I'm going to be involved in a "live chat" on Twitter this coming Thursday, 10NOV2011, 1:30–2:00pm ET. The hashtag is #saspress. Shelly Goodin (@SASPublishing) and SAS Press author recruiter Shelley Sessoms (@SSessoms)
Last week I showed how to use the UNIQUE-LOC technique to iterate over categories in a SAS/IML program. The observant reader might have noticed that the algorithm, although general, could be made more efficient if the data are sorted by categories. The UNIQUEBY Technique Suppose that you want to compute
Being able to reshape data is a useful skill in data analysis. Most of the time you can use the TRANSPOSE procedure or the SAS DATA step to reshape your data. But the SAS/IML language can be handy, too. I only use PROC TRANSPOSE a few times per year, so
I was recently asked the following question: I am using bootstrap simulations to compute critical values for a statistical test. Suppose I have test statistic for which I want a p-value. How do I compute this? The answer to this question doesn't require knowing anything about bootstrap methods. An equivalent
When you analyze data, you will occasionally have to deal with categorical variables. The typical situation is that you want to repeat an analysis or computation for each level (category) of a categorical variable. For example, you might want to analyze males separately from females. Unlike most other SAS procedures,
In SAS/IML 9.22 and beyond, you can call the R statistical programming language from within a SAS/IML program. The syntax is similar to the syntax for calling SAS from SAS/IML: You use a SUBMIT statement, but add the R option: SUBMIT / R. All statements in the program between the
"I think that my data are exponentially distributed, but how can I check?" I get asked that question a lot. Well, not specifically that question. Sometimes the question is about the normal, lognormal, or gamma distribution. A related question is "Which distribution does my data have," which was recently discussed
I was contacted by SAS Technical Support regarding a customer who was trying to use SAS/IML to compute quantiles of the folded normal distribution. I had heard of the distribution, but it is not built into SAS and I had never worked with it. Nevertheless, I set out to understand
In SAS/IML 9.22 and beyond, you can call any SAS procedure, DATA step, or macro from within a SAS/IML program. The syntax is simple: place a SUBMIT statement prior to the SAS statements and place an ENDSUBMIT statement after the SAS statements. This enables you to call any SAS procedure
When I learn a new statistical technique, one of first things I do is to understand the limitations of the technique. This blog post shares some thoughts on modeling finite mixture models with the FMM procedure. What is a reasonable task for FMM? When are you asking too much? I
Normal, Poisson, exponential—these and other "named" distributions are used daily by statisticians for modeling and analysis. There are four operations that are used often when you work with statistical distributions. In SAS software, the operations are available by using the following four functions, which are essential for every statistical programmer
I received the following email: Dear Dr. Wicklin, Why doesn't SYMPUT work in IML? In the DATA step, I can say CALL SYMPUT("MyMacro", 5) but this doesn't work in IML! Frustrated Dear Frustrated, The SYMPUT subroutine does work in SAS/IML software! However, the second argument to SYMPUT must be a
I previously wrote about using SAS/IML for nonlinear optimization, and demonstrated optimization by maximizing a likelihood function. Many well-known optimization algorithms require derivative information during the optimization, including the conjugate gradient method (implemented in the NLPCG subroutine) and the Newton-Raphson method (implemented in the NLPNRA method). You should specify analytic
A popular use of SAS/IML software is to optimize functions of several variables. One statistical application of optimization is estimating parameters that optimize the maximum likelihood function. This post gives a simple example for maximum likelihood estimation (MLE): fitting a parametric density estimate to data. Which density curve fits the
