New Year to me is always a stark reminder of the inexorability of Time. In a day-to-day life, time is measured in small denominations - minutes, hours, days… But come New Year, and this inescapable creature – Time – makes its decisive leap – and in a single instant, we become officially older and wiser by the entire year’s worth.
What’s a better time to re-assess ourselves, personally and professionally! What’s a better time to Resolve to improve your SAS programming skills, as skillfully crafted by Michael A. Raithel in his recent blog post.
I thought I could write a post showing how to be efficient and kill two birds with one stone. The birds here are two New Year’s Raithel’s proposed resolutions:
#2 Volunteer to help junior SAS programmers.
#12 Reduce processing time by writing more efficient programs.
To combine the two, I could have titled this post “Helping junior SAS programmers to reduce processing time by writing more efficient programs”. However, I am not going to “teach” you efficient coding techniques which are a subject deserving of a multi-volume treatise. I will just give you a simple tool that is a must-have for any SAS programmer (not just junior) who considers writing efficient SAS code important. This simple tool has been the ultimate judge of any code’s efficiency and it is called timer.
What is efficient?
Setting aside hardware constraints and limitations (which are increasingly diminishing nowadays), efficient means fast or at least fast enough not to exceed ever-shrinking user tolerance of wait time.
Of course, if you are developing a one-time run code to generate some ad-hoc report or produce results for uniquely custom computations, your efficiency criteria might be different, such as “as long as it ends before the deadline” or at least “does not run forever”.
However, SAS code is often developed for some interactive applications. In these scenarios multiple users run the code over and over again. It may run behind the scenes of a web application with a user waiting (or rather not wanting to wait) for results. In these cases, SAS code must be really fast, and any improvement in its efficiency is multiplied by the number of times it is run.
What is out there?
SAS provides the following SAS system options to measure the efficiency of SAS code:
STIMER. You may not realize that you use this option every time you run a SAS program. This option is turned on by default (NOSTIMER to turn it off) and controls information written to the SAS Log by each SAS step. Each step of a SAS program by default generates the following sample NOTE in SAS Log:
NOTE: DATA statement used (Total process time):
real time 1.31 seconds
cpu time 1.10 seconds
FULLSTIMER. This option (NOFULLSTIMER to turn it off) provides much more information on used resources for each step. A sample Log output of a FULLSTIMER option for a SAS Data Step is listed below:
NOTE: DATA statement used: real time 0.06 seconds user cpu time 0.02 seconds system cpu time 0.00 seconds Memory 88k Page Faults 10 Page Reclaims 0 Page Swaps 0 Voluntary Context Switches 22 Involuntary Context Switches 0 Block Input Operations 10 Block Output Operations 12
While the FULLSTIMER option provides plenty of information for SAS code optimization, in many cases it is more than you really need. On the other hand, STIMER may provide quite valuable information about each step, thus identifying the most critical steps of your SAS program.
Get your own SAS timer
If your efficiency criteria is how fast your SAS program runs as a whole, than you need an old-fashioned timer. The one with start and stop events and time elapsed between them. To achieve this in SAS programs, I use the following technique.
- At the very beginning of your SAS program, place the following line of code that effectively starts the timer and remembers the start time:
- At the end of your SAS program place the following code snippet that captures the end time, calculates duration and outputs it to the SAS Log:
/* Start timer */ %let _timer_start = %sysfunc(datetime());
/* Stop timer */ data _null_; dur = datetime() - &_timer_start; put 30*'-' / ' TOTAL DURATION:' dur time13.2 / 30*'-'; run;
The resulting output in the SAS log will look like this:
------------------------------ TOTAL DURATION: 0:01:31.02 ------------------------------
Despite its utter simplicity, this little timer program is a very convenient tool to improve your SAS code efficiency. You can use it to compare or benchmark your SAS programs in their entirety.
Warning. In the above timer, I used the datetime() function. I insist on using it instead of the time() function as I saw in many online resources. Keep in mind that the time() function resets to 0 at midnight. While time() will work just as well when start and stop times are within the same date, it will produce completely meaningless results when start time falls within one date and stop time falls within another date. You can easily trap yourself in when you submit your SAS program right before midnight while it ends after midnight. This will result in an incorrect, even negative, duration.
I hope using this SAS timer will help you writing more efficient SAS programs.
23 Comments
I cannot tell you how often I use this handy timer. And it's funny, because the simplicity is what makes it so useful. Thanks again, Leonid.
Thank you, John! Similarly, I use it practically in all my SAS programs where time (duration) is a factor.
This worked great, thanks Leonid!
Thank you, John, so nice to hear from you.
Hello, Thank you for your blog, it is very useful. So can I set timer to execute certain program to run at diiferent timings, like one program to execute at 1 minutes, the other as 3 ,5 and 6 minutes, isn't it. Could you please tell me is it possible to set the different timings to execute the SAS program.
Thank you for asking. No, this timer only measures run duration of your SAS program. You are talking about a different task, it is called scheduling. For scheduling, see these resources:
- Four ways to schedule SAS tasks
- Scheduling in SAS® 9.4
Thanks for the great tip.
You are welcome, John!
Thank you - this is a very useful blog. I am stealing your code snippets 🙂
Thank you, Daria! Enjoy! 🙂
Do I have to copy and paste the codes in each of my macros? Is there another way to print the run time of my macros without coping and pasting the same codes into each macro?
Hi Jack,
Yes, one way of implementing this timer is to copy & paste these two code snippets in the beginning and in the end of each macro. In this case you can modify your second snippet to include message on the macro name duration of which you are reporting. You can modify the second snippet to look like this:
When using with a macro, I also suggest declaring _timer_start macro variable as local:
so it would not interfere with the same-named macro variable in the calling program.
Another way of implementing this macro is to place the 2 code snippets before and after the macro call.
If you come up with a simpler method I would like to hear from you.
Thank you, Leonid. I added your timer to a SAS macro that I have been tuning for speed.
Thank you for your feedback, Andrew. Hope your macro will run lightning fast!
Thanks for this *very* helpful tidbit! I implemented it immediately after I finished reading this.
Thank you for your feedback, Michele. Hope you enjoy this SAS timer.
Are you aware of any articles that go through the fullstimer line by line and explains what it means?
See this documentation: http://support.sas.com/rnd/scalability/tools/fullstim/
Recent releases include a timestamp with the fullstimer details. Have that option set and post processing the log provides a useful substitute for forward thinking - almost like hindsight for performance .
Peter, the "timestamp" indicates just when a step was executed, it does tell anything about its duration. At the same time, "real time" and "user cpu time" are indicative of the step duration. The timer described in this post, calculates and shows summary duration of the whole program, not just a single step.
Leonid,
Great article!
I hope that people instrument their mid-to-long running SAS programs with your Start Timer/Stop Timer up front, so that they can measure the performance over time. It's a bit too late to wish that one had done so when there is a perception of a performance problem. At that point, all one can do is begin measuring, going forward.
Looking forward to your next post.
----MMMMIIIIKKKKEEEE
(aka Michael A Raithel)
Thank you, Michael.
However, I think it is never too late to address a performance problem of a living program. I use this timer in most if not all of my SAS code since last millennium.
Pingback: SAS FULLSTIMER—turn it on! - SAS Users