Monitor the progress of a long-running SAS/IML program

2

When you have a long-running SAS/IML program, it is sometimes useful to be able to monitor the progress of the program. For example, suppose you need to computing statistics for 1,000 different data sets and each computation takes between 5 and 30 seconds. You might want to output a message after every 100th computation so that you know at a glance whether the program is close to finishing. (You might also want to check out some efficiency tips and ways to vectorize your program.)

Now some of you might be thinking, "What's the problem? Just put a PRINT statement inside the loop!" Right idea, but it doesn't work like you might expect. For efficiency, SAS/IML loops are compiled and executed in a very efficient manner. One of the consequences of this efficiency is that ODS output is not flushed until the entire DO loop has completed! Run the following program (which takes 10 seconds) and notice that you get NO output until the loop completes, at which point you will see all 10 messages displayed at once:

proc iml;
do i = 1 to 1000;
   if mod(i,100)=0 then do;
      print "Iteration= " i;
   end;
   call sleep(1, 0.01);  /* delay for 1/100th second */
end;
print "Execution finished";

So how can you force SAS/IML to display information while the loop is executing? I know two ways. In the IMLPlus language (available in the SAS/IML Studio application), you can use the PRINTNOW statement to flush the ODS output. In PROC IML, you can use the SUBMIT and ENDSUBMIT statements to write messages to the SAS Log.

The IMLPlus solution: The PRINTNOW statement

In the SAS/IML Studio environment, the IMLPlus language supports the PRINTNOW statement. The PRINTNOW statement tells the IMLPlus program to retrieve and display any pending output from the SAS Workspace server. In the IMLPlus language, the following statement prints the iteration history while the DO loop is executing:

do i = 1 to 1000;
   if mod(i,100)=0 then do;
      print "Iteration= " i;
      printnow;          /* IMLPlus statement: works only in SAS/IML Studio */
   end;
   call sleep(1, 0.01);  /* delay for 1/100th second */
end;

The PROC IML solution: The SUBMIT/ENDSUBMIT statements

If you are running PROC IML as part of a larger SAS program, you can use the SUBMIT/ENDSUBMIT statements to write messages to the SAS Log. Recall that you can pass values from SAS/IML matrices into SAS statements. If you include the name of a SAS/IML variable on the SUBMIT statement, the contents of that variable are substituted into the SAS code before the SUBMIT block is sent to SAS. In the following example, the value of the counter i is sent into a SUBMIT block, and the %PUT macro statement is used to print the iteration value to the SAS Log:

proc iml;
do i = 1 to 1000;
   if mod(i,100)=0 then do;
      submit i;
      %put Iteration= &i;
      endsubmit;
   end;
   call sleep(1, 0.01);  /* delay for 1/100th second */
end;
print "Execution finished";

The previous program displays the iteration count in the SAS Log while the program is running.

Write NOTE: into the log

Some SAS programmers know that if you use the %PUT statement to write a message that begins with 'NOTE:', then SAS will color-code that message in the log so that it looks like a system-issued note. However, to get that feature to work in the SUBMIT block, you need to turn on the NOTES option, which is off by default. That is, use OPTIONS NOTES at the top of the SUBMIT block, as follows:

      /* write NOTE: <message> */
      submit i;
      options notes;               /* turn on notes */
      %put NOTE: Iteration = &i;   /* displays NOTE: in color */
      endsubmit;

Do you have long-running programs in SAS? Have you developed a clever way to monitor their progress while they run? Leave a comment about your technique.

Share

About Author

Rick Wicklin

Distinguished Researcher in Computational Statistics

Rick Wicklin, PhD, is a distinguished researcher in computational statistics at SAS and is a principal developer of PROC IML and SAS/IML Studio. His areas of expertise include computational statistics, simulation, statistical graphics, and modern methods in statistical data analysis. Rick is author of the books Statistical Programming with SAS/IML Software and Simulating Data with SAS.

2 Comments

  1. Daniel Adelsberg on

    Rick, I'm running SAS/Studio with IML and I'd like to print some results in the log. But everything I try sends the results to the output window instead. Any suggestions? Here's my code:

    proc iml;
    reset log print;
    x=5;
    print(x);
    submit / R;
    y<-10
    print(y)
    str(y)
    endsubmit;
    run;
    quit;

    • Rick Wicklin

      You'll notice that this article, which is about printing to the log, does not mention the RESET LOG statement. That's because that is ancient technology from before the days of ODS. The documentation for the RESET LOG statement says:

      RESET LOG specifies whether output is routed to the SAS Log rather than to the LISTING destination. In the log, the results are interleaved with the statements and messages. The RESET LOG has no effect if the ODS LISTING destination is not active.

      I don't think SAS Studio supports the LISTING destination, which is used for batch processing. To write a note to the log in SAS Studio, use the last technique in this article.

Leave A Reply

Back to Top