Little known secrets of DO-loops with index variables

18

Complex loops in SAS programmingIterative loops are one of the most powerful and imperative features of any programming language, allowing blocks of code to be automatically executed repeatedly with some variations. In SAS we call them DO-loops because they are defined by the iterative DO statements. These statements come in three distinct forms:

  • DO with index variable
  • DO UNTIL
  • DO WHILE

In this blog post we will focus on the versatile iterative DO loops with index variable pertaining to SAS DATA steps, as opposed to its modest IML’s DO loops subset.

Iterative DO statement with index variable

The syntax of the DATA step’s iterative DO statement with index variable is remarkably simple yet powerful:

DO statement with index-variable

DO index-variable=specification-1 <, ...specification-n>;

...more SAS statements...

END;

It executes a block of code between the DO and END statements repeatedly, controlled by the value of an index variable. Given that angle brackets (< and >) denote “optional”, notice how index-variable requires at least one specification (specification-1) yet allows for multiple additional optional specifications (<, ...specification-n>) separated by commas.

Now, let’s look into the DO statement’s index-variable specifications.

Index-variable specification


Each specification denotes an expression, or a series of expressions as follows:

start-expression <TO stop-expression> <BY increment-expression> <WHILE (expression) | UNTIL (expression)>

Note that only start-expression is required here whereas <TO stop-expression>, <BY increment-expression>, and <WHILE (expression) or UNTIL (expression)> are optional.

Start-expression may be of either Numeric or Character type, while stop-expression and increment-expression may only be Numeric complementing Numeric start-expression.

Expressions in <WHILE (expression) | UNTIL (expression)> are Boolean Numeric expressions (numeric value other than 0 or missing is TRUE and a value of 0 or missing is FALSE).

Other iterative DO statements

For comparison, here is a brief description of the other two forms of iterative DO statement:

  • The DO UNTIL statement executes statements in a DO loop repetitively until a condition is true, checking the condition after each iteration of the DO loop. In other words, if the condition is true at the end of the current loop it will not iterate anymore, and processing continues with the next statement after END. Otherwise, it will iterate.
  • The DO WHILE statement executes statements in a DO loop repetitively while a condition is true, checking the condition before each iteration of the DO loop. That is if the condition is true at the beginning of the current loop it will iterate, otherwise it will not, and processing continues with the next statement after the END.

Looping over a list of index variable values/expressions

DO loops can iterate over a list of index variable values. For example, the following DO-loop will iterate its index variable values over a list of 7, 13, 5, 1 in the order they are specified:

data A; 
   do i=7, 13, 5, 1;
      put i=;
      output;
   end;
run;

This is not yet another syntax of iterative DO loop as it is fully covered by the iterative DO statement with index variable definition. In this case, the first value (7) is the required start expression of the required first specification, and all subsequent values (13, 5 and 1) are required start expressions of the additional optional specifications.

Similarly, the following example illustrates looping over a list of index variable character values:

data A1;
   length j $4;
   do j='a', 'bcd', 'efgh', 'xyz';
      put j=;
      output;
   end;
run;

Note: For character indexes, make sure to explicitly define a length for a character variable. Otherwise, it will be determined by SAS implicitly from its first occurrence. In this case, j=’a’ and the length of variable j will be assigned 1 which will result in truncating other specified longer value. That is why we have length j $4; statement before the DO-loop.

Since DO loop specifications denote expressions (values are just instances or subsets of expressions), we can expand our example to a list of actual expressions:

data B;
   p = constant('pi');
   do i=round(sin(p)), sin(p/2), sin(p/3);
      put i=;
      output;
   end;
run;

In this code DO-loop will iterate its index variable over a list of values defined by the following expressions: round(sin(p)), sin(p/2), sin(p/3).

Infinite loops

Since <TO stop> is optional for the index-variable specification, the following code is perfectly syntactically correct:

data C;
   do j=1 by 1;
      output;
   end;
run;

It will result in an infinite (endless) loop in which resulting data set will be growing indefinitely.

While unintentional infinite looping is considered to be a bug and programmers’ anathema, sometimes it may be used intentionally. For example, to find out what happens when data set size reaches the disk space capacity… Or instead of supplying a “big enough” hard-coded number (which is not a good programming practice) for the loop’s TO expression, we may want to define an infinite DO-loop and take care of its termination and exit inside the loop. For example, you can use IF exit-condition THEN LEAVE; or IF exit-condition THEN STOP; construct.

LEAVE statement immediately stops processing the current DO-loop and resumes with the next statement after its END.

STOP statement immediately stops execution of the current DATA step and SAS resumes processing statements after the end of the current DATA step.

The exit-condition may be unrelated to the index-variable and be based on some events occurrence. For instance, the following code will continue running syntactically “infinite” loop, but the IF-THEN-LEAVE statement will limit it to 200 seconds:

data D;
   start = datetime();
   do k=1 by 1;
      if datetime()-start gt 200 then leave;
      /* ... some processing ...*/
      output; 
   end;
run;

You can also create endless loop using DO UNTIL(0); or DO WHILE(1); statement, but again you would need to take care of its termination inside the loop.

Changing “TO stop” within DO-loop will not affect the number of iterations

If you think you can break out of your DO loop prematurely by adjusting TO stop expression value from within the loop, you may want to run the following code snippet to prove to yourself it’s not going to happen:

data E;
   n = 4;
   do i=1 to n;
      put i=;
      output;
      if i eq 2 then n = 2;
   end;
run;

This code will execute DO-loop 4 times despite that you change value of n from 4 to 2 within the loop.

According to the iterative DO statement documentation, any changes to stop made within the DO group do not affect the number of iterations. Instead, in order to stop iteration of DO-loop before index variable surpasses stop, change the value of index-variable so that it becomes equal to the value of stop, or use LEAVE statement to jump out of the loop. The following two examples will do just that:

data F;
   do i=1 to 4;
      put i=;
      output;
      if i eq 2 then i = 4;
   end;
run;
 
data G;
   do i=1 to 4;
      put i=;
      output;
      if i eq 2 then leave;
   end;
run;

Know thy DO-loop specifications

Here is a little attention/comprehension test for you.

How many times will the following DO-loop iterate?

data H;
   do i=1, 7, 3, 6, 2 until (i>3);
      put i=;
      output;
   end;
run;

If your answer is 2, you need to re-read the whole post from the beginning (I am only partly joking here).

You may easily find out the correct answer by running this code snippet in SAS. If you are surprised by the result, just take a closer look at the DO statement: there are 5 specifications for the index variable here (separated by commas) whereas UNTIL (expression) belongs to the last specification where i=2. Thus, UNTIL only applies to a single value of i=2 (not to any previous specifications of i =1,7,3,6); therefore, it has no effect as it is evaluated at the end of each iteration.

Now consider the following DO-loop definition:

data Z;
   pi = constant('pi');
   do x=3 while(x>pi), 10 to 1 by -pi*3, 20, 30 to 35 until(pi);
      put x=;
      output;
   end;
run;

I hope after reading this blog post you can easily identify the index variable list of values the DO-loop will iterate over. Feel free to share your solution and explanation in the comments section below.

Additional resources

Questions? Thoughts? Comments?

Do you find this post useful? Do you have questions, other secrets, tips or tricks about the DO loop? Please share with us below.

Share

About Author

Leonid Batkhan

Leonid Batkhan is an independent SAS consultant and blogger. He holds a Ph.D. in Computer Science and Automatic Control Systems and has been a SAS user for more than 25 years. From 1995 to 2021 he worked as a Data Management and Business Intelligence consultant at SAS Institute. During his career, Leonid has successfully implemented dozens of SAS applications and projects in various industries. All posts by Leonid Batkhan >>>

18 Comments

  1. Please keep in mind the definition of "iterate". It means to repeat. Therefore, if a loop (or data step) iterates once, it has executed twice. It seems that "iterate" is habitually misused in SAS documentation & articles.

    • Leonid Batkhan

      I’d agree with you, but then we’d both be wrong. In computer science, not just "in SAS documentation and articles", "iterate" is habitually used in conjunction with "single iteration" as a single pass of a repetitive process. Therefore, if a loop (or a data step) iterates once, it means it has executed once.

  2. Thanks Leonid, excellent stuff as usual. I hadn't realized that you could loop over a list of character values and I'll add that to my mental toolkit.

  3. Fascinating comments to a great article on Do Loops.

    Learned something new. Will keep as a reference.

    Thank you for posting

  4. Peter Lancashire on

    You have shown that the DO statement is powerful, and also that it can be confusing, particularly for those more familiar with other programming languages. For maintainability I prefer simple, easily understood code. For that reason I avoid the DATA step and loops if possible and use PROC SQL.

    The picture of loops is Tiger and Turtle. It is one of my pandemic cycling destinations - I was there on Monday. More: https://de.wikipedia.org/wiki/Tiger_and_Turtle_%E2%80%93_Magic_Mountain (also available, more briefly, in English).

    • Leonid Batkhan

      Thank you, Peter, I appreciate your feedback. It is normal for those who are quite comfortable with one programming language to be reasonably confused when switching to another programming language, at least in the beginning. That is why I wrote this post - to make SAS data step iterative DO loops abundantly clear for all. Making some effort to absorb it may totally switch one's perception of code simplicity and maintainability.

    • Leonid Batkhan

      Great! Thank you for the feedback, Nicole. I am glad you learned something new from my post.
      LEAVE statement is not necessarily used in conjunction with infinite loops. In any loop, e.g. if you search an array for some value and found it, you can cut that loop short by LEAVE-ing it.

  5. Anton Meshcheryakov on

    The ultimate little known secret of SAS loops: the best loop is NONE loop. SAS provides a plethora of implicit looping constructs (DATA step implicit loop, BY groups in most PROCs, GROUP BY in SQL) what eliminate the majority of use cases of loops in other languages. While some bona fide uses of the explicit loops do remain, one should always ponder if there is really a need for the loop at all, or it is just a bad data structure. Implicit loops completely avoid any starting, stopping and iteration rules and thus eliminate the inherent potential of errors and maintenance, should the changed data require the parameters update.

    • Leonid Batkhan

      Wow, Anton, this is by far the most profound and unorthodox programming comment I have ever read. It shakes the very foundation of all existed and existing programming languages as we know it. Not only it eliminates iterative loops in one easy swoop, it also gets rid of explicit programming in favor of implicit (and I naively thought that explicit programming is less prone to errors…) 🙂

      Aside from being ironic, I am looking forward to you coming up with a new generation programming language based on your proposed principles.

  6. Bartosz Jabłoński on

    The DO-loop can also iterate over variables, e.g.

    data _null_;
       X1=1;
       X2=2;
       X3=3;
       do i = X1, X2, X3;
          put i=;
       end;
    run;
    

    But of course it is a special case of "specification".

    All the best
    Bart

    • Leonid Batkhan

      Of course! Variables are subsets (or instances) of "expressions". I thought I covered it by this more general example:

      data B;
         p = constant('pi');
         do i=round(sin(p)), sin(p/2), sin(p/3);
            put i=;
            output;
         end;
      run;
    • Leonid Batkhan

      Thank you, Rick, for showing the “DOLIST” usage outside DATA steps. What you call a “DOLIST” is an implicit DO-loop denoted as (index-variable=specification-1 <, ...specification-n>) or a part of the DO statement denoted as index-variable=specification-1 <, ...specification-n>.
      Therefore, I would clarify that

      do i=7, 13, 5, 1;
      end;

      is not the “DOLIST” per se, but a DO-loop with the “DOLIST”:

      do "DOLIST";
      end;

      where "DOLIST" is i=7, 13, 5, 1.

      In your blog post that you reference in your comment, the “DOLIST syntax” goes beyond just listing the values; it also covers start, to, and by (also, commas are optional):

      proc sgplot data=sashelp.cars;
         scatter x=Weight y=Mpg_City;
         yaxis grid values=(10 to 40 by 5, 50 60); /* DOLIST; commas optional */
      run;

      The point I am making in this post for the DATA step, is that comma-separated list of values (expressions) is not a separate (additional) syntax since it falls under and fully covered by the general definition of DO Statement: Iterative.

      Also, in contrast with your “DOLIST” example pertaining to the PROC SGPLOT, commas are not optional in the explicit DO statement, they are required in the DATA step.

Leave A Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Back to Top