Let SAS write batch scripts for you

20

Automation of creating batch scripts in SASIf Necessity is the mother of Invention, then, perhaps, the father of Automation is Laziness. Automation is all about convenience, comfort, and productivity. Why do it yourself if you can devise something to do it for you!

In my previous post Running SAS programs in batch under Unix/Linux, we learned that in order to automate SAS jobs submissions, they are often run in batch mode. We also learned that we usually create batch scripts as a convenient way to run SAS programs in batch. To create a unique SAS log file generated with each batch submission, a typical batch script may look like follows:

#!/usr/bin/sh
dtstamp=$(date +%Y.%m.%d_%H.%M.%S)
pgmname="/sas/code/project1/program1.sas"
logname="/sas/code/project1/program1_$dtstamp.log"
/sas/SASHome/SASFoundation/9.4/sas $pgmname -log $logname

It will allow you to submit your SAS program /sas/code/project1/program1.sas in batch, and also capture SAS log file with a convenient date-time suffix in the same directory.

SAS program to write batch scripts

But what if we are deploying multiple SAS programs? Well, then we would need to create a batch script for each of them. They will all look similar to each other, and that is when most human errors usually occur – when we do something similar, monotonously, over and over again.  Besides, I found working with the Unix Visual Editor (“vi editor”) is not quite a 21st century experience.

What would a normal SAS programmer do in such a situation? That’s right – we would write a SAS program to write a batch script file! Let’s do it. Let’s automate the automation.

In its simplest form, to replicate the above batch script example our SAS program would look like this:

filename b '/sas/code/project1/program1.sh';
 
data _null_;
   file b;
   input;
   put _infile_;
   datalines;
#!/usr/bin/sh
dtstamp=$(date +%Y.%m.%d_%H.%M.%S)
pgmname="/sas/code/project1/program1.sas"
logname="/sas/code/project1/program1_$dtstamp.log"
/sas/SASHome/SASFoundation/9.4/sas $pgmname -log $logname
;

Setting up batch file permissions

As we already know from my previous post, we need to assign certain permissions to our batch file in order to make it executable. For example, if you want to give yourself (Owner) and Group execution permissions then your script file permissions can be as:

-rwxr-x---, or 750 in octal representation.

In order to do that you can to add to your SAS code the following x-statement:

options noxwait;
x 'chmod 750 /sas/code/project1/program1.sh';

Alternatively, you can use %SYSEXEC macro statement (no quoting for the OS command) or SYSTASK statement, or CALL SYSTEM routine (used within a data step).

When you create a batch script by running the above code in SAS Enterprise Guide (EG), you don’t have to leave the comfort of your SAS environment or even touch Unix vi editor. Moreover, you can even submit your SAS job in batch mode right from your SAS EG Program Editor.

However, all of these will work fine, unless XCMD System Option is disabled (NOXCMD).

Assigning batch file permissions when XCMD System Option is disabled

ERROR: Shell escape is not valid in this SAS session.

Bummer! Have you ever seen this error message in SAS Enterprise Guide while trying to run SAS code with the X statement? It indicates that executing OS commands in the SAS environment is not allowed.

In many organizations, IT department policies do not allow enabling the SAS XCMD system option due to cyber-security concerns. This is usually done system-wide for the whole SAS Enterprise client-server installation via SAS configuration.  In this case, no operating system command is allowed to be executed from within SAS.

Of course, this substantially limits SAS’ automation power, but that is the goal and the price to pay for enhanced security.

Still, even without OS command execution at our disposal, we can set Unix script file permissions using FILENAME statement’s PERMISSION= option. Then our above filename statement will look like this:

filename b '/sas/code/project1/program1.sh'
   permission='A::u::rwx,A::g::r-x,A::o::---';

Permission string 'A::u::rwx,A::g::r-x,A::o::---' here signifies the following.
A stands for Access permissions,
u - user who owns the file (owner),
g - group to which the user belongs,
o - other (not the owner or the owner's group).

To grant access permissions, use the values r (Read), w (Write), and x (Execute), in that order. To deny one of these permissions, enter a – in its place (for example, r-x, means Write permission denied).

A::u::rwx means user gets Read, Write, and Execute permissions GRANTED,
A::g::r-x means group gets Read and Execute permissions GRANTED, Write permission DENIED,
A::o::--- means other gets none of the access permissions granted (all of them are DENIED).

However, it is important to realize that your ability to fully control file permissions via FILENAME statement’s PERMISSION= option is still restricted by the Unix umask value set by your IT system administrator. But usually, it is not overly restrictive, at least for the purpose of creating executable files in the environments I have worked with.

The double benefit of the FILENAME statement’s PERMISSION= option is that it can be used for setting up file permissions in any SAS installation whether the XCMD system option is enabled or disabled.

SAS macro to create batch script files

Let’s wrap all the above SAS code pieces into a SAS macro that writes batch scripts. Here is the macro code definition:

%macro write_shell(code);
   %let fdir = %substr(&code,1,%sysfunc(findc(&code,/,b)));
 
   options dlcreatedir;
   libname _flib "&fdir";
   libname _flib;
 
   %let core = %substr(&code,1,%eval(%length(&code)-4));
   filename _fout "&core..sh" permission='A::u::rwx,A::g::r-x,A::o::---'; 
   data _null_;
      file _fout;
      put
         '#!/bin/sh' //
         'now=$(date +%Y.%m.%d_%H.%M.%S)' /
         "pgmname=""&code""" /
         "logname=""&core._$now.log""" /
         'sas $pgmname -log $logname'
         ;
   run;
   filename _fout;
%mend write_shell;

The single macro parameter (code) represents full path name of your SAS code. And here is a macro invocation example:

 
%write_shell(/sas/code/project1/program1.sas)

The assumption here is that the script file gets created in the same directory as the relevant SAS code and SAS logs for each of the batch runs. It will be assigned the same name as your SAS program, only with the .sh name extension. As you can see, we do some string parsing to derive directory name, script file name and SAS log file name from the single macro parameter representing full path name of your SAS code. As an added bonus, if a specified directory (/sas/code/project1/) does not yet exist, it will be created by this macro. DLCREATEDIR System Option (along with the two subsequent libname statements) are responsible for the directory creation.

If you want to create many script files for your multiple SAS programs, you just invoke the macro as many times. You can even go totally data-driven for mass script file creation.

Do you find this useful?

Please let me know in the comments section below if you find this blog post useful. Thank you for reading! I also invite you to share your ideas and experiences on the topic.

Share

About Author

Leonid Batkhan

Leonid Batkhan, Ph.D. in Computer Science and Automatic Control Systems, has been a SAS user for more than 25 years. He came to work for SAS in 1995 and is currently a Senior Consultant with the SAS Federal Data Management and Business Intelligence Practice. During his career, Leonid has successfully implemented dozens of SAS applications and projects in various industries. All posts by Leonid Batkhan >>>

Related Posts

20 Comments

  1. "In many organizations, IT department policies do not allow enabling the SAS XCMD system option due to cyber-security concerns."
    Sorry to disagree. IT departments are believing the default options of SAS are the secure of working. There must be SAS guys having reasons to do this as default.
    The It departments attitude: Do not think or bother further just accept those defaults.

    Please explain why using the power od SAS would be a cybersecuriyt issue when all cybersecurity controls on Linux are in place.
    The result is SAS is harming SAS by some hayman reasoning

    I did the approach of creating many scripts on several occasions.
    It solves the issue of having too big sized data or too many files pretty easy.
    Executing the process can have an excellent perfromance.
    It is way of problem solving that is very unique when undertanding several worlds.

    • Leonid Batkhan

      Hi Jaap, thank you for your feedback. I agree with you in that proper Linux security controls should be enough. However, different organizations have their own IT policies in place and "SAS guys" (administrators, installers) must follow those policies. But I hear what you are saying, sometimes we have to deal with people ("SAS guys" or non-SAS) having unreasonable "reasons" and who are in charge. At least FILENAME PERMISSION= option gives us some advantage without having to deal with them.

  2. Chris Hemedinger
    Chris Hemedinger on

    Leonid, the FILENAME's PERMISSION= option is the golden nugget of this blog post, at least for me. I didn't know about that one!

  3. Michael A. Raithel on

    Leonid,

    Another very useful, very well-written, and fun-to-read article! I am squirreling this one away for when I need this someday.

    Looking forward to your next article(s)!

    ----Michael

  4. Great series of posts. I too was unaware of the permission option!

    Since I'm not experienced in writing shell scripts, recently I've been playing with using a parent SAS program which uses SYSTASK to spawn child SAS sessions, like : systask command """%sysget(SASROOT)\sas.exe"" -noterminal -sysin ""...\myprogram.sas""" ; A benefit of this is instead of learning how to create a timestamp or conditional logic in the shell script language, I can use SAS macro language. I was inspired by Troy Hughes' book, and many of his papers like http://support.sas.com/resources/papers/proceedings17/0870-2017.pdf.

    One use case for this I really like is test scripts. The parent session can have a macro which does some setup of the test environment, then uses systask to call the program being tested, and then confirms that the results of the program match the expectation, log does not have errors etc. Having all of the logic in the parent program as SAS code rather than shell script code helps with portability etc.

  5. Peter Crawford on

    Other positive benefits of launching SAS processes in batch are "repeatability" and "reliability". I found these particularly important in testing.

    • Leonid Batkhan

      Peter, thank you for your feedback. While I agree that batch's "repeatability" and "reliability" are "particularly important in testing", I would add that they are not less important in production.

  6. John Bentley on

    This is great. Thanks for taking the time to share it. I use a lot of scripts to run SAS jobs via Cron. Now I can incorporate the script generating into the SAS Code so everything is documented in one place. Thanks!

Leave A Reply

Back to Top