If Necessity is the mother of Invention, then, perhaps, the father of Automation is Laziness. Automation is all about convenience, comfort, and productivity. Why do it yourself if you can devise something to do it for you!See also: How to conditionally terminate a SAS batch flow process in UNIX/Linux
In my previous post Running SAS programs in batch under Unix/Linux, we learned that in order to automate SAS jobs submissions, they are often run in batch mode. We also learned that we usually create batch scripts as a convenient way to run SAS programs in batch. To create a unique SAS log file generated with each batch submission, a typical batch script may look like follows:
#!/usr/bin/sh dtstamp=$(date +%Y.%m.%d_%H.%M.%S) pgmname="/sas/code/project1/program1.sas" logname="/sas/code/project1/program1_$dtstamp.log" /sas/SASHome/SASFoundation/9.4/sas $pgmname -log $logname
It will allow you to submit your SAS program /sas/code/project1/program1.sas in batch, and also capture SAS log file with a convenient date-time suffix in the same directory.
SAS program to write batch scripts
But what if we are deploying multiple SAS programs? Well, then we would need to create a batch script for each of them. They will all look similar to each other, and that is when most human errors usually occur – when we do something similar, monotonously, over and over again. Besides, I found working with the Unix Visual Editor (“vi editor”) is not quite a 21st century experience.
What would a normal SAS programmer do in such a situation? That’s right – we would write a SAS program to write a batch script file! Let’s do it. Let’s automate the automation.
In its simplest form, to replicate the above batch script example our SAS program would look like this:
filename b '/sas/code/project1/program1.sh'; data _null_; file b; input; put _infile_; datalines; #!/usr/bin/sh dtstamp=$(date +%Y.%m.%d_%H.%M.%S) pgmname="/sas/code/project1/program1.sas" logname="/sas/code/project1/program1_$dtstamp.log" /sas/SASHome/SASFoundation/9.4/sas $pgmname -log $logname ;
Setting up batch file permissions
As we already know from my previous post, we need to assign certain permissions to our batch file in order to make it executable. For example, if you want to give yourself (Owner) and Group execution permissions then your script file permissions can be as:
-rwxr-x---, or 750 in octal representation.
In order to do that you can to add to your SAS code the following x-statement:
options noxwait; x 'chmod 750 /sas/code/project1/program1.sh';
When you create a batch script by running the above code in SAS Enterprise Guide (EG), you don’t have to leave the comfort of your SAS environment or even touch Unix vi editor. Moreover, you can even submit your SAS job in batch mode right from your SAS EG Program Editor.
However, all of these will work fine, unless XCMD System Option is disabled (NOXCMD).
Assigning batch file permissions when XCMD System Option is disabled
ERROR: Shell escape is not valid in this SAS session.
Bummer! Have you ever seen this error message in SAS Enterprise Guide while trying to run SAS code with the X statement? It indicates that executing OS commands in the SAS environment is not allowed.
In many organizations, IT department policies do not allow enabling the SAS XCMD system option due to cyber-security concerns. This is usually done system-wide for the whole SAS Enterprise client-server installation via SAS configuration. In this case, no operating system command is allowed to be executed from within SAS.
Of course, this substantially limits SAS’ automation power, but that is the goal and the price to pay for enhanced security.
Still, even without OS command execution at our disposal, we can set Unix script file permissions using FILENAME statement’s PERMISSION= option. Then our above filename statement will look like this:
filename b '/sas/code/project1/program1.sh' permission='A::u::rwx,A::g::r-x,A::o::---';
Permission string 'A::u::rwx,A::g::r-x,A::o::---' here signifies the following.
A stands for Access permissions,
u - user who owns the file (owner),
g - group to which the user belongs,
o - other (not the owner or the owner's group).
To grant access permissions, use the values r (Read), w (Write), and x (Execute), in that order. To deny one of these permissions, enter a – in its place (for example, r-x, means Write permission denied).
A::u::rwx means user gets Read, Write, and Execute permissions GRANTED,
A::g::r-x means group gets Read and Execute permissions GRANTED, Write permission DENIED,
A::o::--- means other gets none of the access permissions granted (all of them are DENIED).
However, it is important to realize that your ability to fully control file permissions via FILENAME statement’s PERMISSION= option is still restricted by the Unix umask value set by your IT system administrator. But usually, it is not overly restrictive, at least for the purpose of creating executable files in the environments I have worked with.
The double benefit of the FILENAME statement’s PERMISSION= option is that it can be used for setting up file permissions in any SAS installation whether the XCMD system option is enabled or disabled.
SAS macro to create batch script files
Let’s wrap all the above SAS code pieces into a SAS macro that writes batch scripts. Here is the macro code definition:
%macro write_shell(code); %let fdir = %substr(&code,1,%sysfunc(findc(&code,/,b))); options dlcreatedir; libname _flib "&fdir"; libname _flib; %let core = %substr(&code,1,%eval(%length(&code)-4)); filename _fout "&core..sh" permission='A::u::rwx,A::g::r-x,A::o::---'; data _null_; file _fout; put '#!/bin/sh' // 'now=$(date +%Y.%m.%d_%H.%M.%S)' / "pgmname=""&code""" / "logname=""&core._$now.log""" / 'sas $pgmname -log $logname' ; run; filename _fout; %mend write_shell;
The single macro parameter (code) represents full path name of your SAS code. And here is a macro invocation example:
The assumption here is that the script file gets created in the same directory as the relevant SAS code and SAS logs for each of the batch runs. It will be assigned the same name as your SAS program, only with the .sh name extension. As you can see, we do some string parsing to derive directory name, script file name and SAS log file name from the single macro parameter representing full path name of your SAS code. As an added bonus, if a specified directory (/sas/code/project1/) does not yet exist, it will be created by this macro. DLCREATEDIR System Option (along with the two subsequent libname statements) are responsible for the directory creation.
If you want to create many script files for your multiple SAS programs, you just invoke the macro as many times. You can even go totally data-driven for mass script file creation.
Do you find this useful?
Please let me know in the comments section below if you find this blog post useful. Thank you for reading! I also invite you to share your ideas and experiences on the topic.See also: How to conditionally terminate a SAS batch flow process in UNIX/Linux