Blogs

Blogs

How to refactor SAS code to leverage SAS Viya

5
By Topics | Programming Tips

If you're a SAS programmer who now uses SAS Viya and CAS, it's worth your time to optimize your existing programs to take advantage of the new environment. This post is a continuation of my SAS Global Forum 2020 paper Best Practices for Converting SAS® Code to Leverage SAS® Cloud Analytic Services and my SGF 2020 Super Demo.

The best approach for refactoring SAS code for SAS Viya has a few steps:

  • First, "lift and shift" your existing code to run successfully in the compute server for SAS Viya.
  • Next, create CASLIB statements to all of your data sources: i.e. sas7bdat, CSV files, parquet files, relational databases, cloud data sources, etc.
  • Finally, identify the longest running steps so you know where you have the biggest opportunities. For example, look at steps where the "real time" is 30 minutes or longer, as well as steps that are CPU bound. CPU-bound steps are steps where the CPU time is equal to or greater than the real time for that step.

To help us identify those steps we can leverage a new utility to analyze SAS logs and create reports to help us understand the Real Time and CPU Time for each step. Read on to learn more about this final step in the code refactoring process.

Utility: SAS Log Parser

To generate these reports, I created a SAS program that will read SAS Batch Server, SAS Stored Process Server, and SAS Workspace Server logs and create a descending Real Time (Clock Time) report and a step frequency report. In addition to the reports I also save the data set all reports are based on (logs.sas7bdat). All reports and the data set are located in the "reports" directory.

The reports are derived by picking up on SAS log entries like this:

NOTE: PROCEDURE SGPLOT used (Total process time):
      real time           2.79 seconds
      cpu time            0.08 seconds

NOTE: The SAS System used:
      real time           1:08.86
      cpu time            1:18.18

Descending Real Time Report

Figure 1 contains an example of the descending real time report. In this report we observe in the Step column that the longest running step is a PROC LOGISTIC that takes over 14 hours (Real Time column) and from the SAS log called Sample3.log (File Name column). The best way to use this report is to focus on steps that take longer than 30 minutes. In our case we have 9 steps from 3 SAS logs. Now that we know that we can review the details of each step and then benchmark if that step would run faster by leveraging SAS® Cloud Analytic Services (CAS). Note, for CAS to process data, all data must be in CAS tables and the step must be coded using CAS-enabled steps.

Figure 1. Descending Real Time Report

Source code for the SAS Log Parser

I've bundled the SAS code in my GitHub repository for SAS Global Forum 2020. You'll find these programs along with the other programs that supports my previous topics of adapting SAS 9 code for SAS Viya.

SAS Batch, STP, and Workspace Server Logs

For the "SAS 9.4 Batch, STP, and Workspace server logs" download these two files from my GitHub repository:
sasLogParser.sas
sasLogParserMacros.sas. Note, this file must be name sasLogParserMacros.sas and must be in the same directory as sasLogParser.sas. The reason for this is the program sasLogParser.sas includes this file using the name sasLogParserMacros.sas.

Configuration

The only two statements we need to modify are the first two %LET statements in the program sasLogParser.sas. The first %LET statement (line 8) points to the location of the two SAS programs programs you download i.e. sasLogParser.sas and sasLogParserMacros.sas. The second %LET statement (line 12) points to the directory containing SAS logs files with the extension of ".log". The utility will parse that directory and all sub-directories.

Run the Utility in Batch

Due to the size of the descending real-time report always run the utility in batch.

Conclusion

In order to understand which steps are good candidates for leveraging the in-memory engine CAS, we must first understand the real time and CPU time of each step. Then we can benchmark which engine in SAS Viya is appropriate for that step -- the compute server or CAS.  The code that I've shared can run within SAS 9 or SAS Viya on the Windows or Linux platforms.

Tags CAS SAS Programmers SAS Viya
Share

About Author

Steven Sober

Advisory Solutions Architect, Data Management

Steven is responsible for empowering SAS sales and system engineers in the positioning and integration of SAS® Viya, SAS® In-Database Code Accelerator for Hadoop, SAS® Scalable Performance Data Server, and SAS® Grid Manager. During his tenure at SAS, Steven has traveled globally working with customers to quickly integrate the SAS system to automate processes. Before joining SAS in 1989 as the first employee of SAS Switzerland, Steven worked for the U.S. Geological Survey, Water Resources Division, assisting hydrologists to leverage the SAS system to derive intelligence on ground and surface water in Southern Colorado. He holds a Bachelor of Science degree in Computer Science from the University of Southern Colorado in Pueblo, Colorado.

5 Comments

  1. Sanil Divekar on

    Hi,

    I am executing utility sasLogParserStoredProcessServer.sas for stored process server logs. I am getting below error. I have enabled debug options yet I am not able to find out error solution.

    I am providing you initial error message below. Let me know if you need more details from my side.

    -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    NOTE: Character values have been converted to numeric values at the places given by: (Line):(Column).
    298:156 298:61 298:240
    NOTE: The infile SASLOG is:
    Filename=/sasvol01/Temp/Utility/Stored/Log/SASApp_STPServer_2021-05-04_punevmsinsdi07_11446.log,
    Owner Name=sasecm,Group Name=ecmgrp,
    Access Permission=-rwxrwxrwx,
    Last Modified=05 May 2021 00:05:05,
    File Size (bytes)=83683

    NOTE: Invalid third argument to function SUBSTR at line 298 column 134.
    RULE: ----+----1----+----2----+----3----+----4----+----5----+----6----+----7----+----8----+----9----+----0
    1 Host: 'punevmsinsdi07', OS: 'LIN X64', Release: '3.10.0-514.el7.x86_64', SAS Version: '9.04.01M5P091
    101 32017', Command: '/sasvol01/SASHome1/SASFoundation/9.4/sasexe/sas -noterminal -noxcmd -netencryptalg
    201 orithm SASProprietary -metaserver punevmsinsdi07.in.sas.com -metaport 8562 -metarepository Foundatio
    301 n -objectserver -objectserverparms "protocol=bridge spawned spp=43245 cid=1 dnsmatch=punevmsinsdi07.
    401 in.sas.com pb classfactory=15931E31-667F-11D5-8804-00C04F35AC8C server=OMSOBJ:SERVERCOMPONENT/A5S8KX
    501 6N.AZ000007 cel=credentials lb multiuser port=8612 saslangrunas=client"' 572
    oldline= line=-514.el7.x86_64', SAS Version: '9.04.01M5P091 step= product=SAS9Log fileName=/sasvol01/Temp/Utility/Stored/Log/SASApp_STPServer_2021-05-04_punevmsinsdi07_11446.log realtime=. cputime=. totaltime=. totalcpu=. arg1=514 arg2=el7 x=.
    7 The SAS System 14:53 Tuesday, October 5, 2021

    time= _ERROR_=1 _N_=1
    NOTE: Invalid third argument to function SUBSTR at line 298 column 134.
    RULE: ----+----1----+----2----+----3----+----4----+----5----+----6----+----7----+----8----+----9----+----0
    2 Log continued from /sasvol01/SASConfigBOB/Lev2/SASApp/StoredProcessServer/Logs/SASApp_STPServer_2021
    101 -05-03_punevmsinsdi07_11446.log 131
    oldline= line=toredProcessServer/Logs/SASApp_STPServer_2021 step= product=SAS9Log fileName=/sasvol01/Temp/Utility/Stored/Log/SASApp_STPServer_2021-05-04_punevmsinsdi07_11446.log realtime=. cputime=. totaltime=. totalcpu=. arg1=toredProcessServer
    arg2=Logs x=. time= _ERROR_=1 _N_=2
    NOTE: Invalid third argument to function SUBSTR at line 298 column 134.
    3 2021-05-04T00:05:05,707 DEBUG [00000141] :sassrvecm1 - STP: Session Monitor Starting Purge. 91
    oldline= line=STP: Session Monitor Starting Purge. step= product=SAS9Log fileName=/sasvol01/Temp/Utility/Stored/Log/SASApp_STPServer_2021-05-04_punevmsinsdi07_11446.log realtime=. cputime=. totaltime=. totalcpu=. arg1=STP: arg2=Session x=. time=
    _ERROR_=1 _N_=3
    -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    Thank You,
    Sanil Divekar

    Reply
      • Steven Sober
        Steven Sober on

        @John Ward
        First off thank you for the heads up on the link being broken.

        Instructions:
        1. download the two programs for processing Work Space Server logs to a new directory.
        2. Modify line 8 and 12 in the sasLogParserWorkSpaceServer.sas file:
        /* sasLogParserWorkSpaceServer.sas version 3.14 24Jun2021:11:49 */
        /* Macro variable for the path to sas log parser directory */
        /* Note: Ensure the path ends with the delimiter \ for Windows or / for Linux */
        %let sasLogParser=C:\path\to\utility\;

        /* Macro varialbe for the path containing the SAS Logs */
        /* Note: Ensure the path ends with the delimiter \ for Windows or / for Linux */
        %let path2files=C:\path\to\logs\;

        3. Submit to SAS sasLogParserWorkSpaceServer.sas

        The utility will create a "reports" directory in the directory you copied the two SAS programs to. When the utility finishes go into the "reports" directory to review the reports.

        The utility is recursive so if there are subdirectory in the path you set on line 12 it will parse all ".log" files in those subdirectories too.

        If you need additional assistance please send me an email at steven.sober@sas.com and I will set up a Teams meeting to help you run the utility.

        Best regards,
        Steve

        Reply

Leave A Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Back to Top