How much time will your process flow take to run?

SAS users, by definition, do not embrace the mysterious.

That's one of the main reasons that they use SAS: to demystify some data or process. And so, when you (as a SAS user) have gone to the trouble of designing a process flow in SAS Enterprise Guide, you like to be aware of some basic metrics, such as "how long will it take to run?"

It's difficult to predict how long a SAS process will take to run, as it depends more upon the data than on the actual program instructions. But one thing that we're very good at is telling you how long it took to run the last time that you ran it. In SAS Enterprise Guide, you can find this information at the task level by right-clicking on the task (or program node) and selecting Properties. On the General tab, you'll see the "Last execution time".

How long did the task take?

If you want to gather this information at a process flow or project level, you can repeat these steps for each item in the flow, make a note of the "execution time", then add up the numbers (expressed in hours, minutes, and seconds) to create a grand total. This tedious assignment makes for a perfect torture device for a summer intern who, in this economy, should be grateful to have a job at all.

Or, to make the job less tedious, you could use the Project Reviewer task. This is a custom task (available for download here) that shows a summary view of your process flows and allows you to create a report from the information. The task works with SAS Enterprise Guide 4.3 and 5.1.

Project Reviewer screenshot

Features of the Project Reviewer task include:
1. A selection list with all of the process flows within your project.

2. A list of each "runnable" task; that is, program, task, query, export step, etc. Each task has an "ordinal" (its sequence in the process flow), a name, a descriptive type, the user ID of the last person to modify it, the running time for its most recent iteration, the date/time modified, the date/time created, and whether the task generated errors ("red X" in the flow).

You can sort the items in the list by clicking on the column header for the value you want to sort. Click on the column header again to reverse the sort sequence.

3. A summary of the task count, and total clock time that "running" the flow represents.

4. A Create Report button, which generates a SAS program to produce a simple report of all of the project contents, summarized by each process flow. If you have multiple SAS environments, you can use the Report server list to select which SAS server to use when processing the report.

Here's a sample of the report output:
Sample project reviewer report
The reporting process also generates a SAS program and a data set (which are added to your project), so that you can easily adapt these for custom reports.

Let me know if you find this task to be useful and whether you have any improvements to suggest.

Some final notes/links:

tags: process flows, SAS custom tasks, SAS Enterprise Guide

23 Comments

  1. Posted May 15, 2012 at 12:15 pm | Permalink

    I don't know how you figure out all these cool tricks.

    • Chris Hemedinger Chris Hemedinger
      Posted May 15, 2012 at 12:17 pm | Permalink

      Inside information. Frowned upon for securities trading, but definitely an advantage in software development.

  2. Bob
    Posted May 16, 2012 at 3:59 am | Permalink

    Are there any other add-ins for EG5.1?

  3. Bob
    Posted May 16, 2012 at 4:10 am | Permalink

    Sorry Chris - I'm running 5.1 and this task did not appear automatically when added to the Custom folder (whch I had to create in the designated location). I tried to add it via the Add-in Manager and got message "Unable to load file...Error: Could not load file or assembly 'SAS.Tasks.ProjectReviewer' or one of its dependencies. Operation is not supported. (Exception from HRESULT..) I do not have Admin rights and cannot make changes to the registry - is this the problem?

    • Chris Hemedinger Chris Hemedinger
      Posted May 16, 2012 at 8:30 am | Permalink

      Hi Bob, I'm not sure what's going on here, but you might be able to get some diagnostics by enabling additional logging. Follow the instructions in this SAS Note SN-17730 (applying the operations to the 5.1 folders). You'll end up with a text log in %appdata%\SAS\EnterpriseGuide\5.1\Logs, and it will show some activity where the DLL was examined for inclusion on the Tools->Add-in menu. You might also try a different custom task (see response to your other comment) to make sure the mechanism is working for you in general.

      • Bob
        Posted May 22, 2012 at 7:01 am | Permalink

        Thanks Chris - seems to be a local .net problem:

        An attempt was made to load an assembly from a network location which would have caused the assembly to be sandboxed in previous versions of the .NET Framework. This release of the .NET Framework does not enable CAS policy by default, so this load may be dangerous. If this load is not intended to sandbox the assembly, please enable the loadFromRemoteSources switch. See http://go.microsoft.com/fwlink/?LinkId=155569 for more information.

        • Chris Hemedinger Chris Hemedinger
          Posted May 22, 2012 at 8:00 am | Permalink

          Bob, good to hear you've got a handle on it. I can see this happening if you put the DLL on a mapped drive or UNC path, or perhaps if your %appdata% location maps to something other than your local machine (perhaps a roaming profile). But you can probably workaround it by creating a "Custom" folder in your SAS Enterprise Guide app directory (with an admin's help or if you have permissions) and then placing the DLL in there.

  4. willie mae
    Posted May 17, 2012 at 1:05 am | Permalink

    You're so awesome! I do not believe I've read a single thing like this before. So good to discover somebody with a few genuine thoughts on this topic. Really.. thanks for starting this up. This site is one thing that is required on the internet, someone with some originality!

    • Chris Hemedinger Chris Hemedinger
      Posted May 17, 2012 at 9:03 am | Permalink

      Willie Mae,

      Thank you for your over-the-top response. Despite the fact that your comment contains no specific reference to the content of this post, I shouldn't assume that this is the work of a spam-bot, should I? Instead, I choose to believe that your comment is sincere and, of course, completely justified in its enthusiasm.

      I hope that you don't mind that I removed the trackback link -- I didn't want my large readership to overload your website with traffic.

  5. Bo
    Posted May 17, 2012 at 1:51 pm | Permalink

    Chris,

    Is there any features in SAS 9.2 for Unix that allows for similar report creation? I have used scripts to pull cpu time and real time counts out of the log file, but this would be easier.

    Thanks -

    • Chris Hemedinger Chris Hemedinger
      Posted May 17, 2012 at 1:59 pm | Permalink

      Bo,

      Of course, if your process was driven through SAS Enterprise Guide, this task would work for that as well, even with the SAS processing on UNIX. Aside from that, you might look at the SAS Logging Facility -- which maybe you already use. You would still need to script the extraction of the timings to create a report, I think.

  6. Mark Luders
    Posted August 16, 2013 at 12:53 pm | Permalink

    Hi Chris, you've got a great site! I would like to user your Project Reviewer custom task but ran into trouble. We are running SAS on a UNIX server, with EG running on Windows clients. I tried using the Add In Manager in SAS EG 4.3 to bring in the DLL, but got an error. Does our SAS Admin first need to create a "custom" folder in a location that only he can access and we users cannot? Any other instructions relevant to UNIX?

    The error was: "Unable to load file "...." Error: Could not load file or assembly 'SAS_Tasks_ProjectReviewer.dll' or one of its dependencies. The located assembly's manifest destination does not match the assembly reference. (Except from HRESULT: 0x80131040)

    Thanks so much !
    --Mark

  7. Alex Goldman
    Posted November 8, 2013 at 10:13 am | Permalink

    This looks useful... Thanks
    When I run the report, I see that the data is entered through the datalines statement
    Where does that data originate?
    Can I get it programmatically?
    Maybe through system/macro variables?

    • Chris Hemedinger Chris Hemedinger
      Posted November 8, 2013 at 10:28 am | Permalink

      That data is stored internally in the EG project. The task does the work of pulling it out and creating the DATALINES records. Unfortunately, there isn't a SAS program method to pull out this same information.

      • Pascal Maurice
        Posted December 19, 2013 at 8:57 am | Permalink

        Hi Chris,
        I've been looking for the informations we can see in the properties of a program within SASEGuide, like Last Execution Time, and I do not find it in any property of the SAS EG Scripting objects. Can you help me find it ? to be able to enhance a project reviewer...
        Thank you very much

        • Chris Hemedinger Chris Hemedinger
          Posted December 19, 2013 at 9:03 am | Permalink

          Pascal, there isn't an exposed property for that. You can perhaps glean from the Log.Text, but that's not as neat as a property would be.

          • Pascal Maurice
            Posted December 19, 2013 at 9:19 am | Permalink

            Thank you Chris,
            could I put it in the shopping list for Santa ?
            PS: I hope it is stored somewhere in the project...
            have a nice day

          • Chris Hemedinger Chris Hemedinger
            Posted December 19, 2013 at 9:24 am | Permalink

            Pascal, I'll put in a word to the Big Man, but you should have a backup gift request. How about a nice football?

            Yes, the information is in the project, but not exposed through any public/documented property. That might inspire you to go spelunking; use caution when straying from the documented APIs.

  8. Evgeniy
    Posted January 17, 2014 at 8:38 am | Permalink

    Great tool, thank you very much for it!

  9. hugo
    Posted January 21, 2014 at 1:31 pm | Permalink

    Hi Chris,

    I've the SAS 9.4 with EG 6.1 which is on x64 and the selecte process flow is blank.

    Here for more detail:

    Exception Details:
    ----------------------------------------
    Exception type: System.IO.FileNotFoundException
    Message: Impossible de charger le fichier ou l'assembly 'SAS.EG.ProjectElements, Version=4.3.0.0, Culture=neutral, PublicKeyToken=be58efc3b934219b' ou une de ses dépendances. Le fichier spécifié est introuvable.
    Source: SAS.Tasks.ProjectReviewer

    • Chris Hemedinger Chris Hemedinger
      Posted January 21, 2014 at 1:58 pm | Permalink

      Hugo,

      I've made a small adjustment that should allow it to work in EG 6.1. Can you download another copy and replace the previous version? The new DLL will have today's date (21Jan2014) and a file version of 4.3.0.14021.

3 Trackbacks

  1. [...] Project Reviewer task - shows all of the tasks in your project, grouped by process flow. It summarizes the time elapsed by each task and each process flow, who last modified the tasks and when, and more. And you can automatically include this as a report within your project. [...]

  2. [...] new analysis and reports, productivity tools for programmers and utilities that help you to manage your projects. Many of these tasks also work with the SAS Add-In for Microsoft [...]

  3. [...] Project Reviewer [...]

Post a Comment

Your email is never published nor shared. Required fields are marked *

*
*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <p> <pre lang="" line="" escaped=""> <q cite=""> <strike> <strong>