How much time will your process flow take to run?

51

SAS users, by definition, do not embrace the mysterious.

That's one of the main reasons that they use SAS: to demystify some data or process. And so, when you (as a SAS user) have gone to the trouble of designing a process flow in SAS Enterprise Guide, you like to be aware of some basic metrics, such as "how long will it take to run?"

It's difficult to predict how long a SAS process will take to run, as it depends more upon the data than on the actual program instructions. But one thing that we're very good at is telling you how long it took to run the last time that you ran it. In SAS Enterprise Guide, you can find this information at the task level by right-clicking on the task (or program node) and selecting Properties. On the General tab, you'll see the "Last execution time".

How long did the task take?

If you want to gather this information at a process flow or project level, you can repeat these steps for each item in the flow, make a note of the "execution time", then add up the numbers (expressed in hours, minutes, and seconds) to create a grand total. This tedious assignment makes for a perfect torture device for a summer intern who, in this economy, should be grateful to have a job at all.

Or, to make the job less tedious, you could use the Project Reviewer task. This is a custom task (available for download here) that shows a summary view of your process flows and allows you to create a report from the information. The task works with SAS Enterprise Guide 4.3 and 5.1.

Project Reviewer screenshot

Features of the Project Reviewer task include:
1. A selection list with all of the process flows within your project.

2. A list of each "runnable" task; that is, program, task, query, export step, etc. Each task has an "ordinal" (its sequence in the process flow), a name, a descriptive type, the user ID of the last person to modify it, the running time for its most recent iteration, the date/time modified, the date/time created, and whether the task generated errors ("red X" in the flow).

You can sort the items in the list by clicking on the column header for the value you want to sort. Click on the column header again to reverse the sort sequence.

3. A summary of the task count, and total clock time that "running" the flow represents.

4. A Create Report button, which generates a SAS program to produce a simple report of all of the project contents, summarized by each process flow. If you have multiple SAS environments, you can use the Report server list to select which SAS server to use when processing the report.

Here's a sample of the report output:
Sample project reviewer report
The reporting process also generates a SAS program and a data set (which are added to your project), so that you can easily adapt these for custom reports.

Let me know if you find this task to be useful and whether you have any improvements to suggest.

Some final notes/links:

Share

About Author

Chris Hemedinger

Director, SAS User Engagement

+Chris Hemedinger is the Director of SAS User Engagement, which includes our SAS Communities and SAS User Groups. Since 1993, Chris has worked for SAS as an author, a software developer, an R&D manager and a consultant. Inexplicably, Chris is still coasting on the limited fame he earned as an author of SAS For Dummies

51 Comments

    • Chris Hemedinger
      Chris Hemedinger on

      Inside information. Frowned upon for securities trading, but definitely an advantage in software development.

  1. Sorry Chris - I'm running 5.1 and this task did not appear automatically when added to the Custom folder (whch I had to create in the designated location). I tried to add it via the Add-in Manager and got message "Unable to load file...Error: Could not load file or assembly 'SAS.Tasks.ProjectReviewer' or one of its dependencies. Operation is not supported. (Exception from HRESULT..) I do not have Admin rights and cannot make changes to the registry - is this the problem?

    • Chris Hemedinger
      Chris Hemedinger on

      Hi Bob, I'm not sure what's going on here, but you might be able to get some diagnostics by enabling additional logging. Follow the instructions in this SAS Note SN-17730 (applying the operations to the 5.1 folders). You'll end up with a text log in %appdata%\SAS\EnterpriseGuide\5.1\Logs, and it will show some activity where the DLL was examined for inclusion on the Tools->Add-in menu. You might also try a different custom task (see response to your other comment) to make sure the mechanism is working for you in general.

      • Thanks Chris - seems to be a local .net problem:

        An attempt was made to load an assembly from a network location which would have caused the assembly to be sandboxed in previous versions of the .NET Framework. This release of the .NET Framework does not enable CAS policy by default, so this load may be dangerous. If this load is not intended to sandbox the assembly, please enable the loadFromRemoteSources switch. See http://go.microsoft.com/fwlink/?LinkId=155569 for more information.

        • Chris Hemedinger
          Chris Hemedinger on

          Bob, good to hear you've got a handle on it. I can see this happening if you put the DLL on a mapped drive or UNC path, or perhaps if your %appdata% location maps to something other than your local machine (perhaps a roaming profile). But you can probably workaround it by creating a "Custom" folder in your SAS Enterprise Guide app directory (with an admin's help or if you have permissions) and then placing the DLL in there.

  2. You're so awesome! I do not believe I've read a single thing like this before. So good to discover somebody with a few genuine thoughts on this topic. Really.. thanks for starting this up. This site is one thing that is required on the internet, someone with some originality!

    • Chris Hemedinger
      Chris Hemedinger on

      Willie Mae,

      Thank you for your over-the-top response. Despite the fact that your comment contains no specific reference to the content of this post, I shouldn't assume that this is the work of a spam-bot, should I? Instead, I choose to believe that your comment is sincere and, of course, completely justified in its enthusiasm.

      I hope that you don't mind that I removed the trackback link -- I didn't want my large readership to overload your website with traffic.

  3. Chris,

    Is there any features in SAS 9.2 for Unix that allows for similar report creation? I have used scripts to pull cpu time and real time counts out of the log file, but this would be easier.

    Thanks -

    • Chris Hemedinger
      Chris Hemedinger on

      Bo,

      Of course, if your process was driven through SAS Enterprise Guide, this task would work for that as well, even with the SAS processing on UNIX. Aside from that, you might look at the SAS Logging Facility -- which maybe you already use. You would still need to script the extraction of the timings to create a report, I think.

  4. Pingback: My favorite SAS tips and tools from 2012 - The SAS Dummy

  5. Pingback: Upcoming SAS Talks: Custom tasks for SAS Enterprise Guide - The SAS Dummy

  6. Pingback: Custom tasks for SAS Enterprise Guide: Q&A - The SAS Dummy

  7. Hi Chris, you've got a great site! I would like to user your Project Reviewer custom task but ran into trouble. We are running SAS on a UNIX server, with EG running on Windows clients. I tried using the Add In Manager in SAS EG 4.3 to bring in the DLL, but got an error. Does our SAS Admin first need to create a "custom" folder in a location that only he can access and we users cannot? Any other instructions relevant to UNIX?

    The error was: "Unable to load file "...." Error: Could not load file or assembly 'SAS_Tasks_ProjectReviewer.dll' or one of its dependencies. The located assembly's manifest destination does not match the assembly reference. (Except from HRESULT: 0x80131040)

    Thanks so much !
    --Mark

  8. This looks useful... Thanks
    When I run the report, I see that the data is entered through the datalines statement
    Where does that data originate?
    Can I get it programmatically?
    Maybe through system/macro variables?

    • Chris Hemedinger
      Chris Hemedinger on

      That data is stored internally in the EG project. The task does the work of pulling it out and creating the DATALINES records. Unfortunately, there isn't a SAS program method to pull out this same information.

      • Pascal Maurice on

        Hi Chris,
        I've been looking for the informations we can see in the properties of a program within SASEGuide, like Last Execution Time, and I do not find it in any property of the SAS EG Scripting objects. Can you help me find it ? to be able to enhance a project reviewer...
        Thank you very much

        • Chris Hemedinger
          Chris Hemedinger on

          Pascal, there isn't an exposed property for that. You can perhaps glean from the Log.Text, but that's not as neat as a property would be.

          • Pascal Maurice on

            Thank you Chris,
            could I put it in the shopping list for Santa ?
            PS: I hope it is stored somewhere in the project...
            have a nice day

          • Chris Hemedinger
            Chris Hemedinger on

            Pascal, I'll put in a word to the Big Man, but you should have a backup gift request. How about a nice football?

            Yes, the information is in the project, but not exposed through any public/documented property. That might inspire you to go spelunking; use caution when straying from the documented APIs.

  9. Hi Chris,

    I've the SAS 9.4 with EG 6.1 which is on x64 and the selecte process flow is blank.

    Here for more detail:

    Exception Details:
    ----------------------------------------
    Exception type: System.IO.FileNotFoundException
    Message: Impossible de charger le fichier ou l'assembly 'SAS.EG.ProjectElements, Version=4.3.0.0, Culture=neutral, PublicKeyToken=be58efc3b934219b' ou une de ses dépendances. Le fichier spécifié est introuvable.
    Source: SAS.Tasks.ProjectReviewer

    • Chris Hemedinger
      Chris Hemedinger on

      Hugo,

      I've made a small adjustment that should allow it to work in EG 6.1. Can you download another copy and replace the previous version? The new DLL will have today's date (21Jan2014) and a file version of 4.3.0.14021.

  10. Hi Chris,

    I discovered your blog and it contains a lot of useful information.
    This project reviewer task is really interesting. But I was wondering, would it be possible to also get other information such as CPU time?
    My point is that I'm trying to optimize my code and I would like to see the impact of the optimization. However, the SAS server load has a big impact on the real time. So that I can not really compare two different runs of my project.

    Again thank you for your post

    Seb

    • Chris Hemedinger
      Chris Hemedinger on

      Seb, it sounds like you want the metrics available from the FULLSTIMER option. This SAS system option emits many useful measurements into the log. However, SAS Enterprise Guide doesn't aggregate these for you, so if you enable the option in your code, you'll need to peruse the SAS logs in your project to see the measurements and the effects of your optimizations.

  11. Hi Chris,

    I'd like to use your custom task "Project Reviewer" for the EG 7.1 x64 with SAS 9.4, but I always get the following exception message. The File"SAS.EG.ProjectElements" has the version 7.100.0.2002. Can you tell me if I am making a mistake? Or could you perhaps update the latest version of your custom task?

    have a nice day - Dennis Voigt

    Exception Details:
    ----------------------------------------
    Exception type: System.IO.FileNotFoundException
    Message: Die Datei oder Assembly "SAS.EG.ProjectElements, Version=4.3.0.0, Culture=neutral, PublicKeyToken=be58efc3b934219b" oder eine Abhängigkeit davon wurde nicht gefunden. Das System kann die angegebene Datei nicht finden.
    Source: SAS.Tasks.ProjectReviewer
    Target Site: .ctor

    • Chris Hemedinger
      Chris Hemedinger on

      Dennis,

      I'm sorry that you hit this problem. I have built a special new version for SAS Enterprise Guide 7.1 and added it to the download package. Note that the new version has "71" in the name of the DLL. When you install it, make sure that any previous version (SAS.Tasks.ProjectReviewer.dll) is NOT in the path or already installed. This would cause a conflict that prevents the 7.1 version from loading.

  12. Hi Chris, as always your tips & tricks are very helpful. I had a somewhat related question if you don't mind. My group just upgraded from EG 4.3 to 6.1 in a Grid architecture and one thing I noticed missing is how the log use to display the execution time of each step, even without STIMER. I don't have access to a copy of 4.3 so I'm not sure if there was a setting or what, but I'd love to get this information back in the log without having to add STIMER to everything. thx

  13. Pingback: Project Reviewer task and Copy Files task: refreshed for v7.1 - The SAS Dummy

  14. John Lonsdale on

    Hi Chris,

    love this add-in and have installed it to all our machines.

    I did have one problem when trying to install on the first machine. I got a HRESULT 0x08.... which was solved by opening the file properties in windows explorer and clicking on 'Unblock this file'

    thanks John

  15. Hi Chris really loved his post. Can you give me a link to the source code of the dll or help me through the steps?

    • Chris Hemedinger
      Chris Hemedinger on

      Shubham,

      The source code for this project uses a tiny bit of unpublished APIs, so I haven't shared it out. However, you can achieve much of this with the ISASProject APIs that are part of the custom task APIs. The SAS Task Property Viewer task uses those same APIs, and I shared that as an example in my Custom Tasks book.

  16. Andrea Magatti on

    Problem using EG 7.1.2 X64

    Message is:
    Missingl file or 'assembly 'SAS.EG.ProjectElements, Version=7.100.1.0, Culture=neutral, PublicKeyToken=be58efc3b934219b'

    Seems like a version mismatch. Any option for 7.1.2?

    thanks for your precious work

    • Chris Hemedinger
      Chris Hemedinger on

      Hi Andrea, I had to build a special version for 7.12. I've just placed it in the ZIP package. Re-download and extract the version with the *712 suffix, and REMOVE your old versions of the task DLL.

      • We are running the latest 7.13 version (7.100.3.5408) (64-bit) and would request that a similar DLL solution be added to the ZIP package for that newest version. Tests against the 3 currently supplied DLL files all failed with our current configuration. Thanks Chris!!

        • Chris Hemedinger
          Chris Hemedinger on

          Kevin -- it was on my to-do list, but your comment moved it to the top :)

          This is now done -- the updated package has a 713 version. Remove all other versions and use this instead, if you're on EG 7.13.

  17. Resa Drijsen on

    Hi Chris,
    Came across your tool recently and tried to add it to my EG installation: SAS Enterprise Guide 64-bit version: 7.13 HF5 (7.100.3.5486)
    As per instructions I copied the SAS.Tasks.ProjectReviewer713.dll to the designated folder. However, the Project Reviewer task did not appear.
    When I tried to add it with the add-in manager I get the following error:
    Unable to load file /SAS.Tasks.ProjectReviewer713.dll
    Error: Could not load file or assembly 'SAS.Tasks.ProjectReviewer713' or one of its dependencies. Operation is not supported. (Exception from HRESULT: 0x80131515)

    What am I doing wrong?

    • Chris Hemedinger
      Chris Hemedinger on

      Resa, This is the security “sandbox” that Windows imposes on DLLs that you download from the Internet. You need to Unblock this DLL with these instructions. Follow the point-and-click unblocking step, or use the streams.exe utility referenced in the post. Don't use the Add-In Manager -- that's probably not going to work for this one.

      • Resa Drijsen on

        Hi Chris, thanks for the reply and unblocking the DLL resolved the issue and the add-in works like a charm. Kudos.
        I have often told others, now it is time to tell myself to ... RTFM ;-)

    • Chris Hemedinger
      Chris Hemedinger on

      I haven't yet built an EG 8.1 version. Fortunately, EG 8.1 does include several enhancements to the Task Status window, so you can see the history of tasks/programs that you've submitted. But I think the report that the task generates is useful, so I'll look at updating it.

    • Chris Hemedinger
      Chris Hemedinger on

      I've built an 8.1 and 8.2 version for this task. Instructions for installing are the same, just drop into the %appdata%\SAS\EnterpriseGuide\8\Custom folder. Install just the one version that works with the version of EG that you have.

    • Chris Hemedinger
      Chris Hemedinger on

      There is a version of the task in the ZIP that works with EG 8.3, yes. It's good for 32-bit and 64-bit.

      • Angie Andrews on

        Chris, I followed the instructions and added it to %appdata%\SAS\EnterpriseGuide\8\Custom. I added it in Add_In Manager, but don't know how to implement it. It was really easy to point and click on Tools > Add-In > Project Reviewer, but I don't see Tools > Add-In anymore, just the Add_In Manager.

        • Chris Hemedinger
          Chris Hemedinger on

          In EG 8.3 the Custom Tasks don't appear in the Tools menu. You'll find this in the Task browser -- just search for "Project"

Back to Top