The SAS Dummy - A SAS blog for the rest of us.

A customer posted on the discussion forum that, much his dismay, SAS Enterprise Guide sets the NOFMTERR option automatically when connecting to a SAS session.

The FMTERR|NOFMTERR option specifies whether SAS should report an error when you attempt to reference a data column that has a SAS format applied, but the format definition cannot be found.

SAS Enterprise Guide has always set NOFMTERR, even though it's not the default setting for the option in SAS. Why? Because if you attempt to open data that references a SAS format that SAS cannot find, not only will you be unable to open the data, but you'll get a pop-up message that says something like "Format VAL. cannot be found". Because many SAS Enterprise Guide users are new to SAS, and that message is confusing at best, we erred on the side of allowing you to open the data even without the format. This decision was made in the very early days of the product over 10 years ago, and we've had no suggestions to change it. Then again, we didn't really tell people we were doing it.

In SAS Enterprise Guide 4.2 we made it easier to see exactly what's happening when SAS initializes. No more secrets! Here are the steps:

1. Select View->Server List
   
2. In the Server list, expand the server you are connecting to (ex: "SASApp" or "Local")
   
3. Right-click on the server name and select Properties.
   
4. Click on the Software tab.
   
5. Click on View Initialization Log.
   

You'll see a SAS log window that contains the same startup log that you might know from running SAS interactively (including the NEWS= content, if any!).

I'll tell you right now: one other system option that SAS Enterprise Guide submits is VALIDVARNAME=ANY, which makes it much easier to import those spreadsheet files with funny column names.

If you want to override these options at startup, that's easy to do by providing your own initialization statements. Check out this sasCommunity.org tip for instructions.

When you create SAS Enterprise Guide projects, what's your approach? Do you cram the process flow with thousands of nodes? Do you create different projects for each work task? Do you like one big SAS program? Or a bunch of little programs all linked together?

That's the question that Michael is asking in the SAS Enterprise Guide discussion forum.

What have you learned about organizing your SAS work in SAS Enterprise Guide? Go share your thoughts with Michael (and the world), on the forum...

SAS programmers can often be (um...let's see...how best to put this...) set in their ways -- at least when it comes to their SAS work.

This is due in part to the nature of SAS. The SAS program that you wrote 20 years ago in Version 5 probably still works today in Version 9.2. As a technology company, SAS is big enabler when it comes to allowing customers to keep what they've got going and not forcing them to adopt new technology. That's a strength of SAS, but it has consequences. It means that our long-time users are not forced to take an in-depth look how our offerings have been modernized over the years. (Example: check out these 10 reasons statisticians should consider SAS 9.2.)

I frequently have the opportunity to introduce SAS programmers to SAS Enterprise Guide. They sometimes approach it with skepticism and trepidation. It looks a lot different than what they are used to. But the thing to remember is this: SAS Enterprise Guide is a tool that is meant to help you be more productive, to lift you out of some of the more tedious aspects of programming so that you can concentrate on your specialities, your domain area of expertise. Its role is not to remove programming from your job, but to help you to organize and focus on the tougher problems.

In a recent interview, software pioneer Dan Bricklin talks about tools that enable programmers and the important role they play. Here is an excerpt:

All of the professional-level languages are in the hardest category, as I put it. They are for professionals. They give you the most power. What the development environments do though is take care of some of the tedious manipulation. And they give you better ways of being able to define various data structures and things like that. From a programmer's viewpoint, it is a major step in lowering the complexity in terms of dealing with things -- you can deal with things that are much more complex. As programmers, what we do is we usually go to the level of complexity we can handle. So if our tools let us make things easier, we go up to that same level complexity and can do a lot more than we used to, because of the leverage of the tools.

If you're a programmer (of SAS or of any language), you've got a right to expect tools that make you more productive. Is SAS Enterprise Guide the ultimate SAS programmer tool? I'm not saying that yet...but I personally believe that it's the most comprehensive SAS productivity environment that exists today. And we're sure working to make it your go-to application for crafting and running SAS programs. Our goal is to help you get more out of your SAS investment.

P.S. In an unrelated thread, SAS tech support notified me today of a problem SAS Enterprise Guide 4.1 has when viewing the output of PROC DIF, a SAS procedure to import DIF (Data Interchange Format) files. DIF was developed in the 1980s for use with VisiCalc by -- guess who -- Dan Bricklin. SAS Enterprise Guide 4.2 fixed the reported issue, by the way. Rest easy, DIF users; we got your back.

AnnMaria's blog describing her acceptance of SAS Enterprise Guide, despite the shortcomings she's found, reminded me of this Irish Spring commercial.

I'd like to take the time to craft a more thoughtful response to her post, but that will have to wait for a less-hectic day. In the meantime, I take her critical points: SAS Enterprise Guide is big and it can be hard to find your way around, and those who prefer SAS programming might find the point-and-clickiness of the app gets in their way. (She had plenty of nice things to say about it as well. Don't take my word for it: read the post.)

As a personal testimonial, I can say that I do a fair amount of SAS programming in my job, and I use SAS Enterprise Guide exclusively for this. But I know my way around it better than most folks, and so I know how to play to its strengths. I need to post some more tips for programmers soon, so that you all can learn to love (or at least live with) SAS Enterprise Guide too.

Like AnnMaria, humorist Dave Barry (who spoke at the SAS Global Forum) had something to say about the nature of men, and had this advice: "Women, lower your expectations."

But I appreciate that our customers have high expectations for our software. They should expect to be more productive, and to be able to use our software to let their own brilliance shine through. I believe the pieces are there, and that sometimes all that's needed is a pointer in the right direction.

You might be the sort of person who loves to wait indefinitely. You visit the DMV regularly to tweak your auto registration. You queue up in the supermarket checkout line behind the customer most likely to require "price checks". You map your daily commute along the routes that offer the least certainty for traffic volume and faulty signals.

But most of us, even as we say we enjoy "living in the moment", like to have some idea about when our current activity will end and we'll be free to move on to the next activity. It's that sense of progress that keeps us going, especially during the work day.

When you run a big long SAS program in SAS Enterprise Guide, and you open the task status window to see how it's going, do you like to see the status as "Running...", whether it's been running for 10 seconds or 10 minutes or an hour? Or would you rather see something more meaningful, such as "10 out 12 steps complete - almost done..."?

If you answered the latter, then you will enjoy the new feature in SAS 9.2 and SAS Enterprise Guide 4.2 that allows you to instrument your SAS programs to report the current progress within the task status window. This trick is made possible by a new SAS global statement, SYSECHO.

The remainder of this post provides instructions and a code example for using the SYSECHO statement. (5 more short paragraphs plus a code block, for those keeping track of blog post status.)

The SAS Enterprise Guide 4.2 tutorial is going global, with translated versions now hitting the SAS support site.

First up, after the English version, are Italian and German. Other languages will soon follow.

Update 22Apr2009: just added French and Spanish as well.

This represents a couple of firsts for us:

• The tutorial content is now hosted completely online rather than shipped with the product. This makes it possible for SAS to update the content easily and add support for new languages even after the product has shipped (which it has!).
• The global SAS support site now hosts localized content that is co-mingled with English content, making it a site that can serve international audiences better.

So, you want to do two things at once?

When you use SAS Enterprise Guide to create a project, the work items in that project typically run one at a time, in series. For example, consider the following process flow:





In this flow, SAS Enterprise Guide will run the tasks from left-to-right, top-to-bottom. I've labeled the task items with 1,2,3, etc. to show in what order they'll be run.

Even though some items can theoretically run in parallel (for example, items 2, 4, and 5), SAS Enterprise Guide is using just one SAS workspace session, and it can handle just one program at a time. The question becomes, how can we divide this work so that we use multiple SAS sessions to get the work done quicker?

One approach that customers often use is to divide the project into multiple project files, and then run multiple SAS Enterprise Guide sessions at one time. Each SAS Enterprise Guide instance connects to its own SAS workspace session, so you can submit multiple blocks of work that way. However, it can be pretty darn inconvenient to manage the multiple projects simply to get some parallelism to play out.

Here's another approach: ask your SAS administrator (small shops: that might be you) to define a second logical SAS server. For example, if you have a SAS server named "SASApp", ask for a second identical server named "SASApp2". You don't need any additional hardware for this! A logical server is simply a defined way to access the services of your SAS installation, and you can camp as many of these as you want on the same physical hardware. You do want to make sure the two server definitions have "resource affinity". That's a fancy way of saying that they need access to the same data libraries, files, etc. so that they can work in exactly the same way.

(I won't go into all of the details about how to do this, but you use SAS Management Console to get this done. Add a SAS Application Server of type Workspace, select an available port, use the same SAS command and startup files to launch the SAS session, and add the new server to the SAS object spawner Servers list -- and then restart the SAS object spawner to have it take effect.)

With an additional SAS workspace to use, we can now reallocate some of the project work to allow multiple tasks to run simultaneously. Take a look at this revised flow:




The top branch references programs and data on SASApp, while the bottom branch is associated with SASApp2. Items 1a, 2a, and 3a all must run in serial on SASApp, our original server. Meanwhile, items 1b, 2b, and 3b can run in their own parallel sequence on SASApp2, the session that was added. Optionally, you can use custom user-defined links to join the two flows later, ensuring that one cannot proceed until the work is complete from the other one. Here is an example of that:




In this case, item 4b runs on SASApp2, but SAS Enterprise Guide won't run it until the work from item 3a is complete. That's because I linked 3a to 4b, indicating a dependency.

If you need concurrency that really scales, you'll want to look at SAS grid computing, which, of course, you can also leverage in SAS Enterprise Guide with just a few tweaks.

In her completely random blog entry, AnnMaria says:

I can’t see a lot of people who are experienced SAS programmers switching to Enterprise Guide.

Yeah, we get that a lot. SAS programmers sometimes resist adopting SAS Enterprise Guide citing these (paraphrased) reasons:

• "I don't need a point-and-click interface to generate my programs."
  
• "I heard that my programs run slower in EG." [They can call it "EG". I have to say "SAS Enterprise Guide".]
  
• "I already know my way around SAS display manager. Why learn this new thing?"
  

Well, as Mr. Ziglar might say, that's the sort of stinkin' thinking that hardens the attitudes. It doesn't help our cause when a SAS programmer is coerced to switch because of a change in IT policy -- for example, when your IT staff replaces SAS for Windows on your desktop with a centralized SAS workspace server that you must access via SAS Enterprise Guide.

To encourage SAS programmers to take a fresh look at what you can do with SAS Enterprise Guide, I wrote this paper for SAS Global Forum last year.

SAS Enterprise Guide offers several productivity features that match or improve upon what you can do with traditional SAS tools. For example:

You can organize your work into projects. The project file keeps track of your programs, data, tasks, and results. You can still keep your SAS programs on disk in their own files and folders, even on remote servers. The same is true for the data sources: they still live in libraries or databases. The project file simply serves as a recipe with pointers to all of the necessary ingredients to repeat the SAS process you design.

Control your process with the process flow. The process flow diagram shows the relationship of your tasks, programs, data and results within a project. You can read a process flow from left-to-right, top-to-bottom and see exactly what's going to run and in what order. You can even use the process flow to influence the order of things by creating your own links (arrows) between items on the flow.

Keep a history of what happened in the project log. SAS programmers are familiar with the running content of the LOG window within SAS. SAS Enterprise Guide has a project log feature that keeps track of all SAS log content over time. The project log is saved within the project file and accumulates this history for as long as you want it to. You can clear it or save it to an external file at any time.

Let SAS tasks do the heavy lifting. Like to write your own code, but can't remember that GCHART syntax exactly? Use a built-in chart task to get things started, then copy and paste the code that it generates for you. Or use the custom-code-insertion feature to keep your work inside the point-and-click task, but add just the code snippet you need to make it your own.

Get flexible with project prompts. We used to call these "parameters" in previous releases, and SAS programmers call them "macro variables." That's all they are: a way to display a prompt for values so that you can plug them into macro variables and make your program behave accordingly. It's a great way to reuse your work in a variety of situations.

Apply conditions to your tasks. This point is a bit of a spoiler since it's coming in our 4.2 release -- you don't have it yet. You will be able to apply conditions to evaluate to any item in your process flow, and then change the behavior of the flow depending on the condition. Basically, this is a way to provide IF-THEN-ELSE logic to your process flow and make it do different work depending on the situation.

Now I'll be candid: when you use SAS Enterprise Guide you do give up a few things that you might be accustomed to:

• DDE is D.O.A. Read the paper for more details, but basically since DDE relies on interprocess communication between Microsoft Excel and SAS, and your SAS session is running on another machine, the DDE model doesn't work.
  
• X command and SYSTASK. These functions tell SAS to run operating system shell commands from within your SAS session. By default, this feature is turned off for SAS workspace servers. However, there are workarounds.
  
• SAS/AF programs. SAS/AF programs and certain SAS applications rely on the SAS windowing environment to display a user interface. In the client-server model used by SAS Enterprise Guide, the SAS workspace does not have anywhere to host those windows. Many customers work through this by developing custom tasks, which provide a better experience in SAS Enterprise Guide.
  

Our goal is to make SAS Enterprise Guide the preferred tool for SAS programmers. Many SAS programmers have already (voluntarily!) adopted it. We are working on additional key features (for example, a smarter syntax editor, process flow improvements) to help seal the deal for the rest of you. Got a wish list? Post a comment or send us a suggestion.