Using the ODS statement to add layers in your ODS sandwich

The ODS statement controls most aspects of how SAS creates your output results. You use it to specify the destination type (HTML, PDF, RTF, EXCEL or something else), as well as the details of those destinations: file paths, appearance styles, graphics behaviors, and more. The most common use pattern is the "ODS sandwich." In this pattern, you open the destination with the ODS statement, then include all of the code that generates the substance of the output, and then use an ODS CLOSE statement to finish it off. Here's a classic example:

ods html file="c:\project\myout.html" /* top slice of bread */
  style=journal gpath="c:\project";
  proc means data=sashelp.class;      /* the "meat" */
  proc sgplot data=sashelp.class;
  histogram weight;
ods html close;                       /* bottom slice */

But did you know that you can insert more ODS statements to adjust ODS behavior midstream? These allow you to use a variety of ODS behaviors within a single result. You can create your own "Dagwood sandwich" version of SAS output! For cultural reference:

Dagwood sandwich: A Dagwood is a tall, multi-layered sandwich made with a variety of meats, cheeses, and condiments. It was named after Dagwood Bumstead, a central character in the comic strip Blondie, who is frequently illustrated making enormous sandwiches. Source: Wikipedia

Here's an example program that changes graph style and title behavior within a single ODS output file. You should be able to try this code in any SAS programming environment.

ods _all_ close;
%let outdir = %sysfunc(getoption(WORK));
ods graphics / width=400 height=400;
ods html(id=dagwood) file="&outdir./myout.html"
  style=journal gtitle  
  title "Example ODS Dagwood sandwich";
  proc means data=sashelp.class;
ods layout gridded columns=2;
ods region;
ods html(id=dagwood) style=statdoc ;  
  proc sgplot data=sashelp.class;
  title "This title is part of the graph image, Style=STATDOC";
  histogram weight;
ods region;
ods html(id=dagwood) style=raven nogtitle;
  title "This title is in the HTML, Style=RAVEN";
  proc sgplot data=sashelp.class;
  histogram height;
ods layout end;
ods html(id=dagwood) close;

odsexHere's the result, plus some important items to note about this technique.

  • It's a good practice to distinguish each ODS destination with an ID= value. This allows you to reference the intended ODS stream with no ambiguity. After all, you can have multiple ODS destinations open at once, even multiple destinations of the same type. In my example, I used ID=dagwood to make it obvious which destination the statement applies to.
  • You can use this technique to modify only those directives that can change "mid-file" to apply to different parts of the output. You can't modify those items that apply to the entire file, such as PATH, ENCODING, STYLESHEET and many more. These can be set just once when you create the file; setting multiple different values wouldn't make sense.

You can use this technique within those applications that generate ODS statements for you, such as SAS Enterprise Guide. For example, to modify the default SAS Enterprise Guide HTML output "midstream", add a statement like:

ods html(id=eghtml) /*... plus your options, like STYLE=*/ ;
ods html(eghtml) /* this shorthand works too */ ;

In SAS Studio or SAS University Edition, try this:
ods html5(id=web) /*... plus your options, like STYLE=*/ ;
ods html5(web) /* this shorthand works too */ ;

Example: an ODS Graphics style sampler

samplerHere's one more example that puts it all together. Have you ever wanted an easy way to check the appearance of the dozens of different built-in ODS styles? Here's a SAS macro program that you can run in SAS Enterprise Guide (with the HTML result on) that generates a "sampler" of graphs that show variations in fonts, colors, and symbols across the different styles.

This example uses ODS LAYOUT (production in SAS 9.4) to create a gridded layout of example plots. If you want to try this in SAS Studio or in SAS University Edition, you can adjust one line in the program (as noted in the code comments).

/* Run within SAS Enterprise Guide       */
/* with the HTML result option turned ON */
%macro styleSampler;
proc sql noprint;
  select style into :style1-:style99  
    from sashelp.vstyle 
    where libname="SASHELP" and memname="TMPLMST";
  ods layout gridded columns=4;
  ods graphics / width=300 height=300;
  %do index=1 %to &sqlobs;
    ods region;
    ods html(eghtml) gtitle style=&&style&index.;
    /* In SAS Studio, use this instead: */
    /* ods html5(web) gtitle style=&&style&index.; */
    title "Style=&&style&index.";
    proc sgplot data=sashelp.class;
    scatter x=Height y=Weight /group=Sex;
    reg x=Age y=Weight / x2axis;
  ods layout end;

See also

Take control of ODS results in SAS Enterprise Guide
Best way to suppress ODS output in SAS
Advanced ODS Graphics techniques: a new free book

Post a Comment

Copy SAS variable names to the clipboard in SAS Enterprise Guide

I recently met SAS user "CSC" at the Analytics 2015 conference. It might be generous to say that he's an avid user of SAS Enterprise Guide; it's probably more accurate to say that he's now accustomed to the tool and he's once again productive. But he still misses some features from his PC SAS days, including this one.

He wants to be able to copy just a list of SAS variables names from a SAS data set, so that he can then paste them into a SAS program (or another document). In PC SAS he had a simple GSUBMIT sequence that captured the names and "copied" them to the Windows clipboard with FILENAME CLIPBRD. That does not work in SAS Enterprise Guide, because SAS doesn't have direct access to the clipboard on your local machine.

CSC posted his question to the SAS Enterprise Guide community, and Tom suggested that a custom task might help. Good answer, but there it sat until CSC and I met in person this week in Las Vegas. After a short discussion and a personal plea, I was able to create the task in about 30 minutes.

Actually, it's three tasks, to cover three variations of the "Paste" operation. One supports a CSV-style, another supports CSV over multiple lines, and a third produces just a straight list on separate lines with no commas.


You can download and try this custom task too. It works with SAS Enterprise Guide 4.3 and later. Download the task from the SAS support site as a ZIP file. The instructions for installation and use are in the README.txt in the ZIP file.

Related articles

Post a Comment

The famous SAS cowboy hat now fits all SAS users

cbhat_sgRick Wicklin created a nice example of using the SURFACEPLOTPARM statement to create a surface plot in SAS. As I read it, the question that immediately came to mind was: can I use this to create the famous SAS cowboy hat?

The "cowboy hat" is a highly distributed example of using PROC G3D to create a 3-dimensional rendering of data that resembles...well...a cowboy hat. PROC G3D is a cool SAS proc, but it's part of the SAS/GRAPH product and not everyone has access to that. For example, users of the SAS University Edition cannot run PROC G3D or any SAS/GRAPH programs. But the SG procedures, including SGPLOT and SGRENDER, are built into Base SAS.

Now we can bring the cowboy hat to the next generation of SAS users. Without further ado, here is the SAS program that build the hat. The program works in SAS Display Manager, SAS Enterprise Guide, and SAS University Edition.

/* Graph Template Language that defines the graph layout    */
/* This needs to be run just once within your SAS session   */
/* From Rick's post at:                                     */
/* */
proc template;                        /* surface plot with continuous color ramp */
define statgraph SurfaceTmplt;
dynamic _X _Y _Z _Title;              /* dynamic variables */
 entrytitle _Title;                   /* specify title at run time (optional) */
  layout overlay3d;
    surfaceplotparm x=_X y=_Y z=_Z /  /* specify variables at run time */
       colormodel=threecolorramp      /* or =twocolorramp */
    continuouslegend "surface";
/* DATA step to create the "hat" data */
data hat; 
 do x = -5 to 5 by .5;
  do y = -5 to 5 by .5;
   z = sin(sqrt(y*y + x*x));
ods graphics / width=1000 height=800;
/* And... Render the Hat! */
proc sgrender data=hat template=SurfaceTmplt; 
   dynamic _X='X' _Y='Y' _Z='Z' _Title="Howdy Pardner!";
Post a Comment

Using SAS DS2 to parse JSON

Thanks to the proliferation of cloud services and REST-based APIs, SAS users have been making use of PROC HTTP calls (to query these web services) and some creative DATA step or PROC GROOVY code to process the JSON results. Such methods get the job done (JSON is simply text, after all), but they aren't as robust as an official JSON parser. JSON is simple: it's a series of name-value pairs that represent an object in JavaScript. But these pairs can be nested within one another, so in order to parse the result you need to know about the object structure. A parser helps with the process, but you still need to know the semantics of any JSON response.

SAS 9.4 introduced PROC JSON, which allows you to create JSON output from a data set. But it wasn't until SAS 9.4 Maintenance 3 that we have a built-in method to parse JSON content. This method was added as a DS2 package: the JSON package.

I created an example of the method working -- using an API that powers our SAS Support Communities! The example queries for the most recent posts to the SAS Programming category. Here's a small excerpt of the JSON response.

 "post_time": "2015-09-28T16:29:05+00:00",
  "views": {
  "count": 1
  "subject": "Re: How to code for the consecutive values",
  "author": {
  "href": "\/users\/id\/13884",
  "login": "ballardw"

Notice that some items, such as post_time, are simple one-level values. But other items, such as views or author, require a deeper dive to retrieve the value of interest ("count" for views, and "login" for author). The DS2 JSON parser can help you to navigate to those values without you needing to know how many braces or colons or commas are in your way.

Here is an example of the result: a series plot from PROC SGPLOT and a one-way frequency analysis from PROC FREQ. The program also produces a detailed listing of the messages, the topic content, and the datetime stamp.


This is my first real DS2 program, so I'm open to feedback. I already know of a couple of improvements I should make, but I want to share it now as I think it's good enough to help others who are looking to do something similar.

The program requires SAS 9.4 Maintenance 3. It also works fine in the most recent version of SAS University Edition (using SAS Studio 3.4). All of the code runs using just Base SAS procedures.

/* DS2 program that uses a REST-based API */
/* Uses http package for API calls       */
/* and the JSON package (new in 9.4m3)   */
/* to parse the result.                  */
proc ds2; 
  data messages (overwrite=yes);
    /* Global package references */
    dcl package json j();
    /* Keeping these variables for output */
    dcl double post_date having format datetime20.;
    dcl int views;
    dcl nvarchar(128) subject author board;
    /* these are temp variables */
    dcl varchar(65534) character set utf8 response;
    dcl int rc;
    drop response rc;
    method parseMessages();
      dcl int tokenType parseFlags;
      dcl nvarchar(128) token;
      * iterate over all message entries;
      do while (rc=0);
        j.getNextToken( rc, token, tokenType, parseFlags);
        * subject line;
        if (token eq 'subject') then
            j.getNextToken( rc, token, tokenType, parseFlags);
        * board URL, nested in an href label;
        if (token eq 'board') then
            do while (token ne 'href');
               j.getNextToken( rc, token, tokenType, parseFlags );
            j.getNextToken( rc, token, tokenType, parseFlags );
        * number of views (int), nested in a count label ;
        if (token eq 'views') then
            do while (token ne 'count');
               j.getNextToken( rc, token, tokenType, parseFlags );
            j.getNextToken( rc, token, tokenType, parseFlags );
        * date-time of message (input/convert to SAS date) ;
        * format from API: 2015-09-28T10:16:01+00:00 ;
        if (token eq 'post_time') then
            j.getNextToken( rc, token, tokenType, parseFlags );
        * user name of author, nested in a login label;
        if (token eq 'author') then
            do while (token ne 'login');
               j.getNextToken( rc, token, tokenType, parseFlags );
            * get the author login (username) value;
            j.getNextToken( rc, token, tokenType, parseFlags );
    method init();
      dcl package http webQuery();
      dcl int rc tokenType parseFlags;
      dcl nvarchar(128) token;
      dcl integer i rc;
      /* create a GET call to the API                                         */
      /* 'sas_programming' covers all SAS programming topics from communities */
         '' || 
         'restapi/vc/categories/id/sas_programming/posts/recent' ||
         '?restapi.response_format=json' ||
      /* execute the GET */
      /* retrieve the response body as a string */
      webQuery.getResponseBodyAsString(response, rc);
      rc = j.createParser( response );
      do while (rc = 0);
        j.getNextToken( rc, token, tokenType, parseFlags);
        if (token = 'message') then
  method term();
    rc = j.destroyParser();
/* Add some basic reporting */
proc freq data=messages noprint;
    format post_date datetime11.;
    table post_date / out=message_times;
ods graphics / width=2000 height=600;
title '100 recent message contributions in SAS Programming';
title2 'Time in GMT';
proc sgplot data=message_times;
    series x=post_date y=count;
    xaxis minor label='Time created';
    yaxis label='Messages' grid;
title 'Board frequency for recent 100 messages';
proc freq data=messages order=freq;
    table board;
title 'Detailed listing of messages';
proc print data=messages;

I also shared this program on the SAS Support Communities as a discussion topic. If you want to contribute to the effort, please leave me a reply with your suggestions and improvements!

Post a Comment

Why should we teach Roman numerals?

In my local paper this morning, I read about how a North Carolina state commission plans to recommend changes to our teaching standards for mathematics. One of the topics that they want to bring back: Roman numerals. Why? According to my exhaustive 30 seconds of Internet research, the only practical applications of Roman numerals are: I) understanding Super Bowl numbering, and II) reading the time on old-fashion clocks.

But I don't need convincing. I believe that there are other advantages of teaching Roman numerals. The main lesson is this: the world has not always revolved around "base 10" numbering, and actually it still doesn't today. Having the ability to express numbers in other forms helps us to understand history, passage of time, technology, and even philosophy*.

In the popular media, binary (base 2) is famous for being "the language of computers". That may be so, but binary is not usually the language of computer programmers. When I was a kid, I spent many hours programming graphics on my TI 99/4A computer. I became proficient in translating decimal to hexadecimal (base 16) to binary -- all to express how the pixels would be drawn on the screen and in what color. Due to lack of practice and today's availability of handy tools and higher-level programming languages, I have since lost the ability to calculate all of these in my head. I also lost the ability to solve any Rubik's Cube that I pick up -- there go all of my party tricks.

But the SAS programming language retains many fun math tricks, including the ability to express numbers in many different ways, instantly. Here's an example of one number expressed six (or 6 or VI or 0110) different ways.

data _null_;
  x = 1956;
  put  / 'Decimal: '     x=best12.;
  put  / 'Roman: '       x=roman10.;
  put  / 'Word: '        x=words50.;
  put  / 'Binary: '      x=binary20.;
  put  / 'Octal: '       x=octal10.;
  put  / 'Hexadecimal: ' x=hex6.;

The output:

Decimal: x=1956
Roman: x=MCMLVI
Word: x=one thousand nine hundred fifty-six
Binary: x=00000000011110100100
Octal: x=0000003644
Hexadecimal: x=0007A4

You might never need some of these number systems or SAS formats in your job, but knowing them makes you a more interesting person. If nothing else, it's a skill that you can trot out during cocktail parties. (I guess I attend different sorts of parties now.)

* For example, the number 'zero' has not always been with us. Introducing it into our numbering system allows us to think about 'nothing' in ways that earlier societies could not.

Post a Comment

Copy data and column names from SAS Enterprise Guide

While I've often written about how to get your SAS data to Microsoft Excel in some automated way, I haven't really addressed what's probably the most frequently used method: copy and paste. SAS Enterprise Guide 7.1 added a nifty little feature that makes copy-and-paste even more useful.

The new "Copy with headers" feature creates a tab-delimited version of your selected data cells, complete with a heading row that includes the column names. This is very convenient for copying into a new Microsoft Excel spreadsheet or other table structure (like a Google Docs spreadsheet). Note: this is different than the "copy data attributes" tip that I published a while back. That tip captures the column properties, but not the actual data values.

To get started, simply select the data that you want within the SAS Enterprise Guide data grid. The data cells must be contiguous, but you don't need to select all columns or all cells within the data set. With the selection active, right-click and select Copy with headers.

copy with headers
This action places the data values in tab-delimited form onto the Windows clipboard. The first line of the data will be the SAS variable names from your selected data. If you paste this into a text editor that shows a view of "special characters", you can see the tabs along with the end-of-line delimiters.

tab delimited content
When you paste the same content into Microsoft Excel, the Excel application knows how to automatically distribute these values into distinct columns. That's just what spreadsheet programs do.

paste into Excel
Microsoft Excel isn't the only app that can handle tab-delimited data in this way. You can paste the content into a Google Doc spreadsheet too.

paste into Google Docs
Or, you could simply paste into a text file as-is, and then save that file to read into a SAS program at a later time, bringing it full circle.

Post a Comment

Using Lua within your SAS programs

With apologies to this candy advertisement from the 1980s:

"Hey, you got your Lua in my SAS program."
"You got your SAS code in my Lua program!"

Announcer: "PROC LUA: Two great programming languages that program great together!"

What is Lua? It's an embeddable scripting language that is often used as a way to add user extensions to robust software applications. Lua has been embedded into SAS for some time already, as it's the basis for new ODS destinations like EXCEL and POWERPOINT. But SAS users haven't had a way to access it.

With SAS 9.4 Maintenance 3 (released July 2015), you can now run Lua code in the new LUA procedure. And from within that Lua code, you can exchange data with SAS and call SAS functions and submit SAS statements. (Running SAS within Lua within SAS -- it's just like Inception.)

Paul Tomas, the developer for PROC LUA, presented a demo of the feature and its usefulness in a recent SAS Tech Talk:

Paul also wrote a paper for SAS Global Forum 2015: Driving SAS with Lua.

Like many innovations that find their way into customer-facing features, this new item was added to help SAS R&D complete work for a SAS product (specifically, the new version of SAS Forecast Server). But the general technique was so useful that we decided to add it into Base SAS as a way for you to integrate Lua logic.

PROC LUA can be an alternative to the SAS macro language for injecting logical control into your SAS programs. For example, here's a sample program that generates a SAS data set only if the data set doesn't already exist.

proc lua ;
-- example of logic control within LUA
if not sas.exists("work.sample") then
    print "Creating new WORK.SAMPLE"
	sas.submit [[
	  data work.sample;
	    set sashelp.class;
   else print "WORK.SAMPLE already exists"

First run:

NOTE: Lua initialized.
Creating new WORK.SAMPLE
    data work.sample;
      set sashelp.class;

And subsequent runs:

NOTE: Resuming Lua state from previous PROC LUA invocation.
WORK.SAMPLE already exists
NOTE: PROCEDURE LUA used (Total process time):
      real time           0.00 seconds
      cpu time            0.00 seconds

Unlike other embedded languages (like PROC GROOVY), Lua runs in the same process as SAS -- and not in a separate virtual machine process like a Java VM. This makes it easy to exchange information such as data and macro variables between your SAS and Lua programming structures.

If you have SAS 9.4M3 and have time to play with PROC LUA, let us know what interesting applications you come up with!

Post a Comment

SAS Enterprise Guide now updates itself

I returned to work from a 2+ week vacation this morning. When I fired up SAS Enterprise Guide (as I do each work day and occasionally on weekends), I was greeted with this message:

An update to SAS Enterprise Guide is available!
As a SAS insider, I knew this was coming. It's a new feature that was added in SAS Enterprise Guide 7.11. I intended to write a blog about this before now -- but then I went on vacation instead.

I'm a trusting fellow, but I still clicked on the link in the message to learn more information about the update. All of the improvements seemed good to me, so I clicked Close and Install.

My SAS Enterprise Guide session closed and a moment later I saw the patch being applied:

Applying patch
When complete, I was greeted with this good news:

Your software is now up to date!
And when I clicked Finish and the application restarted, I checked the Help->About SAS Enterprise Guide window to see that the update was in place.

about window with HF number
I think that this "automatic update" is a tremendous feature whose time has come (if it's not overdue). However, not everyone will want to update their software on SAS' schedule. You can defer the update with Remind me later, or select Skip this version in order to not be reminded again (until the next update). You can always check for updates from the Help menu.

UPDATE: Since posting this article last week, I've heard from several concerned citizens who have said "Whoah! I hope that admins can disable this! We can't have our users updating their own applications!" Yes, the "Check for Updates" notification and menu item can be disabled by using the methods in this soon-to-be-published SAS note. And the feature does not allow end users to update their PC applications if they don't already have privileges to do so.

Post a Comment

Copy an entire process flow in SAS Enterprise Guide

I've seen some crazy process flows in SAS Enterprise Guide. Crazy-big, and crazy-complex, used by real customers to accomplish real work. But while these process flows represent a ton of work, this is usually a calculated investment to automate processes that would be difficult to capture in another way.

For years, SAS Enterprise Guide users have asked for a way to reuse their process flows in new projects. You have always been able to copy-and-paste individual items (tasks, queries, programs) from one project to another, or from one flow to another in the same project. But when you tried to copy a collection of items from a flow, everything fell apart when you pasted into the new destination. The links/relationships among the objects were not retained, and that sabotaged your goal of saving time and effort.

In SAS Enterprise Guide 7.1, this is finally improved. You can now copy an entire flow, or multiple selected items from a flow, and paste that content intact into another project or process flow.

To copy an entire flow, right-click on the flow name within the Project Tree, then select Copy. In your destination project, right-click on an empty spot within the Project Tree and you'll see that Paste is enabled. When you paste, you'll see the entire flow transfer over, including the links and layout. It's like magic.

To copy just a portion of the process flow, click-drag the cursor to "rubberband" a selection of connected items. (You can also use Ctrl+click to select the items that you want to include.) With the items selected, right-click on one of the selected items and choose Copy. You can then Paste the content into a new or existing process flow.

You can paste a process flow into a new or an existing SAS Enterprise Guide project. Try it for yourself -- see how much time it can save for you!

Note: this feature was added in SAS Enterprise Guide 7.1. The first update (7.11) improved it further (such as capturing project prompts that are referenced in your flow).

Post a Comment

Ask the Expert: Creating custom tasks for SAS Enterprise Guide

If you have not yet discovered the new Ask the Expert series on the SAS Training site, you are missing out on a treasure. Visit the site right now and review all of the available topics, from "Newbie" to Analytics to Visualization to good ol' SAS programming. Go on; I'll wait.

Ask the Expert - they wear glasses, apparently
Welcome back! Amazing, right? You can get lost for hours learning new stuff or just reviewing what you thought you already knew. Some topics are available as live sessions that you can "attend" as they happen, but many of them are available on-demand, for free, no strings attached.

You might have noticed that I have a humble contribution in the "expert" collection: Developing Custom Tasks for SAS Enterprise Guide. During the 37-minute video, I lead you through the uses of SAS custom tasks and the basic steps for creating your own task. You'll learn what custom tasks can do and what they cannot do. You'll learn about the tools and APIs that support the creation of tasks. And you'll see the "inside" of a completed custom task project. (An aside: I owe a big "thank you" to the team that post-processed the recording of my expert talk -- I know that I wasn't that smooth and concise when I recorded it!)

It's a short introduction and watching it won't make you an expert in the topic, but it will help you to decide whether to learn more. You can learn more from my book on this topic, or you can arrange to attend an offering of the two-day course that we offer occasionally. Or you can learn it all on your own, as many have. There are plenty of examples and references to work from. If you're wondering what skills you should have before taking the class, watch my "about this course" video here.

In the "ask the expert" video, I referenced a collection of API libraries that make it easier to set up your custom task projects. These API libraries are available for each version of SAS Enterprise Guide, and they allow you to create tasks that are compatible with multiple versions of the SAS applications, even if you do not have those particular versions installed. (You still need at least one version of SAS Enterprise Guide or SAS Add-In for Microsoft Office in order to test and run your custom task.)

With the permission of the SAS R&D developers, I have made those libraries available here:

>> Download custom task API libraries (ZIP file 333KB)

The README.txt file in the ZIP file explains how to use the libraries.

If you have questions as you start your custom task adventures or if you just want to brag about your successes, post back here or on the SAS Enterprise Guide community. I'd love to hear from you!

Read More »

Post a Comment