Macro Myth: The closest non-empty symbol table

Robert Virgile

Robert Virgile

This SAS tip is from Robert Virgile and his book “SAS Macro Language Magic: Discovering Advanced Techniques”.

We hope you find this tip useful. You can also read an excerpt from Virgile’s book.

When CALL SYMPUT creates a new macro variable, it places that variable in “the closest non-empty symbol table.”  But what does that mean?  And why does CALL SYMPUT exhibit such unusual behavior?

Read More »

Post a Comment

Get real clinical data

clinicalMany of our authors often ask us where they can find real data that they can use without copyright or other confidentiality issues. Instructors too are always on the look-out for real-life data.

Well, thanks to a new initiative supported by SAS, you can now access data from more than 40 studies and 25,000 patients from clinical trials of prostate cancer, breast cancer, and melanoma treatments. The list of available studies is growing rapidly! See for yourself.

Project Data Sphere, founded by the CEO Roundtable on Cancer, brings together leading pharmaceutical companies to share data and an analytics platform to investigate those data. In support of Project Data Sphere and the research community, SAS hosts the platform and provides access to SAS Drug Development and SAS Visual Analytics at no cost to researchers.  For most studies, data is already available as SAS data sets, consistent with CDISC/SDTM requirements and ready for analysis using the web-enabled analytics platform.  Students, professors, and others in the research community are welcome to join this growing community of clinical trial researchers in the fight against cancer.

So what better time to write a book with all this data to play with! I’m headed to Orlando, May 17- 20, for the annual PharmaSUG conference. I will be at the SAS Publications booth in the demo room and would love to talk to you about your book ideas, whether you want to write your own book, review a book, or suggest a book idea. We are actively looking to expand our offerings to cover a variety of pharmaceutical topics; and so if you have an interesting application or experience in using SAS solutions, then please consider sharing it as a short book.

Please do stop by the demo room at PharmaSUG 2015 or email me if you'd like to set aside some time to talk.

Post a Comment

Top 3 benefits of writing a SAS book

Many colleagues and customers at SAS Global Forum ask me, why I write books beside having a full-time job as SAS consultant and being lecturer at universities. Valid question.

  1. SUSTAINABILITY. I have been working in so many analysis projects in different domains and industries. Some of them were longer, others were shorter engagements, and I found out that I wanted to create and produce something that lasts for a long period of time and that is connected with my name. A book is definitely a good idea for this.
  2. YOU EDUCATE YOURSELF! Yes, the content of both of my books was known to me before I started writing. They are a collection of experiences from interacting with SAS Users in the field of analytics over many years. But a book forces you to structure the content, divide your ideas into chapters, formulate your experiences and ideas in an explicit way. After completing each book I felt more knowledgeable. And even more important: I was able to formulate my ideas, my knowledge and solutions more explicitly. And I often benefit of this experiences in customer meetings, presentations and discussions.
  3. YOU GET FEEDBACK. There are many rewarding moments when being a SAS Author: You are part of the community of SAS authors, you work together with the team of SAS Press and most important you get feedback from your readers. I receive emails from all over the world from readers who let me know how they used the ideas presented in my books. You are always happy to hear that readers like and use your work. You are also happy to hear if they suggest improvements as this shows that they really got involved with your work.

And the most rewarding moment for an author is to see customers holding his books in their hands.

SASBOOKs

 

Post a Comment

SAS Books is heading to PharmaSUG

PharmaSUG_2015_logo_200pxI’m headed to Orlando, May 17- 20, for the annual PharmaSUG conference. PharmaSUG is the Pharmaceutical Industry SAS Users Group, consisting of professionals worldwide in the pharmaceutical and healthcare industries who use SAS software in their work.

I will be at the SAS Publications booth in the demo room and would love to talk to you about your book ideas, whether you want to write your own book, review a book, or suggest a book idea. We are actively looking to expand our offerings to cover a variety of pharmaceutical topics; and so if you have an interesting application or experience in using SAS solutions, then please consider sharing it as a short book.

We will also be offering a special conference discount on books – including the best-selling classics: Analysis of Clinical Trials Using SAS; and; Analysis of Observational Health Care Data Using SAS; and exciting newer titles such as: SAS Programming in the Pharmaceutical Industry, Second Edition; and; Risk-Based Monitoring and Fraud Detection in Clinical Trials Using JMP and SAS.

Please do stop by the demo room at PharmaSUG 2015 or email me if you'd like to set aside some time to talk.

Post a Comment

Doctor, I have this friend who has a pain right about here

We have all heard the old joke about the person who sidles up to a doctor at a party and describes in great detail a medical problem a "friend" is having in order to get free medical advice.  It could just as easily be a person buttonholing an accountant for free tax advice, a lawyer for free legal advice, a personal trainer for free fitness advice, a CEO for free business advice, a chef for free cooking advice, a financial planner for free investment advice, a dietitian for free dietary advice, a college professor for free educational advice, and so on.  But, what about free SAS programming advice?

The good news is that if you have a question about SAS programming, you don't have to sidle up to a SAS expert and ask it in a roundabout way. There are a number of great venues where you can ask SAS technical questions directly and have them answered by SAS experts from the US and from around the world.  And, the only cost to you is the several hundred keystrokes you expend writing a concise description of your problem with supporting information such as:

  • The portion of your SAS program in question
  • A snippet of the log showing an error, if applicable
  • The version of SAS you are using
  • The operating system you are running SAS on

Here are some of the facilities available to help SAS professionals with their programming questions:

  • Support Communities on support.sas.com -- There are very active discussion groups with titles such as Base SAS Procedures, SAS Macro Language and Data Step, ODS and Base Reporting, and many more.  Users post their programming questions and then SAS experts from around the world and from SAS post possible solutions.  Questioners usually get many and varied answers to their questions.  This web-based community even has its own document titled "How to ask a Question in SAS Support Communities?" to help lead you through the process if you are new.  You can access the SAS Support Communities via this link.
  • SAS-L Listserv.  This oldest medium for SAS Q&A is realized as an email listserv.  You subscribe to the SAS-L listserv, and then when people post questions and answers, all subscribers get the messages.  SAS-L has a very loyal base of SAS experts who usually jump on questions immediately with answers from many different angles.  The sasCommunity.org web site has a good write-up on how to subscribe to SAS-L.
  • LinkedIn SAS Groups.  There are over a dozen SAS discussion groups on LinkedIn.  Some notable ones for Q&A are:  SAS Professional Forum, SAS Programming Tips, and SAS Author: Phillip R Holland.  These discussion groups are conducted much the same way as the previously mentioned Q&A sites, but also include articles of general interest to SAS programmers.  The LinkedIn home page is:  https://www.linkedin.com/
  • Code Clinics.  Most of the SAS Local Users Groups (LUGs) and SAS Regional Users Groups (RUGs) feature a "code clinic" at their meetings.  The code clinics are staffed with SAS programming experts who can help you with programming problems one-on-one.  They are expecting drop-ins, so bring your SAS questions and sidle-away.

There are doubtlessly other web sites and other facilities out there in the big wide world where SAS users will be glad to pitch in and help answer your questions. But, the list above is a good starting point.

Hey, as long as I have you here; I have this friend who is interested in competing in his first triathlon; you wouldn't happen to know anything about training for one would you?

Post a Comment

Meet SAS Press authors at SAS Global Forum

BOoksThis year SAS Global Forum attendees have the chance to have lunch with SAS authors and find out what it takes to write a book.

When: Tuesday, April 28, 12 - 12:50 p.m.

Where: Ballroom D2.

Lunch is provided!  We’ll talk a little about what it’s like to write a book and then open the floor for questions. And, those who attend the lunch will have a chance to win a copy of a book of their choice from the SAS Bookstore.

We are continuing to expand the SAS Press program - so if you're interested in writing a SAS book, or know someone who should write a book for SAS then please pop over to the SAS Press booth in the Quad and let us know -- or send us your author recommendations to saspress@sas.com.

And don’t forget the SAS Author Roundup Tuesday at lunch! We look forward to seeing you in Dallas!

Post a Comment

Keep your hands off of my SAS data sets

I am all about sharing.  Knock on my door and I will gladly lend you a stick of margarine, a cup of sugar, an egg or two, some flour, a corkscrew, or a beer.  Not a problem to borrow a tie, one of my extra belts, a white shirt (if it fits), a scarf, or a pair of gloves.  Sure, I will gladly lend you one the books in my home library, a couple of my music CD's, or one of my movie DVDs.  I am reasonably sure that most of these items will either be returned to me in due time, or reciprocated, or paid forward.  But, keep your hands off of my SAS data sets!

I would bet that final sentiment regarding SAS data sets is pretty prevalent in organizations where SAS programmers work with shared storage resources.  Whether you are storing your SAS data sets in server directories or on network directories, you do not want to have them deleted, resorted, updated, or otherwise overwritten by other programmers on your team.  At least, not without your permission.

Fortunately, most organizations create security permissions that largely safeguard against unauthorized access of data.  They implement security packages--either native to the OS or purchased from vendors--to ensure that data is accessed only by those with the need to know.  Groups of programmers are given access to specific directories and other staff, with the exception of systems administrators, cannot get into them or see them at all.  Despite all of this protection, data integrity issues can come from programmers within your own group; programmers who have access to the same directories that you do; programmers who have the same access rights that you do.

Unfortunately, there is no foolproof way for you to keep your teammates from crunching your SAS data sets.  But, here are a few protection measures that you can put into place to help safeguard your SAS data sets:

  • SAS Data Set Passwords.  SAS allows you to specify ALTER, READ, and WRITE passwords.  Users must know the password in order to process password-protected data sets with SAS.  So, you can specify passwords for your do-not-disturb SAS data sets, wait for your co-workers to complain, and determine if they really do need access to those data sets.
  • ACCESS=READONLY Libname Option.  This option specifies that no data sets in the library can be updated and no new files can be written to the library.  This option can be effectively deployed in a shared group AUTOEXEC.sas file.  If your colleagues grumble, tell them to copy the data sets in question to one of their libraries and process them there.  Caution them to be mindful of version control issues.
  • SAS Views.  Create views of your SAS data sets via the DATA step or PROC SQL and allow your colleagues to use the views instead of the actual data sets.  The beauty of this approach is that you can have your valued data sets in one directory and the views in another.  Point your coworkers at the views directory and do not tell them the whereabouts of the permanent data sets.
  • Generation Groups.  You can specify for SAS to keep several generations of your SAS data sets available.  Consequently, when one of them is modified, you will still have the older version available.
  • SAS Audit Logs.  SAS Audit Logs can be used to determine who updated SAS data sets after the fact. They don't stop your colleagues from actually updating SAS data sets, but you can use them to determine the who, what, and when so that you can storm into the right office for an explanation.
  • LOCK Statement.  You could use the LOCK statement to lock your SAS data sets so that no other SAS program can read or write to the file.  This is a bit extreme and would require that your SAS program with the LOCK statements be running for the duration of when you wanted to safeguard your SAS data sets.  This option can be effectively deployed in a shared group AUTOEXEC.sas file.
  • Zip Files.  You could simply zip your SAS data sets up into zip files, delete the permanent SAS data sets, and restore the data sets from the zip files when you need them.  This is another extreme measure, but if your data sets get clobbered on a regular basis, you may find it more appealing than having them constantly restored.

Obviously, the best solution for keeping your important SAS data sets from being updated by your teammates is good communications between all involved, and a shared set of best practices for accessing and modifying SAS data sets.  But, when that is not available, you may have to reach for some of the ideas I have posted here.

Oh, and about that book that you borrowed from my SAS bookshelf library a couple of months ago... can I get it back?

Best of luck in all your SAS endeavors!

Have a great work-week!

Post a Comment

We don’t need no education!

book_appleSo, how did you first learn SAS programming?

Originally, I was self-taught.  Many years ago, I learned SAS on the job when a systems programmer quit and I took over supporting a mainframe performance software package that was written in SAS. I got a copy of the Base SAS users guide and the SAS Procedures guide and learned how to write my own SAS programs.  It was new and fun; and SAS was more powerful than any of the other programming languages I had been using.

After I had been programming with SAS for about three years, my then employer finally paid to send me to a SAS class.  I entered that class pretty confident of my SAS programming abilities, but was humbled within the first hour.  I had been running my SAS programs in batch on mainframe computers.  All of the students in the class ran their SAS programs interactively with SAS Display Manager, which I had never seen before.  So, I had to catch up to their level of expertise just to do the class exercises.

It was a great class!  I learned to use the SAS Display Manager; the proper ways to perform match merges; how the Program Data Vector works in the DATA step; the intricacies of PROC MEANS and its first cousin PROC SUMMARY; and a host of other very useful programming techniques. I learned the fundamentals of SAS that I had missed by simply jumping into using it and getting all of my information from the manuals.  That class made me a much stronger SAS programmer.

Today's programming professionals who are interested in taking formal SAS classes to increase their SAS programming acumen have a lot of choices.  SAS currently has the following training formats:

  • Classroom.  You can take an instructor-led SAS class in one of SAS's state-of-the-art training facilities located throughout the US and in many other countries.
  • Live Web Classroom.  Let an instructor-led SAS class come to you via the Internet.  This format saves on travel costs and time out of the office.
  • E-Learning.  This option allows you to take SAS classes at your own pace via the Internet 24/7.  This is another way to save travel costs and time out of the office.
  • On-Site Training.  Have qualified SAS instructors come on-site to teach you and your colleagues various SAS courses.
  • Mentoring Services.  This option provides you with a SAS instructor who becomes your coach to help you learn how to write SAS programs that address your organization's unique data processing needs.

You can find out much more about available SAS classes being offered, their content, when they are scheduled, and the training formats in the Training section of the support.sas.com web site.

Check it out because there is likely a SAS class that would make you a stronger programmer in a format that fits with your own work schedule.

"We don't need no education"?  Nope, that couldn't be further from the truth!

Best of luck in all your SAS endeavors!

Post a Comment

What to do when all your boss wants is a spreadsheet

Most SAS programmers have been here. Someone just wants a handful of numbers that they can add to a graph or power point presentation that is due tomorrow. You have the data files, you have a job to summarize it, and you have a dilemma. How do I get my data where the boss wants it, into Excel?

Transferring data between SAS and Microsoft Excel may be easier than you think.

I do not know how many times I have “Googled” something and gotten a cryptic answer that was marginally effective or even useless. You know that something somewhere will tell you how to do this, but where is that. Then you remember that the company that wrote the software has information online that will tell you how to do everything that your software can perform. But if you do not know the name of the procedure to use, how do you find the documentation about it?

My new book, Exchanging Data between SAS and Microsoft Excel: Tips and Techniques to Transfer and Manage Data More Efficiently is designed to help solve that problem by culling information from the SAS manuals and my personal experience into a document that shows you how to transfer data between SAS and Excel. In this first article on the subject, I will show you a simple way to transfer data to Excel with very little effort on your part. It is done with a “Right Click” of your mouse.

When viewing your SAS datasets in the SAS Explorer window all of the datasets have an icon or other display representing the SAS dataset. By using a “Right Click” on the dataset icon a menu appears with an option entitled “View in Excel”. Selecting this option creates an HTML file that Excel can open and use to view the data. In fact, SAS actually invokes Excel to open the HTML output file so you can use the data in Excel. The file will typically have a name similar to “#LNxxxxxx.xls”. The three byte extension (xls) allows Excel to open the file without hesitation prior to Excel version 2007. The newer versions check the contents of the file and if the file name ends in .xls but contains HTML or XML formatted commands for Excel then a message is displayed asking you to verify that you want to proceed.  Select “Yes” and Excel opens and your data appears.

Post a Comment

SAS author’s tip: Macro language timing is everything

This SAS tip is from Robert Virgile and his book “SAS Macro Language Magic: Discovering Advanced Techniques”.

We hope you find this tip useful. You can also read an excerpt from Virgile’s book.

In macro language, as in life, timing is everything.  Macro language students need to learn the timing of the DATA step, the timing of macro language, and the relationship between the two.

Let’s begin with the DATA step.  All DATA steps operate in two separate phases:

  1. The compilation phase. In a nutshell, the software checks the syntax of the DATA step statements, and sets up storage space in memory to hold each variable.
  2. The execution phase. Given that there are no syntax errors, the software executes the DATA step … reading data, performing calculations, outputting results.

Macro language statements may have an impact on step 1, the compilation phase.  The resolution of macro variables affects the statements within the DATA step:

%let dataset=MALES;
data &dataset;
   set everyone;
   if gender='M';
run;

During the compilation phase of the DATA step, &DATASET resolves into MALES.  Therefore, the name of the output data set becomes MALES.  However, macro language statements impact only the compilation phase, not the execution phase of the DATA step.  This concept forms a frequent stumbling block when learning macro language.  To illustrate, consider this DATA step (before the programmer complicated it by adding macro language):

data MALES FEMALES;
   set everyone;
   if gender='M' then output MALES;
   else if gender='F' then output FEMALES;
run;

Perhaps the programmer was trying to learn macro language, and using this as an experiment.  Perhaps the programmer sought job security.  But the simple DATA step above morphed into this nonworking version:

data MALES FEMALES;
   set everyone;
   if gender='M' then do;
      %let dataset=MALES;
   end;
   else if gender='F' then do;
      %let dataset=FEMALES;
   end;
   output &dataset;
run;

Mistakenly, the programmer believed that %LET statements could execute as part of the DATA step.  That is just never true.  %LET statements execute immediately … in this case before the compilation phase of the DATA step completes.  So the order of execution of these statements is:

%let dataset=MALES;
%let dataset=FEMALES;
data MALES FEMALES;
   set everyone;
   if gender='M' then do;
   end;
   else if gender='F' then do;
   end;
   output FEMALES;
run;

Clearly, the program revisions alter the outcome, forcing every observation into a single data set.  Remember these basics:

  • %LET statements are never part of a DATA step. Macro language statements execute immediately, and do not wait for the DATA step to begin executing.
  • If you need to control macro variables (either assigning or retrieving a value) while the DATA step executes, tools exist. But they are DATA step tools, not macro language tools.  The primary ones, CALL SYMPUT and SYMGET, will become the subject of a future article.

Let’s consider another example that both illustrates timing and illustrates a basic use of CALL SYMPUT.  Once again, improper use of macro language complicates the program.  Here is the original version, without macro language:

data percentages;
   state_pop=0;
   do until (last.state);
      set cities;
      by state;
      state_pop + city_pop;
   end;
   do until (last.state);
      set cities;
      by state;
      percent_pop = city_pop / state_pop;
      output;
   end;
run;

For each STATE:

  • The top DO loop computes STATE_POP (the total population for the STATE).
  • The bottom DO loop reads the same observations, computes PERCENT_POP for each, and outputs the result.

Now a macro language student might attempt a slightly different, nonworking variation:

data percentages;
   state_pop=0;
   do until (last.state);
      set cities;
      by state;
      state_pop + city_pop;
   end;
   call symputx ('denom', state_pop);
   do until (last.state);
      set cities;
      by state;
      percent_pop = city_pop / &denom;
      output;
   end;
run;

Bad timing is the critical issue:

  • Before the DATA step runs, &DENOM does not exist.
  • The software doesn’t begin to run the DATA step until it encounters the RUN statement.
  • By that time, the reference to &DENOM has already been encountered, generating an error.

There are many ways to introduce timing errors.  The remedy begins with understanding the relationship between macro language statements, DATA step compilation, and DATA step execution.  Most importantly, macro language statements execute immediately, and are never part of DATA step execution.

For more information about the macro language and the magic you can create with it, check out Robert Virgile’s book “SAS Macro Language Magic: Discovering Advanced Techniques”.

Post a Comment