Using SAS and ODS PACKAGE to create ZIP files

47

SAS users are big data consumers and big data creators. Often, we have to deal in large data files (or many smaller files) -- and that means ZIP compression. ZIP compression tools such as gzip, 7-Zip, and WinZip are ubiquitous, but they aren't always convenient to use from within a SAS program. To use an external ZIP utility you must issue a shell command via the X command or SYSTASK function, and that's not always possible within today's complex SAS environments.

Fortunately, SAS can read and write ZIP files directly. Ever since SAS 9.2, we've been able to create ZIP files with ODS PACKAGE. Beginning with SAS 9.4, we can read ZIP content by using FILENAME ZIP.

In this post, I'll review how to create ZIP files using ODS PACKAGE. I'll cover reading ZIP files with FILENAME ZIP in a future post.

Let's pretend that I'm working for a government agency, and that part of my job is to crunch some government data and publish it for the public. Of course, I'm using SAS for the analysis, but I need to publish the data in a non-proprietary format such as CSV. (It seems unbelievable, I know, but not every citizen is lucky enough to have access to SAS.)

First, I'll set up the output directory for this project. Since the ZIP file will contain a couple of files, including a subfolder, I want to mirror that structure here. The FEXIST and FDELETE functions will delete an existing ZIP file (perhaps left over from the last time I ran the process). The DLCREATEDIR option will create a "data" subfolder as needed. All of these mechanisms interact with the file system, but do not require XCMD privileges. This means that they'll work in SAS Enterprise Guide and stored processes.

%let projectDir = c:\projects\sgf2013\filenamezip;
 
/* Clean slate! */
filename newfile "&projectDir./carstats.zip";
data _null_;
  if (fexist('newfile')) then 
  	rc = fdelete('newfile');
run;
filename newfile clear;
 
/* Create folder if it doesn't exist */
options dlcreatedir;
libname out "&projectDir./data";

Next, I need to create the content to include in the ZIP file. In this scenario, I'm crunching some heavy-duty numbers about Cars data, and then putting the results into a CSV file. Then I'm creating a README file in RTF format; the document contains a simple data dictionary plus instructions (such as they are) for using the data. I used ODS TEXT to throw in some ad-hoc text among the SAS output.

/* Create some data */
filename newcsv "&projectDir./data/pct.csv";
proc means noprint data=sashelp.cars;
var msrp;
output out=out.pct median=p50 p95=p95 p99=p99;
run;
ods csv file=newcsv;
proc print data=out.pct;
format _all_; /* clear the formats */
run;
ods csv close;
 
/* Create an informative document about this package */
filename rm "&projectDir./readme.rtf";
ods rtf(readme) 
  file="&projectDir./readme.rtf" style=Printer;
ods rtf(readme) 
  text="These are some instructions for what to do next";
proc datasets lib=out nolist;
contents data=pct;
quit;
ods rtf(readme) close;

Finally, I'm going to take those results and package them in a ZIP file. The ODS PACKAGE mechanism was originally designed to share results from a SAS stored process. By default, it adds a PackageMetaData entry that a consuming SAS application could use to interpret the result. In this case we don't need this entry; the NOPF option suppresses it.

Notice that I specify the PATH= option to place the CSV file in the "data" folder within the archive. As soon as the ODS PACKAGE CLOSE statement executes, the ZIP file is created.

/* Creating a ZIP file with ODS PACKAGE */
ods package(newzip) open nopf;
ods package(newzip) add file=newcsv path="data/";
ods package(newzip) add file=rm;
ods package(newzip) publish archive 
  properties(
   archive_name="carstats.zip" 
   archive_path="&projectDir."
  );
ods package(newzip) close;

Here's a screen shot of the ZIP file opened in WinZip:

That's it! I can add any file that I want to the ZIP archive; I'm not restricted to files that were created by SAS. This makes it easy to use SAS as an automated method to update data archives regularly, creating user-friendly packages for consumers to make use of our data.

Note: A common question: does ODS PACKAGE (and FILENAME ZIP) support password-protected ZIP files (encryption)? The answer is No. If that's a requirement, you'll need to use an external package such as 7-Zip.

Download the complete program (SAS 9.3 or later): createZipODSPackage.sas

You might also enjoy:

Share

About Author

Chris Hemedinger

Senior Manager, SAS Online Communities

+Chris Hemedinger is the manager of SAS Online Communities. He’s also co-author of the popular SAS for Dummies book, author of Custom Tasks for SAS Enterprise Guide using Microsoft .NET, and a frequent participant on the SAS Enterprise Guide discussion forum.

47 Comments

  1. Michelle Homes

    Great walk through scenario stepping through functionality sasusers have been asking about in http://communities.sas.com lately.

    I particularly like that this technique does not require XCMD privileges to be enabled which will benefit many SAS Enterprise Guide users and SAS Administrators.

    Thanks Chris!

    Cheers,
    Michelle

  2. John Zimmerman on

    Thanks for the blog on using ODS to create zip files. Could you do a follow up on how to use SAS to "unpack" the same zip files?

  3. Pingback: Reading and updating ZIP files with FILENAME ZIP - The SAS Dummy

  4. Manya Ansari on

    Using SAS and ODS package I am using a program to read files from one location, compress and archive the files (ZIP) and FTP.

    Our end user who is receiving the FTP communicated that they are getting some Permission to unzip the File. they do not have issue when we manually zip the files and share to them.

    Can you please update me whether zipping the files using SAS ODS package is any different from normal zip? What might be the reason they are not able to unzip the file.

    I am using the same SAS code mentioned in the blog for my process.

  5. Chris, Thanks for the excellent post. Have you found a way to write a self-extracting zip file? I'm using this technique to generate a directory structure and ods html pages with drill-down links and distributing to users as a zip. In practice, many users (in windows) open/read files directly from the Zip (without first extracting to location) and this breaks the links between pages. My work around is to write self-extracting zips - that forces users to first extract files. However, I have yet to find a way to automate the generation of the self-extracting zips. Any thoughts are appreciated.

    • Chris Hemedinger
      Chris Hemedinger on

      Evan, that's an interesting problem. I think to make that work, you would have to automate an external tool such as WinZip Self-Extractor.

      Distributing an EXE has its own challenges, as you probably know. If you surface these in e-mail or in a web page download, the security settings can get in your way. These apps are slightly more permissive for standard ZIP files.

  6. Thanks for your code to zip file using ODS PACKAGE. Is there any way unzip all or few files?

    I am using SAS Eguide 9.1.3. Is it possible to use FILENAME ZIP?

    • Chris Hemedinger
      Chris Hemedinger on

      Sujatha,

      Unfortunately, FILENAME ZIP is new to SAS 9.4. And ODS PACKAGE was added in SAS 9.2. If you're still using SAS 9.1.3, then you are limited to using OS shell commands to zip/unzip files with external utilities.

    • Chris Hemedinger
      Chris Hemedinger on

      Melissa,

      Currently ODS Package and FILENAME ZIP do not support password-protected ZIP files. Many users with this requirement use SYSTASK (or similar OS shell calls) to invoke gzip or other external tools. It's not as convenient as a built-in SAS language method, but it's a workaround.

  7. Eric Hoogenboom on

    Hello Chris,

    Zipping large datasets (> 4 Gb) is also possible using ODS. However, unzipping in Windows 7 does not work and returns a "damaged file" error. It seems that the original Winzip format is not 100% compliant to the SAS adaptation. This only happens when the packed files are really big. Do you know a solution for this? I am currently using 9.3.

    Thanks, Eric

    • Chris Hemedinger
      Chris Hemedinger on

      The ODS PACKAGE method of creating zip files (available since SAS 9.2) is not as current as the FILENAME ZIP method. For very large files that need the zip64 capabilities, the FILENAME ZIP method is better. You can add files that have an uncompressed size > 4GB. But FILENAME ZIP was introduced in 9.4, and isn't available in 9.3.

      In the initial release of 9.4 there was a limitation when your FILENAME ZIP archive file (the compressed ZIP file) exceeds 4GB -- the resulting file could not be read. That has been fixed in 9.4 Maint 2.

  8. Hi Experts,

    How to create SAS permanent library

    libname xys " "; is not permanent library code because suppose i close SAS window and again starts the library disappears.
    Manually I created permanent library by selecting enable at startup option. however I want code

    Regards;
    Jags

    • Chris Hemedinger
      Chris Hemedinger on

      Yes, it should be able to handle that. XML files (as text files) should compress nicely. FILENAME ZIP supports the zip64 standard which allows for large zip file members and archive sizes.

  9. Sanket Sinojia on

    Hi Chris,
    my query is some what off from this topic.
    I want to create RTF files and PDF files from ODS and those file has to be "READ only".
    Any way in SAS to create READ only outputs ?
    Thanks in advance.

    • Chris Hemedinger
      Chris Hemedinger on

      There isn't a direct method in the SAS language to create a read-only file (after all, the file has to be writable as you create it!), but you can use X command or SYSTASK to apply the ReadOnly attribute after the file exists. This conference paper has some tips. Excerpt:

      systask command "attrib +r c:\temp\*.*" wait status=attrfl; 
      

  10. I am using the below code to zip then file but no data is coming in the zip folder.
    pls help..

    ods tagsets.ExcelXP file="/SAS/BIU/ADHOC/OUTPUT/SOLID_WISE_ACH.xls";

    proc report data=SOLID_WISE_ACH style(header)=[background=CX732E53
    foreground=white];
    col geography
    region
    circle
    cluster
    asc
    sol_id
    ( 'YTD'
    Target
    Disb
    "% Achievement"n
    LAG
    "Home Loans"n
    "Auto Loans"n
    "CV-CE"n
    EEG
    LAP
    LAS
    "Personal Loans"n
    EL
    )
    ('MTD'
    Target_MTD
    Disb_MTD
    Ach_mtd
    lag_mtd
    homeloan_mtd
    autoloan_mtd
    cv_ce_mtd
    eeg_mtd
    lap_mtd
    las_mtd
    pl_mtd
    el_mtd
    );
    define Target_MTD/display "Target";
    define Disb_MTD/display "Disb";
    define Ach_mtd/display "% Achievement";
    define lag_mtd/display "LAG";
    define homeloan_mtd/display "Home Loans";
    define autoloan_mtd/display "Auto Loans";
    define cv_ce_mtd/display "CV-CE";
    define eeg_mtd/display "EEG";
    define lap_mtd/display "LAP";
    define las_mtd/display "LAS";
    define pl_mtd/display "Personal Loans";
    define el_mtd/display "EL";
    run;

    ods package(SOLID_WISE_ACH) open nopf;
    ods package(SOLID_WISE_ACH) add file="/SAS/BIU/ADHOC/OUTPUT/SOLID_WISE_ACH.xls";
    ods package(SOLID_WISE_ACH) publish archive
    properties(archive_name="SOLID_WISE_ACH.zip"
    archive_path="/SAS/BIU/ADHOC/OUTPUT");
    ods package(SOLID_WISE_ACH) close;

    • Chris Hemedinger
      Chris Hemedinger on

      Ankit,

      This might be a simple answer, but I noticed that your program does not have:

      ods tagsets.ExcelXP CLOSE;

      Before the ODS PACKAGE statements. I think you need to CLOSE the ExcelXP destination before adding the file to the package.

  11. Valentina Aguilar on

    Hi Chris, thanks for this very easy to follow example. I noticed that the date that appears in the zip file for each of the archives files is the same as the date of creation of the zip and not the date of the file itself. Do you know if there is a way to keep the original date?

    • Chris Hemedinger
      Chris Hemedinger on

      Unfortunately, none of the SAS language methods (ODS PACKAGE, FILENAME ZIP) support passwords. You'll have to use the "old school" method: use X command or SYSTASK to call 7zip or gzip commands to compress with a password.

  12. Thanks for the post Chris.
    Can we add any file to an existing archive created by using FILENAME ZIP without deleting the compressed files in the archive?
    Thanks,
    Siva

  13. Pingback: Add files to a ZIP archive with FILENAME ZIP - The SAS Dummy

  14. I've a loop program and continue to add files to the zip folder in SAS. How can I accomplish adding the files in loop.
    I'm creating 10 CSV file when 10 loops are executed. Every time loops ends, it creates a file that needs to be zipped.
    How can I accomplish this ?
    Thanks in advance !

  15. Mary Rosenbloom on

    One more thing, so far when I am using ODS package it is changing the file modified date and time when it zips it. But, when I use the X statement it does not:

    options symbolgen;
    %let newpath = C:\Program Files\7-Zip;
    %let newzip = C:\try.zip;
    %let newfile = C:\migrate\cars.sas7bdat;
    data _null_;
    x %str(%'&newpath\7z.exe%' u -tzip "&newzip" "&newfile" -r );
    run;

    This x statement code is from here:
    https://groups.google.com/forum/#!msg/comp.soft-sys.sas/QQ3i6n0xdrk/PaOTlrUKHhYJ

    • Chris Hemedinger
      Chris Hemedinger on

      Hi Mary, good observation. Not sure why that is, except that perhaps the underlying implementation is creating a new copy of the file and not retaining the file attributes as it does so. If those file attributes are a requirement, ODS PACKAGE and FILENAME ZIP might not be all that useful as-is.

  16. Hi - Thanks for the article. I have tried this code and was successfully able to create the zip file for the first time the program runs. However after making minor updates and run the program again I get this error message "ERROR: Resource is write-locked by another thread".

    Could you please help me resolve this?

  17. Can you zip a directory and its contents in one shot? Or do you have to create the path and add each file? For 9.2, ODS packages please. Thanks.

  18. Hi All,
    I am using SAS EG 4.3.
    Pls help me on below query.

    How to unzip .sas7bdatd files using sas eg

    Many Thanks

  19. Pingback: Using FILENAME ZIP and FINFO to list the details in your ZIP files - The SAS Dummy

Leave A Reply

Back to Top