Export and download any file from SAS Enterprise Guide

51

Last week I alluded to some very useful applications of the Copy Files task. This is one of them. If you have SAS Enterprise Guide 7.13 or later, the Copy Files task is in the Tasks->Data menu. In earlier versions, you'll have to download/install the task as a custom task.

Using the SAS programming language, you can manipulate data and create files of just about any size, shape, and format: Excel, PDF, CSV, RTF, and more. A challenge for SAS Enterprise Guide users has been: how to capture those files and bring them back to your local PC, when the SAS Workspace is running on a remote machine?

Example: Export to a CSV file and download the result

Here's a typical scenario: You have a simple SAS program that produces one or more CSV files that you will ultimately use in another program. How can you get the CSV files to your PC automatically?

STEP 1: Build a program step to create the CSV file
This program is easy to adapt for any data set and environment. It works on Windows and UNIX. All you need to know is the library and member name of the data that you want to export, and then the destination folder for your local PC. The program will perform the export operation, stage the CSV file in a temp location, and define the macro variables that the next step will use.

/* Data to export */
%let lib  =         sashelp;
%let datafile =     class;
 
/* Local folder to download to */ 
%let download_to =  c:\projects\data\results;
 
/* detect proper delim for UNIX vs. Windows */
%let delim=%sysfunc(ifc(%eval(&sysscp. = WIN),\,/));
 
%let download_from =
  %sysfunc(getoption(work))&delim.&datafile..csv;
 
filename src "&download_from.";
 
proc export data=&lib..&datafile.
  dbms=csv 
  file=src
  replace;
run;
 
filename src clear;

STEP 2: Use Copy Files task to download the result
The Copy Files task accepts SAS macro expressions. That's a key feature, as the macro variables we need are defined in the previous program step. Here's a screen shot of the task settings:

This makes the use of the Copy Files task very "generic". In fact, you can create a Task Template that defines these exact task settings, and thus always have it available on your Tasks menu directly.

STEP 3: Link these steps together in a process flow
Create a user-defined link between the program and the task, ensuring that they will run in the correct sequence.

THAT'S IT!
The power of SAS and the flexibility of the Copy Files task really makes this a simple operation. However, you might want to consider a few variations:

  • Export and download a collection of files in one step. With minor mods to the SAS program, you can loop through a collection of SAS data sets and export multiple CSV files. Instead of defining a single file to download, set the &DOWNLOAD_FROM variable to a file spec with a wildcard. The Copy Files task can handle wildcard notation -- no problem. (Well, no problem anymore, as long as you grab this update.)
    /* specify a wildcard */
    %let download_from =
      %sysfunc(getoption(work))&delim.%str(*).csv;
     
    /* file to create in step */
    filename src "%sysfunc(getoption(work))&delim.&datafile..csv";
  • Add a date stamp to your results file. You might have a requirement to keep older versions of your results. With a simple adjustment to the macro expression, you can append a date stamp to the files you create. This will ensure that even if you download the results to the same location each day, the previous results will not be replaced. When you download the file, the name with the date stamp will be intact.
    filename src 
     "%sysfunc(getoption(work))&delim.&datafile._%trim(%sysfunc(today(),date9.)).csv";

    Sample result from this step:

    NOTE: The file SRC is:
          Filename=/sas/work/class_19MAY2013.csv,
    

This is just one example of the useful things you can do with the Copy Files task. SAS users are a creative bunch. What other uses can you think of for this task?

Related articles

Copying files in SAS Enterprise Guide
Fixes for the Copy Files task in SAS Enterprise Guide

Share

About Author

Chris Hemedinger

Senior Manager, SAS Online Communities

+Chris Hemedinger is the manager of SAS Online Communities. He's also co-author of the popular SAS for Dummies book, author of Custom Tasks for SAS Enterprise Guide using Microsoft .NET, and a frequent participant on the SAS Enterprise Guide discussion forum.

Related Posts

51 Comments

    • Chris Hemedinger
      Chris Hemedinger on

      Andreas,

      Actually, I think that the 5.1 version of this task will work with 6.1 - simply follow the same installation instructions, but place in the ../6.1/Custom folder instead of the ../5.1/Custom folder.

      If it doesn't work for you, let me know.

      Chris

  1. Hi Chris, excellent task extension. I am having an issue with it in SAS EG 4.1 however.

    I am currently exporting a csv file with 65,000+ rows (280+ MB) from SAS to a network drive using the above method. Everything seems to go A-OK, however when you inspect the file after completion there are a substantial amount of blank spaces in the file. This happens apparently at random throughout the file, and it includes the header row.

    I have inspected the source data table, matched up a few specific rows and can confirm the data is there before export but not there after.

    Any idea what could be causing this?

    • Chris Hemedinger
      Chris Hemedinger on

      James,

      I'm not sure what's going on here, but you might experiment with another way to create the CSV file. For example, you could use DATA step with a FILE statement and PUT, or ODS CSV and PROC PRINT. This blog has some examples.

      Are you also using the Copy Files task to download the file from SAS to your PC (network drive)? This task for 4.1 (I think) simply does a binary file transfer. If your SAS session is UNIX and the destination is the PC, the line endings are treated differently. You might try opening the file in a text editor that can detect/convert these, such as TextPad or Notepad++.

    • Chris Hemedinger
      Chris Hemedinger on

      It's a good idea. Right now, the task supports wildcard notation, so you can copy all files in a folder with "*.*". But it does not support subfolders in this way. Part of the limitation is that, for now, the task does not create new folders at the destination. It can copy contents only into existing folders.

  2. Thank you for quick response. Whether I can create directories for all files I want to copy (I'm going to do this using dcreate function) - can I use this add-in for copying from local PC to different direcrories on net drive "in one action", other words without multiple adding this add-in to process flow? I dont know how I can cycle eg process flow, and not sure if it is possible..

    • Chris Hemedinger
      Chris Hemedinger on

      Antony, you can't "loop through" a series of directories with a single instance of the task. But you can create a "task template" of the operation, and easily re-use it throughout multiple projects (or the same project).

      Here's how -- first, add the task with the values you need for one of the directories. Specify all of the options as you need them. Then you can run it or click the Save button in the task to save it in your project without running it immediately.

      Then, right-click on the task in the process flow and select Create Task Template.... Assign a meaningful name to the template, such as "Copy PC files to Server". After you save the template, you'll find it in the Tasks->Task Templates menu or in the Task list window. When you select the template, it will be added to the project with all of your settings remembered. Simply tweak what you need to change for each instance. You'll still have several copies of the Copy Files task in your project, but the template will save you lots of time in defining them.

  3. Shashikanth Rai on

    Hi all, Is there any SAS code to export the SAS programs from EG file? I have an Option of "Export Program as a step in Project" but i don't want to use that as it takes more time to export each file. Please let me know if any code is available.
    Thank you

  4. Chris,

    Have you heard of any file size limitations? Your proc export sends the data to file=src where src is the fileref for the temporary work (i.e., %sysfunc(getoption(work)&delim.&datafile..csv). Could there be a problem if I a large table (~350 GB). Might I run out of space in the temporary directory?

    Thank you!

    • Chris Hemedinger
      Chris Hemedinger on

      Tomas,

      350GB is a very large file to store in scratch space -- yes, you might run out of space. You'll need to work with your system administrator to identify a location that might be appropriate for this. You can always change the example code that I provided to point to whatever agreed-upon path you come up with.

      But the download of a 350GB file is likely to be very slow-going -- over an hour on a Fast LAN connection. I don't think it's practical to move such large files around in this way.

  5. Hi there
    when I tried to download a SAS file 'HELP' using the above data export code I get the error message as shown here. Is it because I don't have proper authorization or something else ?

    ERROR: Insufficient authorization to access D:\HELP.csv.

    • Chris Hemedinger
      Chris Hemedinger on

      If you're using this exact code to test, it sounds like the &DOWNLOAD_FROM macro isn't building out properly. You should add a %PUT &DOWNLOAD_FROM.; statement in your code to see what the value is. In my test it shows as:

      %put &download_from;
      C:\Users\sascrh\AppData\Local\Temp\SEG7028\SAS Temporary Files\_TD9752_L77841_\Prc2\class.csv
      

  6. Hi Chris,

    I have been attempting to get this solution to work but keeping running into the below error:

    ERROR: Target folder (&download_to) does not exist or cannot be accessed on LIBP03P-7H3QRID

    I have tried a few different destination folders and below is the code I am using when attempting to export to my desktop. Any ideas or is this user permission related? Thanks:

    /* Data to export */
    %let lib  =         GRIDWORK;
    %let datafile =     MARKETING_NAMES_EXPORT;
     
    /* Local folder to download to */ 
    %let download_to =  c:\users\n0148198\desktop;
     
    %put &download_to;
    
    /* detect proper delim for UNIX vs. Windows */
    %let delim=%sysfunc(ifc(%eval(&sysscp. = WIN),\,/));
     
    %let download_from =
      %sysfunc(getoption(work))&delim.&datafile..txt;
     
    %put &download_from;
    
    filename src "&download_from.";
     
    proc export data=&lib..&datafile.
      dbms=tab
      file=src
      replace;
    run;
     
    filename src clear;
    

    • Chris Hemedinger
      Chris Hemedinger on

      Is "LIBP03P-7H3QRID" the name of your local desktop machine? And did you click the "Resolve macro variables in source and destination paths" checkbox on the Copy Files task? It looks like the task is not substituting in the macro variable values properly.

  7. Linda Schott on

    In EG 5.1 we had the Run and Save buttons in this Custom Task. Now in EG 7.1 we don't have the Run and Save buttons. We just have OK, which will automatically run the task. I would like to get that Save option back. Can we do so with EG 7.1?

    Here is the reply I got back from our SAS Support team when I posed this dilemma:

    'We do not have any control over the custom task. This custom task is written while back; therefore, some of the features may not compatible with EG 7.1 so that the SAVE option is omitted from EG 7.1.'

    • Chris Hemedinger
      Chris Hemedinger on

      Linda, the Run and Save button are in the 7.1 version of this task. Are you sure that you have the proper version installed? It should be SAS.Tasks.CopyFiles71.dll.

        • Yes, I also installed it on my EG 4.3 and it works perfectly, thanks.
          But I still have a question:
          Is there any way to call this task or task template in SAS code or macro programmingly rather than clicking the menu manually? if so, it will make this feature much more flexible.
          thanks.

          • Chris Hemedinger
            Chris Hemedinger on

            Eric, glad it's working.

            No, there isn't any SAS code that does the same operation, since the code runs on the SAS server and can't reach your local desktop.

            You could use FILENAME FTP or network paths (\\server\folder) to accomplish file-copy operations in code, but often that's not an option in client-server environments.

  8. Thanks Chris. Yes, ours is typical client-server environment. My scenario is: I run my SAS programs on our remote UNIX server via my EG 4.3 on my local Wins7, then the programs create some excel files there. What I am trying to do is to let the programs to copy or transfer these excel files back to my local machine. Is there any option to achieve this?

  9. Hi Chris,
    I am having to read data from sharepoint site (http://sharepoint.umsmed.edu/research...), the following code is not recognizing the files for me. Am I pointing this correctly? It read the file ok if I specify the path for excel file in a local drive
    options validvarname=any;

    libname xl XLSX "http://sharepoint.umsmed.edu/research/JHS/JHSFC/CONA_12312014.xlsx";

    proc datasets lib=xl; quit;

    libname xl CLEAR;
    Here is the warnig:WARNING: No matching members in directory.
    Thanks for the help

    • Chris Hemedinger
      Chris Hemedinger on

      Stanford,

      You can't assign a LIBNAME to a remote URL like that. You need to download the file first into your SAS session, then read it. I documented this technique in this post about Dropbox -- but the same method can apply to your SharePoint case. So something like this:

      filename _input "%sysfunc(getoption(work))/CONA_12312014.xlsx"; proc http method="get" url="http://sharepoint.umsmed.edu/research/JHS/JHSFC/CONA_12312014.xlsx" out=_input ; run; libname xl XLSX _input; proc datasets lib=xl; quit; libname xl CLEAR;

  10. Chris,

    also having an issue saying:
    WARNING: Apparent symbolic reference DOWNLOAD_FROM not resolved.
    WARNING: Apparent symbolic reference DOWNLOAD_TO not resolved.
    ERROR: Target folder (&download_to.) does not exist or cannot be accessed on USAZ29628W2C003

    when I trace back I'm actually getting an warning in the original code line:
    %let delim=%sysfunc(ifc(%eval(&sysscp. = WIN),\,/));
    WARNING: The quoted string currently being processed has become more than 262 characters long. You might have unbalanced quotation
    marks.

    full code I wrote was:
    /* DATA to EXPORT */
    %let lib = '/Retail_Services/mb28697/CSAT_SFT_FILES;
    %let datafile = usan;

    /* Local folder to download to */
    %let download_to = Z:\IVR Reporting\THD NLU\IVR Survey Results\USAN Files;

    /*detect proper delim for UNIX vs. Windows */
    %let delim=%sysfunc(ifc(%eval(&sysscp. = WIN),\,/));

    %let download_from =
    %sysfunc(getoption(work))&delim.&datafile..csv;

    filename src "&download_from.";

    proc export data=&lib..&datafile.
    dbms=csv
    replace;
    putnames=no;
    run;

    filename src clear;

    • Chris Hemedinger
      Chris Hemedinger on

      In the code you supplied, you left off a closing quote in this line:
      %let lib = '/Retail_Services/mb28697/CSAT_SFT_FILES;
      You're missing a closing single quote at the end. Should be:
      %let lib = '/Retail_Services/mb28697/CSAT_SFT_FILES';

  11. nm the previous comment. I'm acutally getting:
    libname CSAT_FLD "/Retail_Services/mb28697/CSAT_SFT_FILES";
    /* DATA to EXPORT */
    %let lib = csat_fld;
    %let datafile = usan;
    %put &lib..&datafile.;

    /* Local folder to download to */
    %let download_to = Z:\IVR Reporting\THD NLU\IVR Survey Results\USAN Files;

    /*detect proper delim for UNIX vs. Windows */
    %let delim=%sysfunc(ifc(%eval(&sysscp. = WIN),\,/));

    %let download_from =
    %sysfunc(getoption(work))&delim.&datafile..csv;

    filename src "&download_from.";

    proc export data=csat_fld.usan
    dbms=csv
    replace;
    putnames=no;
    run;

    filename src clear;
    ERROR: FILE= or TABLE= is required and must be specified.

    • Chris Hemedinger
      Chris Hemedinger on

      Hi, you left off the OUTFILE= option in PROC EXPORT, which tells SAS the name of the file to save. Add OUTFILE=&download_from., like this:
      proc export data=csat_fld.usan OUTFILE=&download_from. dbms=csv replace; putnames=no; run;

  12. Hi Chris, I modified the code in an attempt to export multiple SAS data sets that reside in my work directory to a single excel file, creating a worksheet for each dataset.
    I receive an error message stating ERROR: "WORK." is not a valid name.
    22: LINE and COLUMN cannot be determined.

    %let lib = work;
    %let datafile = %str(*);
    /* Local folder to download to */
    %let download_to = H:\PPACA Reports;

    /* detect proper delim for UNIX vs. Windows */
    %let delim=%sysfunc(ifc(%eval(&sysscp. = WIN),\,/));
    %let download_from = %sysfunc(getoption(work))&delim.%str(*).xlsx;

    /* file to create in step */
    filename src "%sysfunc(getoption(work))&delim.&datafile..xlsx";

    Proc Export data=&lib..&datafile.
    dbms=xlsx
    file=src
    replace;
    Run;

    • Chris Hemedinger
      Chris Hemedinger on

      Janet, while the Copy Files task can process wildcards like *, PROC EXPORT cannot. The way your code is written now, you're trying to do:

      PROC EXPORT DATA=WORK.*
      ...

      That's not going to work. You'll need to use a SAS macro to iterate through each data member in WORK and have a PROC EXPORT step with a SHEET= option.

      Then, your datafile value should be %str(*).xlsx, to download just the one XLSX file to your PC.

      Alternatively, you might be able to use LIBNAME XLSX to add multiple data sets to a single workbook. Requires SAS 9.4M2 or later.

      • Thanks for your prompt response Chris!
        Unfortunately I don't have SAS 9.4 - that looks like an easy solution.
        I did add a macro to identify/iterate the Work datasets, but I'm getting an error message associated with the &inMember. I have 3 datasets in work and the error message returns for each dataset. Example of error message:

        NOTE: Line generated by the macro variable "INMEMBER".
        67 WORK.RX_CLAIMS
        ______________
        22
        201
        ERROR 22-322: Syntax error, expecting one of the following: a name, a quoted string.
        ERROR 201-322: The option is not recognized and will be ignored.

        This is my complete code:
        %let filedate=&sysdate;
        %put filedate=&filedate;

        /*get libname and dataset name of data to export*/
        PROC SQL;
        CREATE TABLE DATASET_LIST AS
        SELECT libname, memname as member
        FROM dictionary.tables
        WHERE libname='WORK'
        and memname not in ("_PRODSAVAIL","DATASET_LIST");
        QUIT;

        %macro ExportReports();
        %local datasetCount iter inLibref inMember;

        /*get number of datasets*/
        proc sql noprint;
        select count(*) into :datasetCount
        from dataset_list
        quit;

        /*Initiate loop*/
        %let iter=1;
        %do %while (&iter. <= &datasetCount);

        /*get libref and dsn for dataset for this iteration*/
        data _NULL_;
        set DATASET_LIST (firstobs=&iter. obs=&iter.);

        /*write libname and dataset name to macro variables*/
        call symput("inLibref", strip(libname));
        call symput("inMember",strip(member)); /*strip fixes trailing blanks*/
        run;

        %let download_to = 'H:\PPACA Reports';
        %let download_from = &inLibref..&inMember;

        %let datafile = _&filedate..xlsx;
        Proc Export data=&datafile.
        dbms=xlsx
        file=&inLibref..&inMember.
        replace;
        sheet=&inMember.;
        Run;

        /*increment the iterator of the loop*/
        %let iter=%eval(&iter.+1);
        %end;
        %mend;

        /*call the macro*/
        %ExportReports()

        • Chris Hemedinger
          Chris Hemedinger on

          So close! I've created a simpler version that works.

          /* create some sample data */ data class; set sashelp.class; data cars; set sashelp.cars; run; %let filedate=&amp;sysdate; %let download_to = 'c:\temp\PPACA Reports'; %let download_from = %sysfunc(getoption(WORK))/_&amp;filedate..xlsx; %macro ExportReports(); /*get number of datasets*/ proc sql noprint; select memname into :member1 - :member9999 FROM dictionary.tables WHERE libname='WORK' and memname not in ("_PRODSAVAIL"); %let datasetCount = &amp;sqlobs.; quit; /*Initiate loop*/ %do iter=1 %to &amp;datasetCount; Proc Export data=WORK.&amp;&amp;member&amp;iter. dbms=xlsx file="&amp;download_from." replace; sheet="&amp;&amp;member&amp;iter."; Run; %end; %mend; /*call the macro*/ %ExportReports()

          • It works!! Hurray!!
            Last of all - the copy/files task doesn't like the &download_to macro, I removed the space in the download to file name but that didn't matter. Also tried single quotes instead of double and that didn't work either. I'm using a virtual machine (thus the reference below to V7333). If I hard-code the file path it works, but using the macro i receive this error message:
            ERROR: Target folder ("c:\temp\PPACA_Reports") does not exist or cannot be accessed on V7333

  13. This is resolved :) I removed the quotes from the &download_to path and the path is now recognized. Thanks again for your assistance with the macro!

  14. Hi Chris,

    I came across this functionality and it helps a lot in transferring files from server to my computer. However, I need to download multiple files from different folders with lots of sub-folders in a SAS Unix server to my computer. Is there anyway I can tweak this program to copy files in different folders?

    Thanks,
    Ben

    • Chris Hemedinger
      Chris Hemedinger on

      Hi Ben,

      No, I'm afraid not. The task supports just one source folder and doesn't follow into subfolders, so you would need to have multiple instances of the task to handle each distinct source directory. With the Task Template feature, you can save yourself time by creating a cookie-cutter task that you can apply quickly -- but you still need to have one per folder.

      • Chris,

        Is there a way to create CSV files all at once?

        I mean to read the &download_from macro to read multiple SAS datasets?

        Thanks Sunny

        • Chris Hemedinger
          Chris Hemedinger on

          Certainly! Instead of naming a specific file, you can use wildcards in your file specification, like "%sysfunc(getoption(work))&delim.*.csv" to pull ALL of the .CSV files from the Work folder.

    • Chris Hemedinger
      Chris Hemedinger on

      Sometimes Sharepoint is also set up as a network share (ex: \\myshare.company.com\documents\...), so that might be an option. Otherwise, you might have to try using PROC HTTP or FILENAME WebDAV methods to access the SharePoint content.

    • Chris Hemedinger
      Chris Hemedinger on

      If you're using DBMS=XLS, that creates a file that uses a very old format for Excel. XLSX is definitely the preferred format to use.

  15. How to export to multiple sheets in the same excel using the sheet= option? Its overwriting the original excel. If I uncheck the overwrite option, then its not writing at all. More precisely, if I have an existing excel and I only want to write to a particular sheet but keep other sheets unmodified can we do that

Leave A Reply

Back to Top