Yes, SAS® Can Do! --- Manage External Files With SAS Programming (Paper 3262 - 2015)
Justin Jia, TransUnion Canada Amanda Lin, CIBC, Canada
April 26-29, 2015 Dallas, Texas, USA Overview
. Introduction . I. System Statements and Shell Commands X command and statement / %Sysexec macro statement/ Systask command . II. SAS Piping Functionality . III. SAS Call System Routine and System Function . IV. New SAS File Management Functions DCREATE function/ RENAME function/ FDELETE function/DLCREATEDIR option . V. Application Examples . Conclusion
2 Introduction . A good file management system is important for efficient documentation and project management. . It would be beneficial to automate and standardize repetitive routine tasks via SAS programming: create, rename, move, copy, delete. . Proc DATASETS, though powerful, is only effective for managing SAS files. . How can we manage non-SAS external files or directories?
3 I. System Statements and Shell Commands
. X <'command '> ; X " mkdir "&path.\ WLMU 999" ";
. %Sysexec macro statement %sysexec mkdir "&path.\WLMU 999" ;
. Systask statement systask command %unquote(%str(%') mkdir "&path.\WLMU 999" %str(%'));
4 Pros / Cons of System Statements/Commands . Advantages Not easy to monitor the execution status of submitted commands They use simple and easy DOS or (no return code). Unix commands. They cannot perform conditional Able to perform all kinds of file management tasks (create, execution since they are global. rename, copy, move, delete). data _null_; They are global commands and Action= 0; can exist anywhere in a SAS if Action=1 then do; program. X "mkdir "&path.\Try“ " ; . Disadvantages end; run; The X command will be executed every They prompt annoying flashing DOS window. time and the new directory Try will be created regardless of the value of Action.
5 II. SAS Piping Functionality . Piping is a channel of parallel communications External between SAS and other SAS Piping external applications. Program . The issued command will be directed to external programs, and the execution results will feed Unnamed Piping back to SAS via piping. FILENAME fileref PIPE . We can use SAS Filename statement to ‘program name /command ’ invoke unnamed piping.
6 Manage External Files via SAS Piping Error messages will write to SAS log for unsuccessful ***** Create a new directory.*****; executions. For example, if the directory already exists: 16 Filename Create PIPE "mkdir Filename Create PIPE &path.\By_Pipe "; "mkdir &path.\By_Pipe"; 23 data Create ; 24 infile Create; 25 input ; data _null_ ; 26 put _infile_; 27 run; infile Create; NOTE: The infile CREATE is: input ; Unnamed Pipe Access Device, put _infile_; *Write the PROCESS=mkdir D:\WLMU2015\Projects\By_Pipe, RECFM=V, LRECL=256 execution results to SAS Stderr output: log. A subdirectory or file D:\WLMU2015\Projects\By_Pipe already exists. run; NOTE: 0 records were read from the infile CREATE. NOTE: The data set WORK.CREATE has 0 observations and 0 variables.
7 III. SAS Call Routine and System Function . Call System Routine data Create; Name="pid1280"; CALL SYSTEM (command); NewDir= "&path.\"||strip(Name); Check= fileexist(NewDir); Command=character string/ if Check=0 then do; expression or a character variable. command="mkdir "||'"'||NewDir||'"'; call system(command); It is a local command rather than a RC=symget("SYSRC"); global one, it must be used in a /*RC= &SYSRC;*/ Not working! DATA step. /*RC=system(command);*/ if RC="0" then put "Note: Directories It is a callable routine, allowing Created Successfully: " NewDir; conditional execution. else put "Note: Directories Cannot Be Created: " NewDir; . System Function end; It issues an operating system command for execution during a else if Check=1 then put "Note: SAS session. Directory Already Exists: " NewDir; Run;
8 IV. New SAS File Management Functions DCREATE Function: create a new directory. . From SAS 9.2, some new Syntax: DCREATE (directory-name,
. A new function since SAS RENAME Function: rename a SAS file, 9.2. external file or directory. . Powerful and capable of working on both SAS files Syntax: and external files, RENAME(old-name, new-name , < type>); directories. . Easy and convenient to use. Old-Name: the current name of a file or directory. It supports library reference use. . It returns a return code of New-Name: the new name of a file or zero for successful execution, and non-zero directory. value for unsuccessful Type: specifies the element type, the possible execution. values are DATA, VIEW, CATALOG, ACCESS, FILE.
10 Rename files and directories
libname Test"&path.\Test“; *define a libref.
data _null_; /***rename a SAS data set using libref.;***/ RC1=rename("Test.AAA", "BBB", "data");
/*rename a SAS data set using the full pathname.;*/ RC2=rename("&path.\Test\AAA.sas7bdat", "&path.\Test\BBB.sas7bdat", "file");
/***rename an external file.;/ RC3=rename("&path.\Test\Sales.pdf", "&path.\Test\2012 Sales.pdf", "file");
/***rename a Windows directory.;/ RC4=rename("&path.\Test", "&path.\New_Test", "file");
/***rename a Unix directory.;/ RC5=rename("/&path./Test", "/&path./New_Test", "file"); Run;
11 Use Rename Function to move and delete files.
RENAME(old-name, new-name, data _null_;
Run; 12 FDELETE Function and DLCREATEDIR Option Syntax:FDELETE(fileref|directory); New global system option in SAS 9.3. When put in effect, the A new function since SAS 9.2, it can LIBNAME can automatically create a permanently delete SAS or non-SAS files new directory and then assign the and empty directories, must use a fileref libref, if it does NOT exist. rather than a physical name. The default NODLCREATEDIR It returns 0 for a successful operation and system option can cancel this effect. other value for an unsuccessful operation. options dlcreatedir; filename DEL "&path.\Test\Sales.pdf"; libname AAA "&path.\AAA"; data _Null_; NOTE: Library AAA was created. RC=fdelete("DEL"); NOTE: Libref AAA was successfully if RC=0 then put "Note: Files deleted assigned as follows: successfully. "; Engine: V9 else put "Files cannot be deleted.“; Physical Name: Run; D:\WLMU2015\Projects\AAA.
13 Summary of Advantages and Disadvantages
Advantages Disadvantages
I. Systems 1. They use simple and easy DOS, Unix or other OS commands. 1. They prompt annoying flashing DOS window. Statements and 2. They are global commands and can exist anywhere in a SAS program. 2. It is not easy to monitor the execution status of the Shell Commands 3. They are able to perform all kinds of file management tasks (create, submitted command (no return code). rename, copy, move, delete). 3. They cannot perform conditional execution since they are global. II. SAS Piping 1. It uses simple DOS, Unix or other OS commands, no fleeting DOS 1. It requires more SAS coding than other methods. Functionality window. 2. It cannot perform conditional execution since it is 2. We can look into SAS log to monitor the execution status. global. 3. It is able to perform all kinds of file management tasks (create, rename, copy, move, delete). III. SAS Call 1. They use simple DOS, Unix or other OS commands. 1. They are local and must exist in a SAS DATA step. Routine and 2. We can look into the return code to monitor the execution status. 2. They trigger the annoying fleeting DOS window. System Function 3. They are able to perform all kinds of file management tasks (create, rename, copy, move, delete). 4. They are local and callable commands, allowing conditional execution.
IV. New SAS File 1. They are true SAS functions with simple syntax, easy to use. 1. They are local and must exist in a SAS DATA step. Management 2. No fleeting DOW window. 2. They cannot perform all the file management tasks. Functions 3. They can work on both SAS files and non-SAS external files. 3. Only few functions are available for use in SAS. 4. They are local and callable, allowing conditional execution. 5. We can look into the return code to monitor the execution status.
14 V. Application Example I: Create Routine Folder Structure
For each project, we need create below %macro Create(path=, pid=); folder structure. options mprint symbolgen ; Filename List "physical path\List.xls"; Below Excel file controls the conditional if Create="Y" then do; procNewDir=Project_Folder||" import datafile=List\ "||strip(Folder_Name);out=List DBMS=Excel creation of wanted directories. replace;Sub_Check=fileexist(NewDir); getnames=Yes; guessingrows=1000; run; No Folder_Name Create dataif Sub_Check=0 Create; then do; 1 Archive Y setCreated=dcreate(strip(Folder_Name), List; 2 Code Y Project_Folder); 3 Data Y Project_Folder="&path.\"||"&pid."; 4 Input Y Folder_Name=upcase(Folder_Name);if not missing(Created) then put "Note: Directory 5 Output N created successfully: " NewDir; 6 Reports Y ifelse _N_=1 put then"Note: do; Directory cannot be created: " Check=fileexist(Project_Folder);NewDir; ifend; Check=0 then do; putelse "Note: if Sub_Check=1 Project folder then putdoes "Note: not exist Directory " Project_Folder;already exists: " NewDir; Created=dcreate(strip("&pid."),end; "&path."); ifrun; not missing(Created) then put "Note: Project folder%mend; created successfully: " Project_Folder; else put "Note: Project folder cannot be created: " Project_Folder; end; end; 15 V. Application Example II: Archive Out-dated Files.
MTH_Year=strip(substr(Report_Name, N-7)); If a file is older than a given DD_MTH_Year="15"||substr(MTH_YEAR, 1, duration(in months), it will be moved 3)||substr(MTH_YEAR, 5, 4); Age=INTCK("Month", input(DD_MTH_Year, date9.), to a specific Archive folder by a SAS today()); macro. run; data Move; %macro Archive(path=, pid=, duration=); length Old_Name New_Name $100; options mprint symbolgen ; set List; if Age > &Duration then do; Filename Files PIPE %unquote(%str(%') dir Old_Name="&path.\&pid.\Reports\"||strip(Full_N "&path.\&pid.\Reports" /b %str(%')); ame); New_Name="&path.\&pid.\Archive\"||strip(Full_N data List; ame); infile Files lrecl=256 truncover; RC=rename(Old_Name, New_Name, 'file'); input Full_Name $256.; if RC=0 then put "Note: Files moved to Archive Report_Name=strip(scan(Full_Name, 1, ".")); folder: " Old_Name ; Suffix=strip(scan(Full_Name, 2, ".")); else put "Note: Files cannot be moved: " N=length(Report_Name); Old_Name ; end; run; %mend; 16 Conclusion Acknowledgement
Managing external files and . Special thanks to directories plays an important part in TransUnion Canada and data analysis and business analytics. SAS Canada for the financial support. A good file management can help to streamline project management and . Thank you so much! improve work efficiency.
It will be both efficient and beneficial to automate and standardize the file management processes by using SAS programming.
17