Batch Processing with SAS®: Beyond Running Programs Overnight Lisa Sanbonmatsu, Consultant, Somerville, MA
echo "Step 4-Delete raw data after it has been zipped" ABSTRACT del "c:\Example\data08.dat" The usefulness of batch processing goes well beyond being able to submit several SAS® programs all at once. Batch processing allows you to automate steps involving several different software applications. This Placing “call” before an application command, rather than just streamlines the process, documents it, and reduces the chance of errors. typing the command, instructs the program to finish executing This paper discusses the use of DOS batch files, the SAS X command, the other application before continuing with the DOS batch file. and the SAS Call System routine. Tips for avoiding quoting problems are “Echo” displays comments during execution. Also note that provided. “del” should be used with extreme caution and never with wild cards.
INTRODUCTION The batch file was set up for the month of August. To run the Many of us are aware that SAS programs can be submitted in batch September report we could manually change all of the ‘08’s to mode and left running over night. However, the usefulness of batch ‘09’s in the file. Another solution would be to parameterize the processing goes beyond this convenience. By automating processes that number of the month. DOS parameters are similar to cross several different applications, batching allows you to streamline, parameters in a macro call. To insert the parameter into your document and reduce the likelihood of error. batch file, simply replace the values you want to parameterize with a ‘%1’ for the first parameter, a ‘%2’ for the second, etc. Consider the following situation. Each month your company needs to create a report from raw data through the following steps: Below is the revised code (“Report.bat”) which parameterizes 1) Use SAS to read and summarize the raw data, the month: 2) Use a conversion software application to convert the summary SAS data set to a spreadsheet, call "c:\Program Files\SAS\sas.exe" 3) Use a zip program to compress and archive the raw data, "c:\example\readsum%1.sas" 4) Use DOS to delete the unzipped raw data after it has been archived. call st "c:\Example\sum%1.sd2" "c:\Example\report%1.xls" Although each step could be implemented separately, this has several call "c:\Program Files\WinZip80\Wzzip.exe" disadvantages. First, it could be time consuming to monitor when each "c:\example\My Archive\archive%1.zip" application is finished so you can start the next. Second, when you need "c:\Example\data%1.dat" to run the report again, you will have no documentation of what you did. del "c:\Example\data%1.dat" And third, manually executing the different programs increases the likelihood of errors due to skipping a step or specifying a wrong To specify the month parameter when we run the batch file, we parameter. just add the parameter after the batch file name:
Batch processing avoids these problems. Batch processing can be >report.bat 09 achieved by writing a DOS batch file, using SAS’s X Command, or using SAS’s Call System routine. (Although this paper focuses on PC SAS, the basic principles of batch processing can be applied to UNIX.) If additional parameters had been used, they would just follow the 09: DOS BATCH FILES DOS batch files allow you to call and execute different programs >report.bat 09 2000 sequentially. DOS batch files are easy to write and execute. The steps are: The DOS Batch file can also be used to change the directory 1. Create a text file with the extension “.bat” location before running a command. For example, we could 2. In this text file, list each command you want to execute followed by a change the directory and then issue a copy command:
To produce the monthly report described in the introduction, we could X COMMAND create a text file called “august.bat” which contains the following code: While DOS batch files provide for some parameterization, batch processing from within SAS is even more flexible. The echo "Step 1-Call SAS to read in & summarize data" easiest way to batch process from within SAS it to use an X call "c:\Program Files\SAS\sas.exe" Command. The general format of the X Command is: "c:\example\readsum08.sas" echo "Step 2-Call transfer software to convert SAS “X” + single space + DOS (or UNIX) command + summary to a spreadsheet" semicolon call st "c:\Example\sum08.sd2" “c:\Example\report08.xls" The command can be unquoted or enclosed in single quotes. echo "Step 3-Call zip program to compress & archive However, if you follow the “X” with a quote, keep in mind that the raw data" SAS assumes that the command ends at the point where that call "c:\Program Files\WinZip80\Wzzip.exe" quote closes. For example, the following X command will not "c:\example\My Archive\archive08.zip" generate the intended results: “c:\Example\data08.dat" X "dir" "c:\example"; /* WRONG RESULT*/; This will yield an error message because each time an X The above command generates a directory listing of the current command is issued DOS is re-invoked. A second limitation of subdirectory rather than the “Example” subdirectory because everything the X Command is that it does not allow conditional execution-- after the closing of the first double quote is ignored. it is always executed immediately.
In this case, fixing the code is simple. We can either place single quotes CALL SYSTEM ROUTINE around the entire command or just start the command without any Call System is similar to the X Command but since it is a quotation mark: routine, it can be executed conditionally. For example, if we only wanted to delete archives that were over 6 months old, we X dir "c:\example"; /* CORRECT RESULT */ could use: X 'dir "c:\example"'; /* CORRECT RESULT */ %let month =09; data _null_; However, if we were not using a DOS prompt command or needed to do i = 1 to 12; insert a macro variable, these solutions would not have worked: if i < (&month - 6) then do; if i < 10 then del_com = /* GENERATES ERROR */; 'del "c:\Example\'|| X '"c:\Program Files\WinZip80\Wzzip.exe" 'My Archive\archive0' "c:\Example\My Archive\archive&month..zip" ||compress(i)||'.zip"'; "c:\Example\data&month..dat"'; else del_com = 'del "c:\Example\'|| 'My Archive\archive' ||compress(i)||'.zip"'; /* GENERATES ERROR */; call system(del_com); X c:\Program Files\WinZip80\Wzzip.exe end; "c:\Example\My Archive\archive&month..zip" end; "c:\Example\data&month..dat"; run;
In the first statement, the “&month” macro will not resolve because of the The general syntax of the Call System routine is: single quotes and in the second statement, the subdirectory will be seen as only “c:\Example\My” if the quotes are left off. The easiest solution to call system(
X Call "c:\Program Files\WinZip80\Wzzip.exe" The command can either be a character variable containing a "c:\Example\My Archive\archive&month..zip" command (such as del_com in the above example) or a "c:\Example\data&month..dat"; command enclosed in quotes:
Using “Call” we could run the monthly report from within SAS: call system(‘md c:\Example\Report09’);
%let month=08; or an expression that evaluates to a command: X call "c:\Program Files\SAS\sas.exe" "c:\example\readsum&month..sas"; call system('rd '||'c:\Example\Report09'); X call st "c:\Example\sum&month..sd2" "c:\Example\report&month..xls"; X call "c:\Program Files\WinZip80\Wzzip.exe" CONCLUSION "c:\example\My Archive\archive&month..zip" Batch processing allows you to automate across SAS and "c:\Example\data&month..dat"; other applications. For simple batch jobs, DOS batch files X del "c:\Example\data&month..dat"; often provide the best solution. For more complex parameters, X commands are also easy to use. However, for the most We could also specify how the X command operates with these global flexibility and conditional execution, the Call System is your options: best option.
XWAIT (default) - Keeps DOS shell open after executing command. The CONTACT INFORMATION user must close it by typing EXIT or clicking the close box. Lisa Sanbonmatsu Harvard University XNOWAIT - Closes DOS shell without any action by user. Malcolm Wiener Center for Social Policy 79 JFK Street XSYNC (default) - Finishes execution of the batch command before Cambridge, MA 02138 continuing to run the rest of the SAS program. Work Phone: 617-495-5131 Email: [email protected] XNOSYNC - Starts the batch command but returns to running the SAS program immediately whether or not the batch command has finished. SAS is a registered trademark or trademark of SAS Institute Inc. in the USA and other countries. ® indicates USA While the X Command is fairly flexible and can be executed anywhere in registration. your code (it is a global statement), it has two major limitations. First, each X Command is executed separately, this means that unlike with a Other brand and product names are registered trademarks or batch file, it makes little sense to try to alter the DOS environment: trademarks of their respective companies. X ‘cd..’; X cd “Example”; X cd “My Archive”; X copy “Archive&month.zip” “Backup&month.zip”;