NESUG 18 Applications Using SAS to Perform Maintenance Tasks on ASCII Files Chris Moriak, AstraZeneca, Wilmington, DE ABSTRACT files in a data set. One can then use this information The ability of SAS® to read and write ASCII files later to loop through the files. The code for this step: makes it a good application for performing maintenance tasks on ASCII files. For example, filename myfile pipe “ls –1” *.sas when copying and modifying programs, often there is specific text, like a trial number, that needs to be data pgms; changed in all programs. It would be great if there length file $30 ; were a utility to make such global changes to all infile mydir ; programs within a directory. Another example is input file $ ; run; comparing two directories of ASCII output to identify any differences between them and to document the differences. SAS programmers dread comparing IMPORT FILE INTO DATA SET two SAS outputs to find what, if anything has Now that one has a useable list of filenames in SAS, changed between program runs. The manual one can begin servicing the file. One by one, import process of dredging through page after page is both the files into SAS by converting them into temporary inefficient and tedious. This paper will discuss how data sets with one variable. This task is SAS can help accomplish both these tasks. accomplished by using the INFILE and FILE statements in a data step: The platform used in this paper is UNIX, but one can modify it to use on other platforms. The files need to filename old “/user/temp/myfile.sas”; be in ASCII format (e.g. SAS or LST files). Topics in filename new the paper include the PROC COMPARE procedure, “/user/temp/myfile_new.sas”; obtaining a list of filenames from the operating system, converting the ASCII output into a data set, data temp; and the DATA _NULL_ statement to manipulate infile old missover sharebuffers files. length=len; file new; COMMON TECHNIQUES length outline inline $200; When performing maintenance tasks on files, the input @; process is the same: identify, retrieve, service, and input @1 inline $varying200. len; export. The two examples discussed in this paper use a common technique that can be applied to PERFORM TASK most maintenance tasks. Once the file line is read into the data set, then one can perform a task or function on that line. The line IDENTIFY FILES below replaces the string “find” with the string To identify a select group of files for processing, one “replace”. first needs to obtain a list of potential files. A SAS program can retrieve a list of files on the UNIX outline = trim(trandwrd(inline, platform by using the filename statement with a pipe: “find”, “replace”)); filename myfile pipe options. The pipe tells SAS that instead of identifying a file, information will EXPORT DATA SET TO FILE be passed to it via the filename. In order to pass a After performing the necessary tasks to the file line, list of files to SAS, the full filename statement is the last step is to export the observation back to the filename myfile pipe “ls –1” file.ext, file. The FILE statement shown above with the where “file.ext” is the file and extension name (wild following PUT statement accomplishes this task: cards can be used). The “ls –1” option tells UNIX to provide a list of files one entry per line. Next, add a data step using the filename to store the identified 1 NESUG 18 Applications ***Get program name; x = length(outline); %let prog = put @1 outline $varying200. x; %sysfunc(reverse(%scan(%sysfunc( run; reverse(&sysprocessname)),1,/))); filename mydir pipe "ls -1 *.sas"; A SEARCH AND REPLACE MACRO *** Create data set of SAS programs; PROGRAM data pgms; length file $30 ; infile mydir ; PROCESS input file $ ; if upcase(file)="%upcase(&prog)" Once one knows this technique of reading an ASCII then delete; file into SAS, servicing it, then exporting it out as a run; file, one can find many uses for it. One such use is performing a search and replace on one or more files in a directory. The steps for this task: 3. LOOP THROUGH FILENAMES 1. Macro Parameters Using PROC SQL, create a macro variable string 2. Get Filenames that contains the names of the files with a separator 3. Loop Through Filenames between each name. The SQL procedure will 4. Copy Files for Backup include a WHERE clause to subset the possible 5. Import File into Data Set files, and SQL will count the final number of files. 6. Find and Replace String The macro variable string allows SAS to loop 7. Export Data Set as File through each file. 8. Clean Up *** Create macro variable of program 1. MACRO PARAMETERS files; proc sql noprint; The macro parameters necessary for this utility: select trim(file) into : pgm separated by '~' DIR - Directory where files are from pgms located %if %length(&where)>0 %then FIND - String to find %do; REPLACE - Replacement string where &where WHERE - Clause to subset files %end; BACKUP - Whether to keep a backup of order by file ; the original file %let cnt=&sqlobs; 2. GET FILENAMES quit; The first step is to identify the number and names of the files in the directory. To do this, use an X- ***Loop through each file; command to change the operating system directory %do i=1 %to &cnt ; %let pgm1= %scan(&pgm,&i,~) ; to the requested directory. Set a filename statement with the pipe option to query the operating system for the filenames within the directory. Then, create 4. COPY FILES FOR BACKUP a data set containing the filenames. The SAS code The worst thing that can happen when performing a to perform these tasks is shown below. The record find and replace is for the user to mistakenly replace containing the program name is deleted so to not text. Therefore, one should allow the user to keep alter itself when performing the search and replace. backups of the original file. Add an X-command to This SAS code for identifying the program name copy the original files to a backup file with an “.xrp” may need to be modified depending on your system. file extension to show that this program created these files. ***Change the directory; x "cd &dir"; 2 NESUG 18 Applications put @1 outline $varying200. x; ***Copy files to a backup file; run; x "cp -f &pgm1 &pgm1..xrp"; %end; 5. IMPORT FILE INTO DATA SET 8. CLEAN UP It is time to read in each file line by line. Create two It is always a good idea to clean up the SAS filename statements following the technique outlined environment when a program finishes processing. earlier. One statement will be for the input file, the This means deleting temporary data sets and other for the output file. Then, read in the file line by clearing the filenames. If the user wishes not to line. One can choose to create a temporary data set keep the backup files, then delete those as well via or simply use a DATA _NULL_ to skip the data set an X-command. creation. ***Delete temporary data sets; ***Create filenames for input and proc datasets lib=work mt=data nolist; output files; delete pgms; filename old "&dir./&pgm1..xrp"; quit; filename new "&dir./&pgm1"; ***Clear filenames; ***Read in input file placing; filename new clear; data _null_; filename old clear; infile old missover sharebuffers filename mydir clear; length=len; file new ; ***Delete backup files if not wanted; length outline inline $ 200; %if %upcase(&backup) ne Y %then input @; %do; x "rm *.xrp" ; ***Input line from file; %end; input @1 inline $varying200. len; EXAMPLE MACRO CALL Here is an example of the macro call. In this 6. FIND AND REPLACE STRING example, the user is searching all the files in the Once the line is read into the data vector as a directory /project/study/pgms and changing every record, then one can perform a task on it. The task occurrence of the string ‘123’ to ‘456’. in this example is find and replace using the TRANWRD function. The additional TRIM function %xreplace(dir=/project/study/pgms will remove any trailing blanks. ,find=123 ,replace=456); ***Perform search and replace; outline=trim(tranwrd(inline,"&find", EXAMPLE RESULT "&replace")); All the programs in /project/study/pgms used to have the statement: libname final ‘/abc/sw-023-5412H123’; 7. EXPORT DATA SET AS FILE Now, all the programs in /project/study/pgms have After performing the find and replace on the record, the statement: it is time to export the record back to the ASCII file. libname final ‘/abc/sw-023-5412H456’; The FILE statement in the DATA _NULL_ tells SAS to output the data to a file instead of to a data set. To output the file, a PUT statement is necessary. Afterwards, add the %END statement to tell SAS to retrieve the next file. ***Output line back to file; x=length(outline); 3 NESUG 18 Applications infile dir2 ; COMPARING MULTIPLE ASCII FILES input file $ ; run; PROCESS 3. FIND MATCHING FILENAMES Another task that can be done using this technique is comparing multiple ASCII files within two For this utility, a prerequisite is that the files to directories. The steps for this task: compare must have the same name. Thus, the next 1. Macro Parameters step is to determine what files can be compared. 2. Get Filenames The user can modify the macro to add the option to 3. Find Matching Filenames specify two specific files to compare. A simple 4. Loop Through Comparison PROC SQL quickly determines the union.