SESUG 2016 Paper CC–134 Using SAS® to Get a List of File Names and Other Information from a Directory in Microsoft® Windows Imelda C. Go, South Carolina Department of Education, Columbia, SC ABSTRACT The ability to create a data set which contains the names of all the files in a directory can be very useful. This listing can be easily generated using the Disk Operating System (DOS) DIR command via the Windows command prompt. Within SAS, you can use the PIPE option in a FILENAME statement and the DIR command to generate the listing. This paper goes through three possible uses of this data set with the file listing. The first example is a situation where many files with the same input structure need to be read from the same directory. Instead of hard-coding the file name to create each of the many data sets, apply a SAS macro on each file name in the directory listing to create each data set. The second example involves the need to collect data files and determine which files are still missing. The third example is a situation where you are running out of computer storage space and need to identify files based on file size/type and/or age for possible deletion or other action. An often overlooked source of solutions is the SAS Knowledge Base online. Two SAS samples (24820 and 41880) can be used to address the above situations. Generating a text file which contains file information for a directory, or a directory and all of its subdirectories is easily accomplished using a DOS command issued at the Windows command prompt. Disk Operating Systems (DOS) started in the early 80s and are operating systems that use the command line. DOS was the most common operating systems for IBM-PCs and compatibles from the 80s till the 90s. As true DOS systems became obsolete, the Microsoft Windows operating systems continued to provide users with access to the command prompt. Younger programmers may have never worked with the command prompt in a true DOS environment. Let us first discuss how a user can obtain the directory listing through the Windows command prompt: The DIR DOS command provides a listing of files. The information in the listing will depend on the exact form of the DIR command used. Options and parameters could vary, which change the exact form of the listing. A search online for “DIR DOS commands” will lead to many references regarding the DIR DOS command. Just be aware that exact syntax specifics might vary depending on the DOS version being referred to. The DIR syntax notes provided in this paper are not exhaustive. Once the user decides on the final form of the DIR command to use, the redirection operator may be used to send the output of the command to a file. The listing may be redirected using the > or the >> operator. The > operator writes the output of the command into the specified file. For example, the following command generates the default listing for the c:\ directory and writes the listing into c:\dircontents.txt dir c:\ > c:\dircontents.txt The file specified is created if it did not exist prior to the DIR command being issued and the file is overwritten if the specified file already exists. Instead of overwriting an existing file, the >> redirection appends the DIR command output to the contents of the file specified. dir c:\ >> c:\dircontents.txt Using SAS® to Get a List of File Names and Other Information from a Directory in Microsoft® Windows, continued SESUG 2016 DIR Syntax The basic syntax will look like this where anything in brackets and parentheses is an option: DIR [d:][path][filename][/A:(attributes)][/O:(order)][/B][/C][/CH][/L][/S][/P][/W] Here are a few basic forms of the DIR command: dir displays the listing for the current directory dir *.csv displays the listing for files with a .csv extension in the current directory dir c:/ *. displays files with no file extension in the c:/ directory /A Switch and Attributes [/A:(attributes)] The attributes help limit the listing of files and directories in the path. The specific attributes are specified right after /A:. Attributes that help limit the listing include: • hidden (H) or not hidden (-H) files • system (S) or non-system (-S) files • directories only (D) or files only (-D) • read-only files (R) or read/write files (-R) For example: /A:HS is for a listing with hidden system files only /A:-DR is for read-only files only and excludes directories in the listing /O Switch and Order Options [/O:(order)] There are DOS options that affect the order of the listing. This is not really critical since the data can be sorted within SAS. Sort options include: • by alphabetical file name (N) or reverse alphabetical file name (-N) • by alphabetical extension (E) or reverse alphabetical extension (-E) • by chronological date and time (D) or reverse chronological date and time (-D) • by increasing size (S) or decreasing size (-S) Other Switches There are other switches that can be useful. In particular, the /S switch is important for this paper’s third example. • /S displays the listing in the specified directory ([path]) and all subdirectories below it. • /L which displays the listing in lowercase letters. • /W displays only the file and directory names and excludes other file information but in a five-column format. 2 Using SAS® to Get a List of File Names and Other Information from a Directory in Microsoft® Windows, continued SESUG 2016 THE DIR LISTING FROM THE COMMAND PROMPT Let us look at how we can acquire this listing via the command prompt in a Windows system. Let us suppose the following three files are in c:\Users\igo\Desktop\example1 Open the Windows command prompt and then make your way to the directory by issuing the CD (change directory) command. (A discussion of the CD command’s syntax is not included in this paper.) In the example below, we are now in the c:\Users\igo\Desktop\example1 directory. We then issue the DIR command in its simplest form without parameters, attributes, or switches. In its simplest form, the DIR command provides a listing in the current directory. The listing is displayed on the screen. We can issue the “dir > example1dir.txt” command to save the listing to a file. This command takes the listing generated by the DIR command and writes it into the example1dir.txt file in the same (current) directory. If we issue the DIR command right after issuing the redirection command, we see that the example1dir.txt file is in the current directory. Since we did not specify a different path for example1dir.txt, the TXT file was written in the current directory, which is c:\Users\igo\Desktop\example1 in the example. 3 Using SAS® to Get a List of File Names and Other Information from a Directory in Microsoft® Windows, continued SESUG 2016 The text file has the same contents that the DIR command displayed in the command prompt window. We can observe that most of the file details are displayed in fixed width format and that it is possible to isolate the information for the date, time, size in bytes, and the file name by counting the column positions. The strategy is simple: generate a listing with the DIR command and then create a data set using the information in the file. The following example demonstrates the use of the /s and /a switches. The /a:-d portion of the command specified that only files be included in the listing. Therefore, directories will not be included. 4 Using SAS® to Get a List of File Names and Other Information from a Directory in Microsoft® Windows, continued SESUG 2016 PIPE OPTION AND DIR COMMAND IN FILENAME STATEMENT Now that we know what the DIR command produces, let us look at what we can do from within SAS directly. Using a FILENAME statement with the PIPE option and the DIR command, we can generate the listing and use a DATA step to extract the data elements we need. The example below shows that the path for which you need a listing for is in double quotes (“c:\Users\igo\Desktop\example1”) and the exact DIR command specification is in single quotes. The DIR command below specifies the directory path for which you need the listing for. This technique involves an unnamed pipe, which allows you to take the output generated by the DIR command and use it as input for the data set. Note how you can do this directly in SAS and not need the listing to be saved in a file as illustrated above with the redirection of output. The listing variable shown below contains every line in the listing. We use the fact that the listing contains data in fixed width format. We are able to use the SUBSTR function to isolate the filename value. The datasetname value is just the filename value in compressed form without spaces and the period. This is just an arbitrary dataset name used for the sake of providing an example. Right now we have more lines than needed. We can fix this by adding another line of code that eliminates the extra lines and leaves us with observations 8-11 only. This is a very simple example and the condition for deletion works for this example. Depending on the actual from of the DIR listing, you might have to adjust the condition for deletion. The need to adjust the condition will occur in Example 3.