Introduction to Unix Security for Security Practitioners

84-01-19

DATA SECURITY MANAGEMENT INTRODUCTION TO UNIX SECURITY FOR SECURITY PRACTITIONERS

Jeffery J. Lowder

INSIDE Operating System Security Services; Identiﬁcation and Authentication; Access Control; Availability and Integrity; Audit; Security Facilities for Users; Vulnerabilities in Traditional UNIX; TCP Wrappers; Login or Warning Banners

In an age of increasingly sophisticated security tools (e.g., ﬁrewalls, vir- tual private networks, intrusion detection systems, etc.), many people do not consider operating system security a very sexy topic. Indeed, given that the UNIX operating system was originally developed in 1969 and that multiple full-length books have been written on protecting UNIX machines, one might be tempted to dismiss the entire topic as “old hat.” Nevertheless, operating system security is a crucial component of an overall security program. In the words of Anup Ghosh, the operating system is “the foundation for any software that runs on a machine,” and this is just as true in the era of E-commerce as it was in the past. Thus, security practitioners who are even indirectly responsible for the protection of UNIX machines need to have at least a basic understanding of UNIX security. This article attempts to address that need by providing an overview of security services common to all ﬂavors of UNIX; security mech- anisms available in trusted UNIX are beyond the scope of this article (but see Exhibit 1).

OPERATING SYSTEM SECURITY SERVICES PAYOFF IDEA Summers1 (1997) lists the following Operating system security is a crucial component security services that operating sys- of an overall security program. Thus, security tems in general can provide: practitioners who are even indirectly responsible for the protection of UNIX machines need to have at least a basic understanding of UNIX security. 1. Identiﬁcation and authentica- This article addresses that need by providing an tion: A secure operating system overview of security services common to all ﬂa- vors of UNIX.

EXHIBIT 1 — Versions of Trusted or Secure UNIX

A1 (Veriﬁed Design) No operating systems have been evaluated in class A1 B3 (Security Domains) Wang Government Services, Inc., XTS-300 STOP 4.4.2 B2 (Structured Protection) Trusted Information Systems, Inc. Trusted XENIX 4.0 B1 (Labeled Security Protection) Digital Equipment Corporation ULTRIX MLS+ Version 2.1 on VAX Station 3100 Hewlett Packard Corporation HP-UX BLS Release 9.09+ Silicon Graphics Inc. Trusted IRIX/B Release 4.0.5EPL C2 (Controlled Access Protection) No UNIX operating systems have been evaluated in class C2 C1 (Discretionary Access Protection) Products are no longer evaluated at this class D1 (Minimal Protection) No operating systems have been evaluated in class D1

Note: Various versions of UNIX have been evaluated by the U.S. Government’s National Security Agency (NSA) according to the Trusted Computer System Evaluation Criteria. (By way of comparison, Microsoft Corporation’s Windows NT Worksta- tion and Windows NT Server, version 4.0, have both been evaluated at class C2.) The above chart is taken from the NSA’s Evaluated Product List.

must be able to distinguish between different users (identiﬁcation); it also needs some assurance that users are who they say they are (authentication). Identiﬁcation and authentication are crucial to the other operating system security services. There are typically three ways to authenticate users: something the user knows (e.g., a password), something the user has (e.g., a smart card), or something the user is (e.g., a retinal pattern). Passwords are by far the most common authentication method; this method is also extremely vulnerable to compromise. Passwords can be null, easily guessed, cracked, written down and then discovered, or “sniffed.” 2. Access control: An operating system is responsible for providing log- ical access control through the use of subjects, objects, access rights, and access validation. A subject includes a userID, password, group memberships, privileges, etc. for each user. Object security information includes the owner, group, access restrictions, etc. Basic access rights include read, write, and execute. Finally, an operating system evaluates an access request (consisting of a subject, an object, and the requested access) according to access validation rules. 3. Availability and integrity: Does the system start up in a secure fash- ion? Does the system behave according to expectations during an attack? Is the data on the system internally consistent? Does the data correspond with the real-world entities that it represents?

4. Audit: An audit trail contains a chronological record of events. Audit trails can be useful as a deterrent; they are even more useful in inves- tigating incidents (e.g., Who did it? How?). Audit trails have even been used as legal evidence in criminal trials. However, for an audit trail to be useful in any of these contexts, the operating system must record all security-relevant events, protect the confidentiality and integrity of the audit trail, and ensure that the data is available in a timely manner. 5. Security facilities for users: Non-privileged users need some method for granting rights to their files and changing their passwords. Privi- leged users need additional facilities, including the ability to lock accounts, gain access to other users’ files, configure auditing options, change ownership of files, change users’ memberships in groups, etc.

The remainder of this article explores how these services are implemented in the UNIX family of operating systems.

IDENTIFICATION AND AUTHENTICATION UNIX identifies users according to usernames and authenticates them with passwords. In many implementations of UNIX, both usernames and passwords are limited to eight characters. As a security measure, UNIX does not store passwords in plaintext. Instead, it stores the password as ciphertext, using a modified Digital Encryption Standard (DES) algorithm (crypt) for encryption. The encrypted password, along with other perti- nent account information (see Exhibit 2), is stored in the /etc/passwd file according to the following format:

username:encrypted password:UserID:GroupID:user’s full name:home directory:login shell

Unfortunately, the /etc/passwd ﬁle is world-readable, which can place standard, “out-of-the-box” conﬁgurations of UNIX at risk for a brute-force password-guessing attack by anyone with system access. Given enough computing resources and readily available tools like Alec Muffet’s crack

EXHIBIT 2 — Sample /etc/passwd Entries

keith::1001:15:Keith Smith:/usr/keith:/bin/csh greg:Qf@14pL1aqzqB:Greg Jones:/usr/greg/:/bin/csh cathy:*:1003:15:Cathy Jones:/usr/cathy:/bin/csh

Note: In this example, user keith has no password, user greg has an encrypted password, and user cathy has a shadowed password.

utility, an attacker can eventually guess every password on the system. In light of this vulnerability, all current implementations of UNIX now provide support for so-called “shadow” passwords. The basic idea is to store the encrypted passwords in a separate file (/etc/shadow to be exact) that is only readable by the privileged “root” account. Also, although vanilla UNIX does not provide support for proactive password checking, add-on tools are available. Finally, password aging is not part of standard UNIX but is supported by many proprietary implementations. UserIDs (UIDs) are typically 16-bit integers, meaning that they can have any value between 0 and 65,535. The operating system uses UIDs, not usernames, to track users. Thus, it is entirely possible in UNIX for two or more usernames to share the same UID. In general, it is a bad idea to give two usernames the same UID. Also, certain UIDs are reserved. (For example, any username with an UID of zero is considered root by the operating system.) Finally, UNIX requires that certain programs like /bin/passwd (used by users to change their passwords) and /bin/login (executed when a user initiates a login sequence) run as root; however, users should not be able to arbitrarily gain root permissions on the system. UNIX solves this problem by allowing certain programs to run un- der the permissions of another UID. Such programs are called Set UserID (SUID) programs. Of course, such programs can also be risky: if attackers are able to interrupt an SUID program, they may be able to gain root access and ensure that they are able to regain such access in the future. Group IDs (GIDs) are also typically 16-bit integers. The GID listed in a user’s entry in /etc/passwd is that user’s primary GID; however, in some versions of UNIX, a user can belong to more than one group. A complete listing of all groups, including name, GID, and members (users), can be found in the file /etc/group. Once a user successfully logs in, UNIX executes the global file /etc/profile along with the .profile file in the user’s home directory using the user’s shell specified in /etc/passwd. If the permissions on these files are not restricted properly, an attacker could modify these files and cause unauthorized commands to be executed each time the user logs in. UNIX also updates the file /usr/adm/lastlog, which stores the date and time of the latest login for each account. This information can be obtained via the finger command and creates another vulnerability: systems with the finger command enabled may unwittingly provide attackers with useful information in planning an attack.

ACCESS CONTROL Standard UNIX systems prevent the unauthorized use of system resources (e.g., ﬁles, memory, devices, etc.) by promoting discretionary access control. Permissions are divided into three categories: owner, group, and other. However, privileged accounts can bypass this access control. UNIX

treats all system resources consistently by making no distinction between files, memory, and devices; all resources are treated as files for access control purposes. The UNIX filesystem has a tree structure, with the top-level directory designated as /. Some of the second-level directories are standards. For example, /bin contains system executables, /dev contains devices, /usr contains user files, etc. Each directory contains a pointer to itself (the ‘.’ file) and a pointer to its parent directory (the ‘..’ file). (In the top-level directory, the ‘..’ file points to the top-level directory.) Every file (and directory) has an owner, a group, and a set of permissions. This information can be obtained using the ls -l command:

drwxr-xr-x 1 jlowder staff 1024 Feb 21 18:30 ./ drwxr-xr-x 2 jlowder staff 1024 Oct 28 1996 ../ -rw------3 jlowder staff 2048 Feb 21 18:31 file1 -rw-rw---- 4 jlowder staff 2048 Feb 21 18:31 file2 -rw-rw-rw- 5 jlowder staff 2048 Feb 21 18:31 file3 -rws------6 jlowder staff 18495 Feb 2118:31 file4

In the above example, file1 is readable and writable only by the owner; file2 is readable and writable by both the owner and members of the ‘staff’ group; file3 is readable and writable by everyone; and file4 is readable and writable by the owner and is a SetUID program. Devices are displayed a bit differently. The following is the output of the command ls -l /dev/cdrom /dev/tty02:

br------1 root root 1024 Oct 28 1996 /dev/cdrom crw------2 root root 1024 Oct 28 1996 /dev/tty02

UNIX identifies block devices (e.g., disks) with the letter ‘b’ and character devices (e.g., modems, printers) with the letter ‘c.’ When a user or process creates a new file, the file is given default permissions. For a process-created file (e.g., a file created by a text editor), the process specifies the default permissions. For user-created files, the default permissions are specified in the startup file for the user’s shell program. File owners can change the permissions (or mode) of a file by using the chmod (change mode) command. UNIX operating systems treat directories as files, but as a special type of file. Directory “files” have a specified structure, consisting of filename- inode number pairs. Inode numbers refer to a given inode, a sort of record containing information about where parts of the file are stored, file permissions, ownership, group, etc. The important thing to note about the filename-inode number pairs is that inode numbers need not be unique. Multiple filenames can (and often do) refer to the same inode number. This is significant from a security perspective, because the rm

command only removes the directory entry for a file, not the file itself. Thus, to remove a file, one must remove all of the links to that file.

AVAILABILITY AND INTEGRITY One aspect of availability is whether a system restarts securely after fail- ure. Traditional UNIX systems boot in single-user mode, usually as root. And, unfortunately, single-user mode allows literally anyone sitting at the system console to execute privileged commands. Thus, single-user mode represents a security vulnerability in traditional UNIX. Depending on the flavor of UNIX, the security administrator has one or two options for closing this hole. First, if the operating system supports it, the security practitioner should configure the system to require a password before booting in single-user mode. Second, tight physical controls should be implemented to prevent physical access to the system console. System restarts are also relevant to system integrity. After an improper shutdown or system crash, the UNIX fsck command will check filesystems for inconsistencies and repair them (either automatically or with administrator interaction). Using the fsck command, an administrator can detect unreferenced inodes, used disk blocks listed as free blocks, etc. Although there are many ways to supplement UNIX filesystem integrity, one method has become so popular that it deserves to be mentioned here. Developed by Gene Kim and Gene Spafford of Purdue University, Tripwire is an add-on utility that provides additional filesystem integrity by creating a signature or message digest for each file to be monitored. Tripwire allows administrators to specify what files or directories to monitor, which attributes of an object to monitor, and which message digest algorithm (e.g., MD5, SHA, etc.) to use in generating signatures. When executed, Tripwire reports on changed, added, or deleted files. Thus, not only can Tripwire detect Trojan horses, but it can also detect changes that violate organizational policy.

AUDIT Different flavors of UNIX use different directories to hold their log files (e.g., /usr/adm, /var/adm, or /var/log). But wherever the directory is lo- cated, traditional UNIX records security-relevant events in the following log files:

• lastlog: records the last time a user logged in • utmp: records accounting information used by the who command • wtmp: records every time a user logs in or out; this information can be retrieved using the last command • acct: records all executed commands; this information can be obtained using the lastcomm command.

Unfortunately, there is no way to select events or users to record; thus, this log can consume an enormous amount of disk space if implemented. Furthermore, most versions of UNIX support the following logﬁles:

• sulog: logs all su attempts, and indicates whether they were successful • messages: records a copy of all the messages sent to the console and other syslog messages

Additionally, most versions of UNIX provide a generic logging utility called syslog. Originally designed for the sendmail program, syslog ac- cepts messages from literally any program. (This also creates an interest- ing audit vulnerability: any user can create false log entries.) Messages consist of the program name, facility, priority, and the log message itself; the system prepends each message with the system date, time, and name. For example:

Nov 7 04:02:00 alvin syslogd: restart Nov 7 04:10:15 alvin login: ROOT LOGIN REFUSED on ttya Nov 7 04:10:21 alvin login: ROOT LOGIN on console

The syslog facility is highly configurable; administrators specify in /etc/syslog.conf what to log and how to log it. syslog recognizes multiple security states or priorities, including emerg (emergency), alert (immedi- ate action required), crit (critical condition), err (ordinary error), warning, notice, info, and debug. Furthermore, syslog allows messages to be stored in (or sent to) multiple locations, including files, devices (e.g., console, printer, etc.), and even other machines. These last two options make it much more difficult for intruders to hide their tracks. (Of course, if intruders have superuser privileges, they can change the logging con- figuration or even stop logging altogether.)

SECURITY FACILITIES FOR USERS Traditional UNIX supports one privileged administrative role (the “root” account). The root account can create, modify, suspend, and delete user accounts; configure auditing options; administer group memberships; add or remove filesystems; execute any program on the system; shut the system down; etc. In short, root accounts have all possible privileges. This violates both the principle of separation of duties (by not having a separate role for operators, security administrators, etc.) and the principle of complete mediation (by exempting root from access control). Non-privileged users can change their passwords using the passwd command, and they can modify the permissions of their files and directories using the chmod program.

MISCELLANEOUS TOPICS Finally, there are a few miscellaneous topics that pertain to UNIX security but do not neatly fall into one of the categories of operating system security listed at the beginning of this article. These miscellaneous topics include tcpwrapper and fundamental operating system holes.

Vulnerabilities in Traditional UNIX Many (but by no means all) of UNIX’s security vulnerabilities result from ﬂaws in its original design. Consider the following examples:

1. Insecure defaults. Traditional UNIX was designed for developers; it is shipped with insecure defaults. Out-of-the-box UNIX configurations include enabled default accounts with known default passwords. Traditional UNIX also ships with several services open by default, password shadowing not enabled, etc. Administrators should imme- diately disable unnecessary accounts and ports. If a default account is necessary, the administrator should change the password. 2. Superuser and SUID attacks. Given that UNIX does not have different privileged roles, anyone who compromises the root account has compromised the entire system. When combined with SUID programs, the combination can be disastrous. An attacker need simply “trick” the SUID program into executing an attack, either by modify- ing the SUID program or by supplying bogus inputs. If the SUID program runs as root, then the attack is likewise executed as root. Given this vulnerability, SUID programs should be prohibited if at all feasi- ble; if not, the system administrator must continually monitor SUID programs to ensure they have not been tampered with. 3. PATH and Trojan horse attacks. When a user requests a file, the PATH environment variable specifies the directories that will be searched and the order in which they will be searched. By position- ing a Trojan horse version of a command in a directory listed in the search path, such that the Trojan horse directory appears prior to the real program’s directory, an attacker could get a user to execute the Trojan horse. Therefore, to avoid this vulnerability in the PATH variable, administrators can specify absolute filepaths and place the user’s home directory last. 4. Trust relationships. UNIX allows both administrators and users to specify trusted hosts. Administrators can specify trusted hosts in the /etc/hosts.equiv file and users in a file named .rhosts in their home directory. When a trust relationship exists, a user on a trusted (remote) machine can log into the local machine without entering a password. Furthermore, when the trust relationship is defined by an administrator in the /etc/hosts.equiv file, the remote user can log into the local machine as any user on the local system, again without entering a

Auerbach Publications © 2000 CRC Press LLC password. Clearly, this is extremely risky. Even if one trusts the users on the remote machine, there are still two signiﬁcant risks. First, the trust relationships are transitive. If one trusts person A, then one im- plicitly trusts everyone who person A trusts. Second, if the remote machine is compromised, the local machine is at risk. For these rea- sons, trust relationships are extremely risky and should almost al- ways be avoided.

TCP Wrapper Written by Wietse Venema, tcpwrapper allows one to filter, monitor, and log incoming requests for various Internet services (systat, finger, ftp, tel- net, rlogin, rsh, exec, tftp, talk, etc.). The utility is highly transparent; it does not require any changes to existing software. The chief advantage of tcpwrapper is that it provides a decent access control mechanism for network services. For example, an administrator might want to allow incoming FTP connections, but only from a specific network. tcpwrapper provides a convenient, consistent method for implementing this type of access control. Depending on the implementation of UNIX, tcpwrapper might also provide superior audit trails for the services it supports.

Login or Warning Banner UNIX can be configured to display a “message of the day,” specified in the file /etc/motd, to all users upon login. At least part of this message should be a so-called login or warning banner, advising would-be attackers that access to system resources constitutes consent to monitoring and that unauthorized use could lead to criminal prosecution (see Exhibit 3).

CONCLUSION Traditional UNIX implements some of the components of operating systems security to varying extents. It has many well-known vulnerabilities; out-of-the-box conﬁgurations should not be trusted. Furthermore, add- on security tools can supplement core UNIX services. With proper con- ﬁguration, a UNIX system can be reasonably protected from would-be intruders or attackers.

EXHIBIT 3 — Sample Warning Banner

WARNING: THIS SYSTEM FOR AUTHORIZED USE ONLY. USE OF THIS SYSTEM CONSTITUTES CONSENT TO MONITORING; UNAUTHORIZED USE COULD RESULT IN CRIMINAL PROSECUTION. IF YOU DO NOT AGREE TO THESE CONDITIONS, DO NOT LOG IN!

Jeffrey J. Lowder is chief of the network security element at the U.S. Air Force Academy, Colorado Springs, Colo- rado. He is a contributing author of Information Security Management Handbook, 4th ed., Vol. 2, chap. 21, Tipton, H.F. and Krause, M., Eds., Auerbach Publications, Boca Raton, FL, 2001, from which this article is extracted.