UNIX: Introduction Unix Family Tree (Simplified) • Unix first developed in 1969 at Bell Labs (Thompson & Ritchie) 1969 First Edition 1973 Fifth Edition • Originally written in PDP-7 asm, but then (1973) 1974 rewritten in the `new' high-level language C 1975 1976 Sixth Edition ) easy to port, alter, read, etc. 1977 1978 Seventh Edition th 32V • 6 edition (\V6") was widely available (1976). 1979 3BSD 1980 4.0BSD { source avail ) people could write new tools. 1981 4.1BSD { nice features of other OSes rolled in promptly. 1982 System III 1983 System V Eighth Edition 4.2BSD SunOS • By 1978, V7 available (for both the 16-bit PDP-11 1984 SVR2 and the new 32-bit VAX-11). 1985 1986 Mach 4.3BSD SunOS 3 • Since then, two main families: 1987 SVR3 Ninth Edition 1988 4.3BSD/Tahoe { AT&T: \System V", currently SVR4. 1989 SVR4 Tenth Edition 4.3BSD/Reno { Berkeley: \BSD", currently 4.3BSD/4.4BSD. 1990 OSF/1 SunOS 4 1991 • Standardisation efforts (e.g. POSIX, X/OPEN) to 1992 Solaris homogenise. 1993 4.4BSD Solaris 2 • Best known \UNIX" today is probably linux, but also get FreeBSD, NetBSD, and (commercially) Solaris, OSF/1, IRIX, and Tru64. OS Fdns Part 4: Case Studies | UNIX Introduction 149 OS Fdns Part 4: Case Studies | UNIX Introduction 150 Design Features Structural Overview Ritchie and Thompson writing in CACM, July 74, identified the following (new) features of UNIX: Application Application Application (Process) (Process) (Process) 1. A hierarchical file system incorporating demountable volumes. 2. Compatible file, device and inter-process I/O. User System Call Interface 3. The ability to initiate asynchronous processes. Kernel Memory Process File System 4. System command language selectable on a Management Management per-user basis. Block I/O Char I/O 5. Over 100 subsystems including a dozen languages. 6. A high degree of portability. Device Driver Device Driver Device Driver Device Driver Hardware Features which were not included: • real time • Clear separation between user and kernel portions. • multiprocessor support • Processes are unit of scheduling and protection. Fixing the above is pretty hard. • All I/O looks like operations on files. OS Fdns Part 4: Case Studies | UNIX Overview 151 OS Fdns Part 4: Case Studies | UNIX Overview 152 File Abstraction Directory Hierarchy • A file is an unstructured sequence of bytes. / • Represented in user-space by a file descriptor (fd) • Operations on files are: bin/ dev/ etc/ home/ usr/ { fd = open (pathname, mode) hda hdb tty { fd = creat(pathname, mode)) steve/ jean/ { bytes = read(fd, buffer, nbytes) { count = write(fd, buffer, nbytes) unix.ps index.html { reply = seek(fd, offset, whence) { reply = close(fd) • Directories map names to files (and directories). • Devices represented by special files: • Have distinguished root directory called '/' { support above operations, although perhaps • Fully qualified pathnames ) perform traversal with bizarre semantics. from root. { also have ioctl's: allow access to • Every directory has '.' and '..' entries: refer to self device-specific functionality. and parent respectively. • Hierarchical structure supported by directory files. • Shortcut: current working directory (cwd). • In addition shell provides access to home directory as ~username (e.g. ~steve/) OS Fdns Part 4: Case Studies | UNIX Files and the Filesystem 153 OS Fdns Part 4: Case Studies | UNIX Files and the Filesystem 154 Aside: Password File File System Implementation • /etc/passwd holds list of password entries. data • Each entry roughly of the form: type mode userid groupid data user-name:encrypted-passwd:home-directory:shell size nblocks nlinks flags data • Use one-way function to encrypt passwords. timestamps (x3) data { i.e. a function which is easy to compute in one direct direction, but has a hard to compute inverse. direct blocks (x12) blocks (512) • To login: data 1. Get user name single indirect double indirect to block with 512 2. Get password single indirect entries triple indirect data 3. Encrypt password to block with 512 4. Check against version in /etc/password double indirect entries 5. If ok, instantiate login shell. • Inside kernel, a file is represented by a data • Publicly readable since lots of useful info there. structure called an index-node or i-node. • Problem: off-line attack. • Holds file meta-data: • Solution: shadow passwords (/etc/shadow) 1. Owner, permissions, reference count, etc. 2. Location on disk of actual data (file contents). • Where is the filename kept? OS Fdns Part 4: Case Studies | UNIX Files and the Filesystem 155 OS Fdns Part 4: Case Studies | UNIX Files and the Filesystem 156 Directories and Links On-Disk Structures Filename I-Node Hard Disk . 13 .. 2 Partition 1 Partition 2 hello.txt 107 / unix.ps 78 Filename I-Node . 56 .. 214 unix.ps 78 home/ bin/ doc/ Inode Data Inode Data index.html 385 Table Blocks Table Blocks misc 47 Boot-Block Super-Block Super-Block 0 1 2 i i+1 j j+1 j+2 l l+1 m steve/ jean/ hello.txt • A disk is made up of a boot block followed by one misc/ index.html unix.ps or more partitions. • (a partition is just a contiguous range of N • Directory is a file which maps filenames to i-nodes. fixed-size blocks of size k for some N and k). • An instance of a file in a directory is a (hard) link. • A Unix file-system resides within a partition. • (this is why have reference count in i-node). • Superblock contains info such as: • Directories can have at most 1 (real) link. Why? { number of blocks in file-system { number of free blocks in file-system • Also get soft- or symbolic-links: a `normal' file { start of the free-block list which contains a filename. { start of the free-inode list. { various bookkeeping information. OS Fdns Part 4: Case Studies | UNIX Files and the Filesystem 157 OS Fdns Part 4: Case Studies | UNIX Files and the Filesystem 158 Mounting File-Systems In-Memory Tables Process A process-specific Root File-System file tables / Mount 0 11 Point 1 3 2 25 Process B 3 17 4 1 0 2 bin/ dev/ etc/ usr/ home/ File-System 1 27 on /dev/hda2 2 62 3 5 4 19 N 6 hda1 hda2 hdb1 / steve/ jean/ N 32 User Space Kernel Space 0 47 acitve inode table 1 135 17 • Entire file-systems can be mounted on an existing system-wide 19 78 directory in an already mounted filesystem. open file table Inode 78 (includes any process 78 -specific info e.g. file pointer) • At very start, only `/' exists ) need to mount a root file-system. • Subsequently can mount other file-systems, e.g. • Recall process sees files as file descriptors mount("/dev/hda2", "/home", options) • In implementation these are just indices into • Provides a unified name-space: e.g. access process-specific open file table /home/steve/ directly. • Entries point to system-wide open file table. Why? • Cannot have hard links across mount points: why? • These in turn point to (in memory) inode table. • What about soft links? OS Fdns Part 4: Case Studies | UNIX Files and the Filesystem 159 OS Fdns Part 4: Case Studies | UNIX Files and the Filesystem 160 Access Control Consistency Issues Owner Group World Owner Group World • To delete a file, use the unlink system call. R W E R W E R W E R W E R W E R W E • From the shell, this is rm <filename> = 0640 = 0755 • Procedure is: 1. check if user has sufficient permissions on the • Access control information held in each inode. file (must have write access). • Three bits for each of owner, group and world: 2. check if user has sufficient permissions on the read, write and execute. directory (must have write access). 3. if ok, remove entry from directory. • What do these mean for directories? 4. Decrement reference count on inode. • In addition have setuid and setgid bits: 5. if now zero: { normally processes inherit permissions of (a) free data blocks. invoking user. (b) free inode. { setuid/setgid allow user to \become" someone • If crash: must check entire file-system: else when running a given program. { e.g. prof owns both executable test (0711 and { check if any block unreferenced. setuid), and score file (0600) { check if any block double referenced. ) any user can run it. ) it can update score file. ) but users can't cheat. OS Fdns Part 4: Case Studies | UNIX Files and the Filesystem 161 OS Fdns Part 4: Case Studies | UNIX Files and the Filesystem 162 Unix File-System: Summary Unix Processes • Files are unstructured byte streams. Kernel Address Space • Everything is a file: `normal' files, directories, Unix (shared by all) symbolic links, special files. Kernel • Hierarchy built from root (`/'). Stack Segment Address Space • Unified name-space (multiple file-systems may be grows downward as per Process mounted on any leaf directory). functions are called Free • Low-level implementation based around inodes. Space • grows upwards as more Disk contains list of inodes (along with, of course, memory allocated actual data blocks). Data Segment • Processes see file descriptors: small integers which map to system file table. Text Segment • Permissions for owner, group and everyone else. • Setuid/setgid allow for more flexible control. • Recall: a process is a program in execution. • Care needed to ensure consistency. • Have three segments: text, data and stack. • Unix processes are heavyweight. OS Fdns Part 4: Case Studies | UNIX Files and the Filesystem 163 OS Fdns Part 4: Case Studies | UNIX Processes 164 Unix Process Dynamics Start of Day • Kernel (/vmunix) loaded from disk (how?) and parent process parent process (potentially) continues execution starts. fork wait • Root file-system mounted. child zombie • process process Process 1 (/etc/init) hand-crafted.