AN INTRODUCTION TO UNIX SYSTEMS ADMINISTRATION David Jones Front Cover
Taken from an original slide by Kenny Paul ([email protected]) and used with his kind permission.
Original text: LISA VII New Systems Administrators BOF November 1st, 1993, Monterey California Fig. 1: A Typical Systems Administrator.
David Jones (11/30/94) Page 2. Table of Contents
INTRODUCTION...... 9 Exercises and Review Questions...... 9 Format and Fonts ...... 9 Reading the Guide...... 10 Chapter Descriptions ...... 11 Acknowledgements...... 14 Trademarks ...... 14 Feedback ...... 14
FOUNDATIONS...... 15 Objectives...... 15 Introduction...... 15 What Systems Administrators do...... 16 What Systems Administrators need to know ...... 18 Why UNIX? ...... 18 UNIX Past, Present and Future...... 19 Linux ...... 19 Some Necessary Knowledge ...... 20 Pre-emptive versus Non-preemptive...... 21 The vi Editor ...... 21 Conclusions...... 26 Review Questions ...... 27
BASIC UNIX...... 28 Objectives...... 28 Philosophy and Format of UNIX Commands ...... 28 A Command for Everything...... 29 The Basic Commands...... 29 File Permissions ...... 30 Symbolic and Absolute Modes...... 33 Changing File Permissions ...... 34 When is a Command not a Command?...... 37 The Shell...... 38 Executing a Command ...... 39 Other Special Characters ...... 42 Quotes...... 42 Getting Help...... 44 Conclusion ...... 46 Review Questions ...... 46
David Jones (11/30/94) Page 3 Table of Contents
SHELL PROGRAMMING...... 47 Objectives...... 47 Introduction...... 47 The Basics of Shell Programming...... 47 Shell Support for Programming...... 49 Shell Comments ...... 50 Using Shell Variables...... 50 Predefined Shell Variables...... 52 The exit Command...... 55 The Conditional Commands...... 56 The test Command ...... 58 Repeated Action Commands...... 61 Some Other Useful Commands...... 63 Conclusion ...... 64 Review Questions ...... 65
ADVANCED UNIX USE ...... 66 Objectives...... 66 Introduction...... 66 Shell Functions...... 67 Input/Output for Shell Programs...... 68 Trapping Signals...... 71 Regular Expressions ...... 75 ex, vi and Regular Expressions...... 78 awk ...... 81 Perl ...... 82 Conclusions...... 83 Review Questions ...... 84
SYSTEMS ADMINISTRATION THEORY ...... 85 Objectives...... 85 Introduction...... 85 Policy and Penalty ...... 86 Records ...... 87 User Liaison ...... 89 Observation ...... 92 Conclusion ...... 93 Review Question...... 94
USERS AND ACCOUNTS ...... 95 Objectives...... 95 Introduction...... 95 A UNIX Account...... 96 Special Accounts ...... 101 Account Configuration Files...... 102 Creating a New Account...... 105 Deleting an Account...... 108 Automating the Procedure ...... 109 Allocating Super User Privilege...... 111 Conclusions...... 112 Review Questions ...... 112
David Jones (11/30/94) Page 4 Table of Contents
FILE SYSTEMS AND DISK DRIVES ...... 113 Objectives...... 113 Introduction...... 113 The File Hierarchy...... 114 The Components of the UNIX File I/O System ...... 117 File System Data Structures...... 123 I-Nodes and Links ...... 127 Device and Partition Names...... 129 Operations on File Systems...... 130 Creating a New File System...... 131 Reasons to Partition a Drive ...... 132 Mounting and Unmounting a File System...... 133 Checking a File System ...... 136 Conclusions...... 137 Review Questions ...... 138
BACKUPS...... 139 Objectives...... 139 Introduction...... 139 The Components of Backups ...... 140 Scheduler ...... 140 Transport...... 141 Characteristics of a Good Backup Strategy...... 142 Considerations for Developing a Backup Strategy...... 144 Commands ...... 144 dump and restore...... 145 The tar Command ...... 147 The dd Command...... 149 The mt Command...... 150 Compression Programs...... 151 Conclusions...... 153 Review Questions ...... 153
STARTING UP AND SHUTTING DOWN...... 154 Objectives...... 154 Introduction...... 154 The UNIX Boot Process...... 155 Single User, Multi-User and Different Run-Levels...... 157 System V Run Levels...... 158 The Startup Scripts...... 160 Why Won't My System Boot? ...... 161 Solutions ...... 162 Daemons ...... 163 Shutting the System Down...... 164 Reasons for Shutting the System Down...... 164 Being Nice to the Users ...... 165 Commands for Shutting Down and Rebooting ...... 165 Conclusions...... 168 Review Questions ...... 169
David Jones (11/30/94) Page 5 Table of Contents
TERMINALS ...... 170 Objectives...... 170 Introduction...... 170 Hardware...... 170 Software ...... 174 The Login Process ...... 175 Line Configuration ...... 177 Terminal Characteristics ...... 179 Conclusions...... 182 Review Questions ...... 183
PRINTERS ...... 184 Objectives...... 184 Introduction...... 184 Hardware...... 184 Software ...... 186 The SysV Print Software ...... 186 The BSD Print Software ...... 188 /etc/printcap ...... 190 The BSD Print Commands ...... 192 Conclusions...... 194 Review Questions ...... 194
UNIX NETWORK BASICS...... 195 Objectives...... 195 Introduction...... 195 What is a Network? ...... 195 TCP/IP and the Internet...... 197 The Protocols of TCP/IP...... 198 Names and Addresses...... 200 Name Resolution ...... 203 Routing ...... 206 Network Services...... 209 Some Useful Commands...... 215 Conclusions...... 217 Review Questions ...... 217
MORE NETWORKING...... 218 Objectives...... 218 Introduction...... 218 The Berkeley r Commands ...... 219 Network File System...... 222 NFS Daemons ...... 223 Exporting NFS File systems...... 224 Importing NFS File Systems...... 226 Other Useful Commands ...... 227 Adding a Host ...... 229 Conclusion ...... 232 Revision Questions ...... 233
David Jones (11/30/94) Page 6 Table of Contents
SECURITY ...... 234 Objectives...... 234 Introduction...... 234 Physical Threats ...... 235 Logical Threats ...... 236 Passwords...... 238 Search Paths...... 240 File Permissions ...... 241 Log Files and User Activity ...... 245 Network Security...... 247 Public Domain Security Tools...... 249 Conclusions...... 249 Review Questions ...... 250
THE KERNEL ...... 251 Objectives...... 251 Introduction...... 251 Kernel Responsibilities of the Systems Administrator ...... 253 Valid Kernels ...... 253 Configuring the Kernel...... 254 Modifying the Kernel ...... 257 Compiling the Kernel...... 259 Conclusions...... 260 Review Questions ...... 261
CRON, ACCOUNTING, AND QUOTAS...... 262 Objectives...... 262 Introduction...... 262 cron...... 262 Quotas and Process Resource Limits...... 267 Process Resource Limits ...... 267 The BSD Disk Quota System...... 268 BSD Accounting...... 271 syslog ...... 274 Conclusions...... 275 Review Questions ...... 276
PROFESSIONAL ISSUES ...... 277 Objectives...... 277 Introduction...... 277 Professional Organisations ...... 277 Useful Books and Magazines...... 279 Information Sources on the Network...... 284 Conclusion...... 285
David Jones (11/30/94) Page 7 Table of Contents
SOLUTIONS TO EXERCISES...... 286 Chapter 2 ...... 286 Chapter 3 ...... 289 Chapter 4 ...... 291 Chapter 5 ...... 294 Chapter 6 ...... 295 Chapter 7 ...... 298 Chapter 8 ...... 300 Chapter 9 ...... 301 Chapter 10...... 302 Chapter 11...... 303 Chapter 12...... 303 Chapter 13...... 305 Chapter 14...... 307 Chapter 15...... 309 Chapter 16...... 311
SOLUTIONS TO REVIEW QUESTIONS ...... 312 Chapter 1 ...... 312 Chapter 2 ...... 313 Chapter 3 ...... 313 Chapter 4 ...... 315 Chapter 5 ...... 318 Chapter 6 ...... 318 Chapter 7 ...... 318 Chapter 8 ...... 319 Chapter 9 ...... 320 Chapter 10...... 321 Chapter 11...... 322 Chapter 12...... 322 Chapter 13...... 324 Gandalf ...... 324 Chapter 14...... 325 Chapter 15...... 326 Chapter 16...... 326
David Jones (11/30/94) Page 8 Introduction
INTRODUCTION
That's a UNIX book. Cool! -- Garth Elgar (Wayne's World II)
Greetings and welcome to the wonderful world of Systems Administration. After working through this guide you will hopefully have some appreciation of the complexities, hassles, intricacies, pitfalls, purpose and fun of being a Systems Administrator.
Systems Administration is not a science! There is no law handed down to Moses by a burning bush that states "You shalt.....". Each situation has to be dealt with on its own merits and according to the environment at the time. This guide won't tell you the way to fix a problem. Instead (hopefully) it will help you develop the ability and knowledge required to decide between the plethora of choices and follow the option that is best for your site at any particular time.
Exercises and Review Questions
Throughout each chapter of this guide there are a number of Exercises that have been designed to test your understanding of the concepts introduced. It is suggested that you try these exercises when you meet them. Each chapter finishes with a set of Review Questions. These questions form the basis of the tutorial work you should complete with each chapter.
Solutions to both the exercises and review questions are included at the end of the guide.
Format and Fonts
The guide follows a number of simple formatting styles that should assist you in reading it.
Readings
Any text of the following format
Reading
Where the reading is taken from
Purpose
Explanation of why the reading is useful.
David Jones (11/30/94) Page 9 Introduction
refers to a reading from the Resource Materials book provided with this guide. When you come across text of this format you should complete the associated reading before progressing.
Exercises
Text in the following format
Exercises
1.1 text of a question
are exercise questions included to test your understanding of a concept. It is important that you complete the exercise when you first come across this text.
Computer Input and Output
Text in the following format
bash> ls Mail core masters.tar.z rcos.uu telnet.doc Masters dead.letter mbox research
is used to signify text produced or entered into a computer. Text in bold has been entered by a user. Text in the normal font is output by the computer.
Any text that is in italics is meant as a symbol that should be replaced by a real value.
For example: ls filename filename is a symbol that should be replaced with the name of a file.
Reading the Guide
The primary use of this guide is for the subject 85321, Systems Administration offered by the Department of Maths & Computing of the Central Queensland University. Table 0.1 summarises how the guide should be read for that subject.
David Jones (11/30/94) Page 10 Introduction
Week Chapters Topic 11 and 2Introduction to Systems Administration and some basic UNIX 23UNIX Shell Programming 34Advanced UNIX Use 45 and 6Systems Administration Theory and Users 57 and 8File Systems and Backups 69Starting Up and Shutting Down 710 and 11Terminals and Printers 8 12 Unix Network Basics 9 13 More Networking 10 14 Security 11 15 The Kernel 12 16 cron, Accounting and Quotas 13 17 Professional Issues
Table 0.1. Timetable for reading the guide.
Resource Materials
The extra reading material referred to is from the Resource Materials bookthat is provided to students of the subject 85321.
Chapter Descriptions
The following sections briefly describe the individual chapters of this guide.
Chapter 1 Foundations
Chapter 1 provides an introduction to the role and responsibilities of a Systems Administrator. It also introduces a number of basic concepts regarding operating systems, UNIX and a brief introduction to the vi editor.
Chapter 2 Basic UNIX
The aim of this chapter is to provide an introduction to how to use the UNIX operating environment. It examines UNIX command format, the role of the shell, file and directory manipulation, various UNIX commands and the on-line help system.
This chapter can be skipped if you are already an experienced user of the UNIX operating system.
David Jones (11/30/94) Page 11 Introduction
Chapter 3 Shell Programming
Much of what a Systems Administrator does includes writing and being able to understand shell programs. This chapter examines all of the basic shell programming knowledge necessary for a Systems Administrator. The chapter only discusses the syntax of the Bourne shell.
If you are already an experienced writer of Bourne shell programs you may skip this chapter.
Chapter 4 Advanced Unix Use
Chapter 4 rounds off the UNIX user information. It examines more advanced shell programming, advanced commands including awk and sed and regular expressions.
Chapter 5 Systems Administration Theory
This chapter examines some of the generic, hardware independent tasks that a Systems Administrator must complete including management, policies, procedures, communication with the users, and most importantly, documentation.
Chapter 6 Users and Accounts
Computers were created so people could use them to do work. Chapter 6 examines the procedures and files involved in letting people use a UNIX machine. It examines how to add and delete users and where and what information is stored about users.
Chapter 7 File Systems
All operating systems store information for future use onto disk drives. This chapter examines how the UNIX operating system stores and retrieves information from disk drives.
Chapter 8 Backups
Chapter 8 examines what might be the most important task that a Systems Administrator is responsible for and one that is usually neglected, the backing up of important data.
Chapter 9 Starting Up and Shutting Down
The UNIX operating system is a complex beast. This chapter examines the process the machine goes through when it boots and when it is shut down. It also examines the responsibilites of the Systems Administrator with regards starting a system and shutting it down.
David Jones (11/30/94) Page 12 Introduction
Chapter 10 Terminals
A computer must not only store information but it must also provide methods by which users can access the information. Chapter 10 examines how terminals are added to a UNIX machine.
Chapter 11 Printers
In addition to showing how to add a printer to a UNIX machine. Chapter 11 also examines the software systems UNIX uses to implement printer support.
Chapter 12 Unix Network Basics
Networking is an increasingly important part of computing. Chapter 12 introduces you to the basics of networking within the UNIX environment.
Chapter 13 More Networking
Chapter 13 extends the coverage of networking to including additional features like NFS. It also provides some insight into adding hosts to a network and other more advanced networking topics.
Chapter 14 Security
Chapter 14 examines some of the security problems and issues related to the Unix environment and provides solutions and tips on how to solve them.
Chapter 15 The Kernel
The kernel is the central work horse of the operating system. Chapter 15 discusses the kernel, its responsibilities, and how the Systems Administrator can control it.
Chapter 16 cron, Accounting and Quotas
Chapter 16 brings together and discusses the • cron system, The method by which the Systems Administrator can schedule tasks to be carried out automatically at set times. • accounting, The methods by which the UNIX operating system allows you to track what is being done to your system and by whom. • quotas. Methods by which limitations can be placed on what users can do and the resources they can consume.
David Jones (11/30/94) Page 13 Introduction
Chapter 17 Professional Issues
Chapter 17 discusses, to some part, life as a professional Systems Administrator. Topics covered include • sources of information including books and magazines, and • professional organisations for Systems Administrators.
Solutions
The solutions for both the review questions and the chapter exercises are included at the end of the guide.
Acknowledgements
This guide has developed over a couple of years and has been improved by the efforts and feedback from a number of people. This particular version of the guide has benefited from the efforts of Peter Hardy, Shaun Teeney and Peter Van Heck, three CQU distance students who trialed the guide (and the subject) during late 1994.
Extra special thanks must go to Mary Cranston and Elizabeth Tansley who provided invaluable assistance, both with proof-reading and as a source of many useful ideas.
Trademarks
UNIX and various other trademarks are all owned by their various owners. While every attempt has been made to guarantee its correctness there is no warranty about the information in this document.
Feedback
This guide is still undergoing development and there are certain to be typos, inconsistencies and some information that is just plain wrong. If you find anything you think resembles any of these, please tell me about them.
I can be contacted at the following address David Jones Department Maths & Computing Central Queensland University Phone: 079 309856 Fax: 079 309729 E-mail:[email protected]
David Jones (11/30/94) Page 14 Foundations Chapter 1
Chapter 1
FOUNDATIONS
A beginning is the time for taking the most delicate care that the balances are correct. -- Frank Herbet (Dune)
Objectives
This chapter provides the gentle introduction to the profession (some say art) of Systems Administration. This chapter has been written to • show what Systems Administrators do, • introduce some of the concepts necessary to become a Systems Administrator, • provide background on the UNIX operating system and in particular the Linux operating system, • give an overview of the current state of play in the operating system market place, and • provide an introduction to the vi editor.
Introduction
Before starting, write down what you think the responsibilities of a Systems Administrator are. At the end of this guide come back and see if you agree with what you have written.
Systems Administration is one of the most complex, fulfilling and misunderstood professions within the computing arena. Everybody who uses the computer depends on the Systems Administrator doing their job correctly and efficiently. However the only time users tend to give the Systems Administrator a second thought is when the computer system is not working.
Very few people, including other computing professionals, understand the complexity and the time consuming nature of Systems Administration. Even fewer people realise the satisfaction and challenge that Systems Administration presents to the practitioner. It is one of the rare computing professions in which the individual can combine every facet of the computing field into one career.
David Jones (11/30/94) Page 15 Foundations Chapter 1
At some time during their career a Systems Administrator will make use of knowledge from the following (far from exhaustive) list of fields, both computing and non-computing. • programming, • hardware maintenance and installation, • documentation and testing, • Human Computer Interface, • networks and computer communication, • user education, • diplomacy, • legal issues and contracts, • detective work, • management and policy setting, and • public relations.
What Systems Administrators do
Systems Administration is an old responsibility gaining new found importance and acceptance as a profession. It has come into existence because of the increasing complexity of modern computer systems and networks and because of the economy's increasing reliance on computers. Any decent size computer site now requires at least one person to keep the computers running happily. If the computers don't work the business suffers.
It can be said that Systems Administrators have two basic reasons for being • ensuring that the computing system runs correctly and as efficiently as possible, and • ensuring that all users can and do use the computing system to carry out their required work in the easiest and most efficient manner.
These two reasons often conflict with one another. Management will wish to restrict the amount of money spent on computer systems. The users on the other hand will always want more disk space and faster CPUs. The System Administrator must attempt to balance these two conflicting aims.
The real work required to fulfill these aims depends on the characteristics of the particular computing system and the company it belongs to. Factors that affect what a Systems Administrator needs to do come from a number of categories including the following.
Users
• How many users are there? Two hundred users are more difficult to help than two users. • The level of the user's expertise. This is a combination of the user's actual expertise and their perceived expertise. A user who thinks they know a lot (but doesn't really) can often be more trouble than a user who knows nothing and admits it.
David Jones (11/30/94) Page 16 Foundations Chapter 1
Users who know what they know.
Picture it. You are a Systems Administrator at a United States Air Force base. The people using your machines include people who fly million dollar weapons of destruction that have the ability to reduce buildings if not towns to dust. They are supremely confident in their ability.
What do you do when an arrogant, abusive Colonel contacts you saying he cannot use his computer? What do you say when you solve the problem by telling him he did not have it plugged in? What do you do when you have to do this more than once?
It has happened.
• What are the users trying to do? If the users are scientists doing research on ground breaking network technology it is going to be harder to maintain their machines than if they are simply doing word processing. • Are they responsible or irresponsible? Do the users follow the rules or do they make their own? Do the users like to play with the machines?
Computers
• How many, how big and how complex? Once again greater numbers imply more work. • Are they networked? The existence of a network connecting the machines together raises additional problems. • Are the computers heterogenous or homogenous? It is much simpler if all the computers on the site are exactly the same. This almost NEVER happens.
Support
• Are you alone? At some sites there is one administrator who does everything from installing peripherals, fixing computers, doing backups etc. At other sites there are other administrators, operators and technicians. • Are you a full time administrator? In some cases the administrator looks after the machines in addition to performing their "real job".
David Jones (11/30/94) Page 17 Foundations Chapter 1
What Systems Administrators need to know
The short and sweet answer is that to be a really good Systems Administrator you need to know everything about the entire computer system including operating system, hardware, software, users, management, network and anything else you can think of that might affect the system in any way.
Failing that lofty aim the System Administrator must have the ability to gain this all encompassing knowledge. The discovery process may include research, trial and error, or begging. The abilities to learn and problem solve may well be the two most important for a Systems Administrator.
Why UNIX?
Some parts of Systems Administration are independent of the type of computer being used, for example handling user complaints and getting on with management. However by necessity there is a great deal of complex platform dependent knowledge that a Systems Administrator must have in order to carry out their job.
Tell me and I’ll forget; show me and I may remember; involve me and I’ll understand.
This text has been written with the UNIX operating system in mind as the main computing platform. In particular this text has been written with the Linux operating system, a version of UNIX that runs on IBM PC clones, in mind. It is necessary to have access to the root password of a UNIX (not necessarily Linux) machine to get the most benefit from this guide.
The reasons for choosing UNIX, and especially Linux, over any of the other available operating systems include • UNIX has a long history both in industry and academia. • Knowing UNIX is more likely to help your job prospects than hinder them. • UNIX is one of the current industry buzz words. • It is hardware independent. • Linux is free and runs on a cheap, popular type of computer.
Just as there are advantages in using UNIX there are also disadvantages. "My Operating System is better than yours" is a religious war that I don't want to discuss here.
David Jones (11/30/94) Page 18 Foundations Chapter 1
UNIX Past, Present and Future
The history of UNIX is an oft told tale and it is sometimes hard to pick the right version. The story has been told many ways and the following is one version.
Reading.
Resource Materials Chapter 1.1
Purpose.
To provide some idea of how UNIX developed in an attempt to explain why it is where it is today.
At the current point in time it appears that UNIX has esconced itself into the following market niches • server operating system, and Machines running UNIX are acting as file servers and network servers for local area networks (LANs) of smaller client machines (running MS-DOS, Windows or Macs). • workstation operating system. Workstations are nominally powerful computers usually used by by a single user. They are generally used by engineers, scientists and other people who require a lot of computing power.
Both these roles are being challenged by the arrival of new operating systems like Windows NT.
Linux
This guide has been specifically written to centre on the Linux operating system. Linux was chosen because it is a free, complete version of the UNIX operating system that will run on cheap, entry level machines. The following reading provides you with some background into the development of Linux.
Reading.
Resource Materials Chapter 1.3
Purpose.
Explain the origins and development process that brought Linux to life so that you understand where it fits into the entire picture.
David Jones (11/30/94) Page 19 Foundations Chapter 1
Some Necessary Knowledge
The rest of this chapter is taken up with imparting to you some of the basic knowledge that is required. The list of topics and jargon is by no means complete but it is a start.
The Operating System
Essentially the operating system is the most important piece of software on a machine. This piece of software acts like the elected government of the computer system it provides • essential support services required by the inhabitants of the computer, • maintains and polices the laws and rules by which the computer system operates, and • manages the use of the systems' resources.
One of the major acts to be carried out during the booting of a computer is the loading and running of the operating system kernel. The kernel is the name given to the executable program that is the operating system. The kernel contains all the code that specifies how the operating system is to perform its duties. The actual format and content of the kernel will depend on the type of operating system. A later chapter of this guide will examine specifics about the kernel of a UNIX operating system.
One of the defining characteristics of an operating system is the number of people and the number of programs that can use the operating system at any one time. Table 1.1 outlines some of the terminology.
Term Meaning single user Only one person can use the operating system. multi user Many people can use the system at the same time. The OS must prevent different users from interfering with each other. single tasking * Only one process (a running program) can be running at any time. multi tasking * Multiple process can appear to be executing at any one time. preemptive multi tasking The operating system decides when a process has had enough time on the CPU. non-preemptive multi The process itself decides when it has had enough tasking time. * sometimes known as single and multi programming or processing. Table 1.1. Characteristics of an Operating System.
Table 1.2 relates these characteristics to some of today's operating systems.
David Jones (11/30/94) Page 20 Foundations Chapter 1
Operating System Characteristics MS-DOS single-tasking, single user (not much of an OS) Windows non-preemptive multi tasking, single user Windows NT preemptive multi tasking, single user * OS/2 preemptive multi tasking, single user UNIX preemptive multi tasking, multi user * Microsoft say its multi-user and it is after a fashion. You can have multiple users but only one of them can use the machine at any one time. Table 1.2. Charactericistics of Today's Operating Systems. Pre-emptive versus Non-preemptive
When an operating system allows multitasking it appears that many processes are all executing at the same time. This is not the case!
The CPU (central processing unit) can only execute one instruction at a time. So only one process is actually being executed. The others are waiting for their turn on the CPU. Multitasking works by the processes sharing the CPU time. One process will use the CPU for a time and then another will use it. It appears that many processes are running because this change occurs very quickly.
Who decides when to change the process on the CPU?
The answer to this question is the basic difference between preemptive and non- preemptive multitasking. In a preemptive system the operating system decides when a process has had enough time on the CPU. In a non-premptive OS each process can decide when it has had enough.
Under a non-preemptive system problems arise if you have a process that is greedy or goes into an endless loop. It will never get off the CPU.
The vi Editor
The mention of vi sends shudders through the spines of some people, while other people will quiver with pleasure. vi is a standard UNIX editor that has the deserved status of being a mongrel to learn. However hand in hand with the difficulty of learning comes a very powerful editor. It is strongly suggested that you use and become familiar with vi.
Why vi?
vi is an anachronistic antique of an editor hated by most people but it is very powerful once you understand its intricacies and more importantly, it is the only screen editor available on every Unix system which is why we will cover it.
David Jones (11/30/94) Page 21 Foundations Chapter 1
How to use vi.
vi is a modal editor. That means that it has different modes and depending on the mode it is in, certain keys will perform different tasks.
Mode Name Purpose Command Mode most keys perform a specific vi command, when vi is started it is in this mode Insert Mode is initiated by many different vi commands, in this mode letters typed are inserted at the current cursor location, hitting the ESC key will return vi to Command Mode Ex Escape Mode allows the use of ex commands for the editing of text. (last line mode) Enter this mode by typing a colon : while in command mode. The colon will appear at the bottom of the screen, when you hit enter you will be returned to Command Mode
Table 1.3. vi Modes.
vi performs all editing on a copy of the file that is held in an editing buffer. Changes are not made to the actual file until they are explicitly saved using the appropriate command.
During one editing session you can be editing multiple files. This is done by specifying all the files you wish to edit when starting vi, and using special editing commands to change between the different files.
vi also supplies a mechanism to enable file recovery. If the system crashes while editing a file with vi the file can be recovered.
vi accepts a number of command-line switches that modify its operation. Table 1.4 lists some of these command line options.
Switch Purpose -r file attempt to recover file -L list all files that can be recovered after a system crash -R invokes vi using read only mode. The file will not be able to be modified +command after invoking vi execute the vi command signified by command command any cursor positioning command
Table 1.4. vi Command-Line Switches.
David Jones (11/30/94) Page 22 Foundations Chapter 1
Exiting vi.
Table 1.5 lists some of the different methods that can be used to exit from vi. All methods shown must be carried out from command mode.
Command Purpose :q quit only if no changes were made :q! exit and don't save any changes :wq save the editing buffer and then exit :x same as ZZ ZZ save the editing buffer only if changes were made and then exit
Table 1.5. Commands to exit vi.
vi's File Manipulation Commands.
vi provides many methods for manipulating files. Table 1.6 lists some of them.
Command Purpose :r file read a file into the buffer after the current line :xr file read a file into the buffer after the line specified by x :w [file] save the edit buffer to a file, if no filename is specified the current filename is assumed :w! [file] :w may not work all the time (e.g. the file already exists, the file is readonly), the ! tells vi to try its best :s,ew [file] save the section of the buffer specified s (start line) and e (end line) including those lines
Table 1.6. vi's File Manipulation Commands.
vi's Text Insertion Commands.
Table 1.7 lists the commands used to take vi from command mode into insert mode.
vi has a number of special features that work in conjunction with most of the commands listed in Tables 1.7-1.10 including • redo the last command, and By hitting the . key vi will repeat the last command. • perform a command a set number of times. Entering a number before these commands will cause vi to repeat the command that many times.
David Jones (11/30/94) Page 23 Foundations Chapter 1
For example: Typing (while in command mode) 75i hello
Command Purpose i insert text before current cursor location a insert text after the current cursor location I insert text at the start of the current line A insert text at the end of the current line o insert text on a new line below the current line O insert text on a new line above the current line
Table 1.7. vi's Text Insertion Commands.
Inserting Control Characters
There will come times when it is wished to manipulate control characters.
For example: A user has just copied a text file from and MS-DOS machine. The differences between what MS-DOS uses to indicate the end of a line and how UNIX performs the same task means that there are additional ^M characters at the end of each line. A command to remove these characters might be sed ’1,$s/^M//g’ msdos.file
This command is designed to replace the control M character (ASCII value 13) with nothing. Simply typing a ^ character followed by an M character will not work (this will replace the two characters ^ and M with nothing).
Some systems allow a control character to be specified by • hitting the CTRL-V key combination, and • then entering the desired control character.
David Jones (11/30/94) Page 24 Foundations Chapter 1
vi's Text Modification Commands
Table 1.8 lists the vi commands used to move, delete and copy text.
Command Purpose dw delete word under the cursor dd delete current line D delete to the end of the current line X delete the character before the cursor x delete the character under the cursor cw delete current word and enter Insert Mode cc delete current line and enter Insert Mode rc replace current character with the character c J join the current and next lines together
Table 1.8. vi's Text Modification Commands.
vi's Screen Control Commands
Table 1.9 lists the vi commands used to move the screen.
Command Purpose
Table 1.9. vi's Screen Control Commands.
vi's Cursor Movement Commands.
Table 1.10 lists the commands used to move the cursor.
David Jones (11/30/94) Page 25 Foundations Chapter 1
Command Purpose l, spacebar or move one space right right cursor key h, backspace or move one space left left cursor key j, + or down move one line down cursor key k, - or up cursor move one line up key H move to the top left hand corner of the screen M move to the middle of the screen L move to the start of the last line on the screen $ move to end of the current line 0 move to the start of the current line ^ move to the first non-blank character of the current line w move to next word e move the end of the current word b move to previous word ( move to the start of a sentence ) move to the start of the next sentence { move to the start of a paragraph } move to the start of the next paragraph nG move to line number n
Table 1.10. vi's Cursor Movement Commands.
Conclusions
Systems Adminsitration is a complex and interesting field requiring knowledge from most of the computing area. It provides a challenging and interesting career. The UNIX operating system is an important and available competitor in the current operating systems market and forms the practical system for this subject.
David Jones (11/30/94) Page 26 Foundations Chapter 1
Review Questions
1.1. What does a Systems Administrator do?
1.2. What characteristics would make a good Systems Administrator?
1.3. What are the primary aims of an operating system?
1.4. List the current operating systems you consider important in the commerical environment.
1.5. What is a kernel?
David Jones (11/30/94) Page 27 Basic UNIX Chapter 2
Chapter 2
BASIC UNIX
UNIX - A popular operating system that works on many different sizes and types of computers and drives most users positively loony. - The Little Grey Book
Objectives
Before you can become a UNIX Systems Administrator you have to become a competent if not expert UNIX user. This chapter starts you on the metamorphosis into an expert UNIX user.
At the end of this chapter you should • be familiar with the command format used by UNIX • be aware of the role of a shell • have gained an understanding of some of the intricacies of using a shell including special characters, I/O redirection, wildcards and shell variables • become familiar with the UNIX file and directory manipulation commands • received an introduction to the UNIX directory hierarchy, • become familar with some of the UNIX commands that report the status of the machine • have used the UNIX on-line help system at least once
Philosophy and Format of UNIX Commands
A UNIX system comes with hundreds of executable commands and programs (it is quite easy to get to a count of 600 without really looking hard). Typically each of these programs carries out a particular job and will usually have some obscure and obtuse name that means nothing to the uninitiated.
There are no set rules about UNIX commands however there is a UNIX philosophy that is used by many of the commands, but not all. • small is beautiful, UNIX provides the mechanisms to join commands together so commands should do one thing well. • 10 percent of the work solves 90 percent of the problems, UNIX was never designed to solve all problems, it was designed to solve most requirements without too much hassle on the programmer's part. • solve the problem, not the machine, and Commands should ignore any machine specific information and be portable.
David Jones (11/30/94) Page 28 Basic UNIX Chapter 2
• solve at the right level, and you will only have to do it once. The key to UNIX problem solving is only to do it once e.g. pattern matching is only implemented once, in the shell, not in every command.
A Command for Everything
A fairly intelligent and experienced would be computer professional has just started using UNIX seriously. He gets to a stage where he wants to change the name of some files.
Being an MS-DOS junkie from way back what command does he look for? rename of course. It doesn't work! "That's a bit silly!", he thinks, "You would think that UNIX would have a rename command."
It just so happens that this person has just completed a C programming subject in which one of the assignments was to write a rename command. So he spends the next day trying to write and compile this program. After much toil and trouble he succeeds and follows good administration policy and informs all the other users of this brand new wonderful program he has written. He goes into great detail on how to use the command and all the nice features it includes.
They all write back and tell him about the UNIX command mv that does the same thing.
The moral of this story is that if you want to do something under UNIX, then chances are that there is already a command to do it.
The Basic Commands
The following readings provide an introduction and revision to some of the basic UNIX user commands. It is necessary that you as a Systems Administrator know these commands intimately, not only because you will use them but because the Systems Administrator is often expected to also be the expert user of the system, capable of and responsible for answering any question about any program on the system.
Reading.
Resource Materials Chapter 2.1.
Purpose.
Revise and introduce the basic UNIX commands.
David Jones (11/30/94) Page 29 Basic UNIX Chapter 2
Exercises.
2.1 Complete exercise 1 from the end of Resource Materials Chapter 2.1.
2.2 Complete exercise 2 from the end of Resource Materials Chapter 2.1.
File Permissions
As well as being associated with a filename each file has in addition a number of other attributes. Some of these other attributes can be seen by using the -l flag of the ls command. An explanation of the output of the command ls -l is shown below.
-rw-rw-rw- 1 david staff 227 Dec 12 19:33 note 1. 2. 3. 4. 5. 6. 7.
1. File access permissions, who can do what to the file/directory 2. The number of links to this file. 3. The owner's user name. 4. The group owner's group name. 5. The size of the file in bytes. 6. The date and time the file was last modified. 7. The name of the file.
Figure 2.1. Explanation of the output of the ls -l command.
Numbers five, six and seven should by now be fairly self-explanatory. The following sections aim to explain numbers one, three and four.
Users, Groups and Others
As mentioned in the text book UNIX provides a very simple but powerful protection mechanism for files which is based on the concept of individual users and groups of users.
Access to a UNIX file is grouped into three categories • user The individual user who owns the file (by default the user that created the file). In figure 2.1 the owner is david. • group The collection of people that belong to the group that owns the file (by default the group to which the user owner belongs). In figure 2.1 the group owner is staff.
David Jones (11/30/94) Page 30 Basic UNIX Chapter 2
• other Anybody that doesn't fall into the first two categories.
The first field (the permissions field) from Figure 2.1 holds the information that restricts what actions the users from each of the above three categories can perform on the file or directory. The permissions field consists of 10 single letters that are separated into four separate fields. The meanings of those fields is summarised in Figure 2.2.
The very first character is used to indicate the type of file. Some of the possible file types are listed in table 2.1.
tuuugggooo
t = type of file u = permissions for user who is the owner of the file g = permissions for the group that owns the file o = permissions for everyone else
Figure 2.2. Format of File Permissions.
File Type Meaning - a normal file d a directory l symbolic link b block device file c character device file p a fifo or named pipe s a XENIX semaphore m a XENIX shared data section
Table 2.1. Different File Types.
Table 2.1 outlines some of the different file types on a UNIX system. Two types mentioned are block and character special files (sometimes known as block and character device files). These files will be discussed elsewhere.
The three remaining fields for user, group and others all use the same format. The first entry is the read attribute, the second is the write attribute and the third is the execute attribute. If there is a - character in a particular spot then that attribute is turned off (i.e. the user can't do it). If there is a r, w or a x in the appropriate spot then the attribute is turned on and the user is allowed to perform the associated operation.
David Jones (11/30/94) Page 31 Basic UNIX Chapter 2
For example: if a particular file had the permissions -rwx-w---x Then the owner of the file has the attributes rwx which means they can read, write and execute the file. Users in the group have the attributes -w- which means they can only write to the file. All other users have the attributes --x which means they can only execute the file.
File permissions have slightly different meanings when applied to a directory. Table 2.2 summarises the meanings of the three basic attributes for both files and directories.
There are two other possible letters which may appear in the permissions field for a file.
• s - the set user/group id bit (set uid/ set gid bit) When someone runs a program the resulting process runs with the privileges of the user who ran it (both user and group). If the set user/group bit are set the resulting process will run with the privileges of the user/group owner of the file. e.g. The passwd command has the set uid bit set, and is owned by root. This means that whoever runs can write their new password to the passwd file. YOU MUST BE CAREFUL WHEN SETTING THESE BITS. They are a common source of security problems. • t - the sticky bit When you run a program UNIX reads the program code from disk into memory and creates a process. This takes time. If you have a large program which you run frequently it might make more sense to keep it in memory all the time. To do this you set the sticky bit.
Figure 2.3. Special Permission Bits.
David Jones (11/30/94) Page 32 Basic UNIX Chapter 2
Attribute Meaning for a file Meaning for a directory Type r the contents of the file can be ability to obtain a directory listing viewed w the contents of the file can be ability to create and remove files changed from directory x the file can be executed as a ability to change into that directory, command and access its contents
Table 2.2. The Meanings of Permissions.
Exercises.
2.3. Using the following directory hierarchy.
jonesd rwxr-x--x
all owned by jonesd group owner admin
assignr-x------docs rwxr-xrwx
and the following facts • astudent belongs to the group users • astaff belongs to the group admin Answer the following a)Can astudent obtain a directory listing of the jonesd directory? b)Can astaff obtain a directory listing of the docs directory? c) Can astudent obtain a directory listing of the docs directory? d)Can astudent create a file in the docs directory?
Symbolic and Absolute Modes
So far you have only seen symbolic modes in action. That is where the symbols r, w and x are used to represent access permisions. UNIX recognises another method called absolute mode that uses numbers instead.
In absolute mode each access class (user, group and other) are represented by a number no bigger than seven. This number is arrived at by converting the symbolic permissions into a binary digit as shown in figure 2.4
David Jones (11/30/94) Page 33 Basic UNIX Chapter 2
Converting Symbolic to Numeric.
r w x r - - r - x symbolic 1 1 1 1 0 0 1 0 1 7 4 5 754 numeric
Figure 2.4. Converting Symbolic Permissons to Numeric.
Exercises.
2.4. Convert the following symbolic permissions into numeric. a)rw----r-x b)rwxrwxrwx c) ---rwx-wx 2.5. Convert the following numeric permissions into symbolic. a)111 b)550 c) 750
Changing File Permissions
The UNIX system provides a number of commands for users to change the permissions associated with a file. Table 2.3 provides a summary.
Command Purpose chmod change the file permissions for a file umask set the default file permissions for any files to be created. Usually run as the user logs in. chgrp change the group owner of a file. chown change the user owner of a file.
Table 2.3. File Permission Commands.
The chmod Command
The command to change the permissions for a file or a directory is the chmod command. The format of the command is explained on the next page.
For example: chmod u+rwx temp.dat turn on all permissions for the owner of the file chmod gw-rwx temp.dat
David Jones (11/30/94) Page 34 Basic UNIX Chapter 2
turn off all permissions for all the users except the owner of the file chmod -R a-rwx / turn off all permissions for everyone for all files chmod -R a= / turn off all permissions for everyone for all files chmod 770 temp.dat allow the user and group read, write and execute and others no access
chmod Command Format.
chmod [-R] operation files
-R recursively descend each directory
operation can be either symbolic or absolute permissions. When using absolute permissions operation is simply the numeric permissions e.g. 770 200
When using symbolic permissions operation takes the form of whooppermission where:
who u for owner of file g for group o for others a for all categories op + add permission - remove permission = set permission permission r read w write x execute s set uid/gid t set sticky bit
Figure 2.5. chmod Command Format.
The chown and chgrp Commands
There are times when you are required to change the owner of a file or the group owner of a file. One such time is when the root user creates a home directory for a new user. When root creates the directory the owner of the directory will be root, when in fact we really want the owner of the directory to be the new user.
David Jones (11/30/94) Page 35 Basic UNIX Chapter 2
There are some limitations on how you can use chown and chgrp. Only the root user or the current owner of the file can use chown to change the ownership of a file. Only the root user can arbitrarily change groups. The owner of a file can only change the group to another group to which the owner belongs.
On some systems you cannot give away ownership of files at all, only root can. Two reasons with this are • in a filesystem with quotas (quotas place an upper limit of how many files and how much disk space a user can use) a person could avoid the quota system by giving away the ownership to another person • if anyone can give ownership of a file to root they could create a program that is setuid to the owner of the file and then change the owner of the file to root.
chown/chgrp Command Format.
chown [-R] owner files chgrp [-R] group files
-R change the ownership on all sub-directories and the files within them
owner is either a numeric user identifier or a user name listed in /etc/passwd .* group is either a numeric group identifier or a group name listed in /etc/group. files is a list of files of which you wish to change the ownership
Figure 2.6. chown/ chgrp Command Format.
* Some systems allow owner to take the format owner.group this allows you to change the owner and the group owner of a file with one command.
The umask Command
Everytime a file is created it is automatically given some default access permissions. The purpose of the umask command is to set these default access permissions. One of the responsibilities of a Systems Administrator is to ensure that by default files are provided with secure access permissions.
For example imagine what would happen if all the files and directories that were created were given the default access permissions 777. Anyone could read, write or execute those files.
David Jones (11/30/94) Page 36 Basic UNIX Chapter 2
For example: When files are created with the following umask values umask 027 the user will have all permissions, group will not have write and others will have none umask 022 the user will have all permissions, group and others will not have write.
The umask Command.
umask [ ooo ]
With no parameter the current umask value is displayed.
ooo are octal digits (numbers ranging from 0-7). The specified digits are subtracted from the default access permissions. e.g. 027 will subtract write permission for the group and all permissions for others
Figure 2.7. umask Command.
When is a Command not a Command?
Most UNIX operating systems supply a command called which or one called whereis. The purpose of these commands is to search through all the directories in the user’s current search path for a particular command.
For example, the command which ls on my machine aldur returns /usr/bin/ls. This means that the program for ls is in the directory /usr/bin. If which can't find the command it reports no command in path. This implies one of two things • there is no such program, or • your search path does not include the directory in which the program is located.
Try the command which umask. The reason you can't find the program for umask is that umask is recognised by the shell and the shell performs its operation. (The code for umask is part of the shell there is no program for it).
David Jones (11/30/94) Page 37 Basic UNIX Chapter 2
The Shell
When people say that the UNIX operating system is difficult to use they are wrong. Most of them will never have used the UNIX operating system. What most people find difficult to use is the interface UNIX presents them, the shell. A shell is a program that has been written to perform a number of tasks (outlined below) including taking commands from the user.
The shell you will more than likely use under Linux is called bash (Bourne Again Shell). The basic syntax used by bash is identical to the Bourne shell but it also provides additional abilities including command line editing using cursor keys.
Different people have their favourite shells. As a Systems Administrator you will have to know the Bourne shell syntax. This is because almost all of the shell programs that are used to maintain a UNIX operating system are written using the Bourne shell syntax.
All of the commands talked about in the previous readings are actually executable programs stored somewhere in the directory hierarchy. When you ask the shell to cd /home it runs the executable program cd to perform the task.
The shell itself is just an executable program. Table 2.4 lists some of the program names for the various shells. If you enter one of these program names, for example csh, it will execute the program csh and start a version of the C shell. (You exit the shell program by typing logout, exit or using the key combination CTRL-D.)
Shell Program Name Description Bourne shell sh the original shell from AT&T, available on all UNIX machines C shell csh shell developed as part of BSD Korn shell ksh AT&T improvement of the Bourne shell Bourne again bash Shell distributed with Linux, version of shell Bourne shell that includes command line editing and other nice things
Table 2.4. Examples of UNIX Command Shells.
Exercises
2.6. Type the following command set. set displays all the shell variables that are currently set. You should see one called SHELL. This variable is defined to contain what shell you are using. What shell are you using?
David Jones (11/30/94) Page 38 Basic UNIX Chapter 2
Exercises
2.7. Execute one of the other shells on your system? Does it change the variable shell?
Among the shell variables displayed by entering the set command you should see PATH or path. This is the shell variable that holds your current search path, the list of directories UNIX looks in to find executable programs.
Shell Responsibilities
A UNIX shell is responsible for the following • executing programs or commands as requested by the user, • performing variable and file name substitution, It is the shell that performs the translation of let* into the list of filenames that start with let. • performing I/O redirection, Making sure that input/output ends up in the correct place as specified by the user's use of the > >> << and other characters. • environment control, and The shell also controls and stores settings about the type of terminal being used amongst other information. • providing an interpreted programming language. Shells provide the ability to write programs using UNIX commands. The shell provides the looping and conditional commands necessary for a programming language.
Executing a Command
As part of executing a command a shell performs the following tasks. • takes a line of text from the user, • parses the text and performs the necessary I/O redirection, • replaces any shell variables in the text with the actual value, • replaces any wild card characters with the appropriate file names, and then • executes the command.
Read the command line
The shell waits until the user hits the enter key. The shell places what the user types into the following format program_name arguments.
David Jones (11/30/94) Page 39 Basic UNIX Chapter 2
Perform I/O Redirection
Table 2.5 outlines the different type of redirection that the shell must recognise. As part of this step the shell must recognise which arguments are commands.
For example: ls | cat > hello.dat
Character Meaning command < file Take standard input from file. command > file Put the output of command into file. Overwrite file if it already exists. command >> file Put the output of command into file. Append the output onto the end of file if it already exists. command << label Take standard input for command from the following lines until a line that contains only label `command` Execute command and replace `command` with the standard output of the command. command1 | Use the standard output of command1 as the command2 standard input of command2
Table 2.5. Types of I/O Redirection.
Replace Shell Variables
To the shell a $ signifies that what follows is a shell variable. The shell must replace that shell variable with its actual value. (Chapter 3 covers the use of shell variables in more detail.)
For example: echo $SHELL will display on my system /bin/bash
What happens is the shell sees the $ and replaces the shell variable name SHELL with its value.
Replace Wildcard Characters
It is also the shell that replaces the wildcard characters * ? etc with the list of filenames that match them.
David Jones (11/30/94) Page 40 Basic UNIX Chapter 2
Execute the Command
The last step in the process is to actually find the executable program, load it into the computer's memory and run it.
Order is Important
It is important to remember the order in which the shell carries out the above tasks. 1. I/O redirection 2. Shell variables, and then 3. Wildcard characters
For example: pipe=\| creates a shell variable called pipe. The \ character is explained in the following section echo cat $pipe more remember I/O redirection is checked for first. What would happen if the shell variable substitution was done first?
Doing it Twice
Under some circumstances you may wish the shell to evaluate a command line twice (some examples of when will be demonstrated later). To force the shell to evaluate a line twice you use the eval command.
For example: name="david" variable=name echo $variable these three commands will produce the output name replace the last command with eval echo \$$variable will produce the output david
It works because the shell first evaluates the line echo \$$variable. This produces the line echo $name (replace \$ with $ and $variable with name). The shell then evaluates that line to produce echo david and then it executes the command.
David Jones (11/30/94) Page 41 Basic UNIX Chapter 2
Other Special Characters
So far we've seen that the shell recognises characters such as $ > and | as having special meaning. There are many more, some of which are summarised in table 2.6
For example: echo hello there my friend displays hello there my friend the shell ignores multiple spaces ls ; cd /etc ; ls the ; is used to separate the three commands which are executed one after the other
Character(s) Meaning white space Any white space characters (tabs, spaces) are used to separate arguments (multiple white space characters are ignored) newline used to indicate the end of the command-line character ' " \ special quote characters which change the way the shell interprets special characters & Used after a command, tells the shell to run the command in the background e.g. ls & < >> << ` | redirection characters change where I/O is sent $ used to indicate a shell variable name (more on these later) ; used to execute more than one command in one command line
Table 2.6. Special Shell Characters.
Quotes
What happens when you want to use some of these special characters as a normal character? For example, what if you wanted to display on the screen the message 6 * 5 = 30. Theoretically you might try echo 6 * 5 = 30 but remember what the shell does to *?
Exercise.
2.8. What's the output from this command? echo Multiply is signified by the * symbol
2.9. What's the output from the following shell program? string=hello there how are you echo $string
David Jones (11/30/94) Page 42 Basic UNIX Chapter 2
There are obvious problems in instances where you want to use one of these special characters. The shell does provide mechanisms by which these problems can be surmounted. This is done by using some other special characters called quotes.
Character Name Purpose ' single quote causes the shell to ignore all special characters contained within a pair of single quotes " double quote causes the shell to ignore all special characters EXCEPT $ ` \ contained within a pair of double quotes \ backslash causes the shell to ignore any special character immediately following a backslash
Table 2.7. The Quote Characters.
Examples of Using Quote Characters
hello_string='hello there' echo $hello_string echo '$hello_string' echo "$hello_string" echo \$hello_string echo "\$hello_string"
echo * echo I\'m David. echo '*' echo \*
echo one two three four echo 'one two three four' echo "one two three four"
echo hello there \ my name is david Here the \ is used to ignore the special meaning of the newline character at the end of the first line
echo > temp.dat echo \> temp.dat
Exercises
2.10. Create files with the following names a)stars* b)hello my friend c) "goodbye" d)Now delete them from the file system.
David Jones (11/30/94) Page 43 Basic UNIX Chapter 2
Exercises
2.11. What is the output of the following commands? Explain the output. a)echo "** hello **" b)echo this is a star * c) echo ain\\\\'t you my friend d)echo "the output of the ls command is `ls`" e) echo 'the output of the pwd command is `pwd`'
Getting Help
The UNIX operating system comes with its own on-line help system referred to as the man pages. It is not the best designed help system and some of the documentation can be a little difficult to understand. The main reason for this is that the man pages have been written more as reference material than as learning material.
UNIX People have a Sense of Humour?
Some people say that UNIX warps the mind of its users. The following is a list of comments taken from actual UNIX man pages.
"Acts oddly on nights with full moon." "This manual page is confusing." "This manual page is still confusing." (next release of same command) "It can be used if a disk or the processor is on fire."
Figure 2.8. UNIX People have a sense of humour. Getting access to the manual pages is relatively simple, you use the man command. For example man ls will get you the manual page on ls. Finding something useful from the man pages or finding a man page if you don't know the command can get frustrating.
If you are not sure of the exact word to give the man command you can use one of man -k or apropos (same command different name). This command searches for occurrences of a given keyword in the manual pages and displays the proper word for use with man.
The manual pages are divided into sections with each section dealing with a specific area. Table 2.8 provides a summary of the major sections. These can differ slightly from machine to machine. In particular, manufacturers will add sections specific to commands they add to the operating system.
David Jones (11/30/94) Page 44 Basic UNIX Chapter 2
Section Number Purpose 1User Commands 2System Calls 3Library routines 4Device drivers 5File formats 6Games 7Miscellaneous: ASCII, macro packages etc. 8Commands for system administration
Table 2.8. Section Numbers for Manual Pages (BSD).
For example: Under a Linux box the following command can be used to display all the contents of a specific manual section. ls /usr/man/preform/catn | -F \. ’{ print $1 }’ Where the n is replaced with the number of the section.
Manual pages are stored in a special format called troff or nroff. It is difficult to read a manual page on the screen and use that information at the same time. It is often preferable to print the manual page or produce a file with the manual page in it. The formatting language used by manual pages can make the result difficult to read.
The following sequence of commands should produce a file containing a man page that can be read. (It is up to you to use the man command to figure out what is going on here)
man manpage | ul -tlp > filename
manpage represents the manual page you require filename represents the file in which to place the output
Figure 2.9. Printing a man page.
The man Command
Is used to display on-line help pages of commands and files. Example usage includes
1. man [ section ] title 2. man -k keyword 3. man -f filename
David Jones (11/30/94) Page 45 Basic UNIX Chapter 2
1. Displays an entire manual page (page at a time) with the title title. section refers to the section of the manual the manual page will come from. Refer to Table 2.8 for more information on manual sections. 2. Display a one-line summary of a manual page which discusses the keyword. Exactly the same as the apropos command. 3. Display a one-line summary of a manual page which refers to the file with name filename.
Conclusion
By now you have commenced the long and interesting journey of becoming an expert UNIX user. The more you use UNIX the better you will become at it. You have been introduced to • the UNIX commands to manipulate files and directories, • the concepts of filename substitution and I/O redirection, • the concept of a shell and its responsibilities, • commands to list users and processes on a UNIX machine, • commands to display the status of a machine,
Review Questions
2.1. Write commands to carry out the following tasks: a) change into the directory /usr/local b) produce a file called listing that contains the names of all the files in the /etc directory c) count the number of words in the file /etc/passwd
2.2. What is a link?
2.3. What are the major responsibilities of a UNIX shell?
2.4. What do the following commands do? a) echo 'the *'s are out tonight\' b) ls | cat | wc -l > count c) ls `cat file.list`
2.5. Explain what filenames the following file specifications will match a) ?* b) [!a-z]??[a-z] c) *\*? d) '*?*?'abc*
David Jones (11/30/94) Page 46 Shell Programming Chapter 3
Chapter 3
SHELL PROGRAMMING
It’s a UNIX System. I know this. -- Lex (Jurassic Park)
Objectives
At the end of this chapter you will • have been introduced to the art of shell programming and should be capable of writing simple Bourne shell programs, • be aware of the programming constructs supplied by the shells to enable the writing of shell programs, • be able to use shell variables, • write shell programs that make use of the conditional and repeated action commands supplied by the shell, • be aware of the different types of quote characters recognised by a shell, and • be aware of the importance of program exit status and how to set the exit status of a shell program.
Introduction
Shell programming is a basic skill that every System Administrator should have. The Systems Administrator must be able to read and write shell programs because • there are many tasks that can and should be quickly automated by using shell programs, • on some systems users may ask the Systems Administrator to fix their shell programming problems, and • many software products come with install scripts that have to be modified for your system before they will work.
The Basics of Shell Programming
Simply put, a shell program (sometimes called a shell script or a shell procedure) is a text file that contains standard Unix commands and maybe some specific shell commands. Each command must be on its own line. A shell program is created usually to perform some repetitious job that requires too much typing to be typed in everytime the task must be done.
David Jones (11/30/94) Page 47 Shell Programming Chapter 3
A shell program is executed by the appropriate shell reading the commands in the program one at a time and executing the commands almost exactly as if you were typing them in one at a time.
For example: The following command will display the number of files and directories in the current directory ls | wc -w
Not much typing but if you were going to want to use it many times you might want to cut down on the number of keystrokes. The simplest way would be to create a shell program that contains this command.
How to Create a Shell Program
1. Choose which shell to use. Most Systems Administrators will usually write shell programs using the Bourne shell as this is the only shell guaranteed to be on all Unix systems.
All shell programs in this guide will use the syntax of the Bourne shell.
2. Create a text file containing the commands. Using your favourite text editor create a file containing all the commands (one to a line) you want in the shell program. The first line of the file is used to indicate the name of the shell to use when executing your shell script. This is signified by a line starting with #! followed by the path of the shell you wish to use. (Table 2.4 lists some of the names of shells).
For example: #!/bin/sh for the Bourne shell #!/bin/csh for the C shell
3. Make the shell program executable and readable. Using the chmod command from Section give the appropriate users execute and read privilege on the file. (The shell has to be able to read the file to execute it.) The following command should perform the required task (the chmod command will be explained in a later chapter). chmod u+rx filename
4. Execute the shell program. Execute the program by typing the name of the file.
David Jones (11/30/94) Page 48 Shell Programming Chapter 3
A shell script can also be executed by executing the shell with the shell script as a parameter. This method makes obvious the way in which shell scripts are executed. The shell takes the script as input, executing each command in turn.
For example: To execute a shell script called username with the Bourne shell sh username To execute the shell script username2 with the C shell csh username2
Exercises.
3.1 Create a shell procedure called fc (file count) that uses the command ls | wc -w to count the number of files in a directory.
3.2. Create a shell procedure called d that displays the message The current directory is /home/temp and it contains and on the following line displays the output of the ls command on the current directory.
Shell Support for Programming
To be truly useful a programming language must provide the following services • comments, • variables, • conditional commands, and • repeated action commands.
Conditional and repeated action commands are some of the extra commands that each shell recognises and are not Unix commands. Different shells have different syntax for the two types of commands. All the examples of shell programming in this chapter will use the syntax of the Bourne shell.
How do you tell a Unix command from a Shell command?
A Unix command will always have an executable program stored as a file somewhere on the disk. Try the command which commandname. This command searches through the current path for an executable program called commandname.
The code to perform a shell command is built into the shell program and won't be stored in a file.
David Jones (11/30/94) Page 49 Shell Programming Chapter 3
The Bourne shell is used by most administrator's because it is the one shell guaranteed to be on all Unix machines.
Shell Comments
Any good programmer can tell you the reasons why comments are important. It is especially true for Unix shell procedures that can be truly mystifying, even to the expert.
Comments are especially important when any shell programs you write as a System Administrator are likely to have to be understood and maintained by a fellow administrator in a few years time.
Shell programs use the # character to signify comments. Anything after a # character until the end of the line is considered a comment and is ignored by the shell.
As an administrator you should choose a format for all the shell programs you and anyone else on the administration team will write.
For example: # NAME: test_shell # AUTHOR: David Jones # PURPOSE: Demonstration of comment style # HISTORY: 19/05/94 Created #25/05/94Modified to show mods
ls | more
Using Shell Variables
Like any language the shell has to provide a variable mechanism that can be used to access, store and modify values.
Most programming languages have rules that restrict the format of variable names. For the Bourne shell, variable names must start with either a letter or an underscore character. It can then be followed by zero or more letters, numbers or underscores
To assign a value to a variable you use the following format variable=value (Remember this is the Bourne shell syntax. Using the C shell the format would be set variable=value)
No spaces around the equals sign. If you want to assign a string that contains spaces you must surround it with double quotes
David Jones (11/30/94) Page 50 Shell Programming Chapter 3
For example: count=1 prompt="hello there my friend" bin_dir=/usr/bin
If you wish to use a shell variable's value you must precede the name of the variable by a dollar sign. The shell will examine the command line and see that there is a dollar sign. The shell recognises the $ as a special character that signifies a shell variable name. It will then replace the shell variable name with its value. Finally it will execute the command. (This process is discussed in chapter 2.)
For example: echo $prompt cd $bin_dir
A shell variable does not have to be used for data only. It can be used for commands.
For example: command=mv old_location=/home/david/temp.dat new_location=/usr/adm/account $command $old_location $new_location
Using {}
In some cases you will wish to use the value of a shell variable as part of a larger word. Curly braces {} are used to separate the variable name from the rest of the word
For example: The user wants to copy the file /etc/passwd into the directory /home/david. The following shell variables have been defined. directory=/etc/ home=/home/david
First attempt cp $directorypasswd $home This won't work because the shell is looking for the shell variable called directorypasswd (there isn't one) instead of the variable directory.
Correct Method cp ${directory}passwd $home Surround just the variable name with curly braces, the shell will substitute in the variable's value first
David Jones (11/30/94) Page 51 Shell Programming Chapter 3
Exercises.
3.3 Which of the following are valid shell variable names. (a) temp (b) temp.doc (c) 1abc (d) abc#1 (e) abc_1_ (f) TeMpVaR
3.4 Assuming the following command has already been executed. home=/home/david/ Write commands using the home shell variable to do the following (a) list the contents of the directory /home/david (b) rename the file temp in the directory /home/david to temp2
Predefined Shell Variables
Not only does the shell allow the user to assign values to shell variables, it also supplies a number of predefined shell variables which are used for a variety of purposes including • environment control Controlling settings of the environment in which the shell executes. • parameter passing in shell scripts Allowing the passing of parameters into shell programs.
Shell Variables and Environment Control
When you first log onto a Unix machine the environment in which you are working is initialised. This environment includes things like • your home directory, • the type of terminal you are using, • the type of shell you are using, • the current working directory, • the name of your X-Windows display.
For most of these objects there is an associated shell variable that holds the value. To examine the current state of your environment execute the shell command set.
Table 3.1. lists some of the shell variables typically set for a user's environment (there are numerous others).
David Jones (11/30/94) Page 52 Shell Programming Chapter 3
Variable Purpose HOME your home or login directory PS1 PS2 your first and second command prompts SHELL your current shell UID your user id USER your username TERM the type of terminal you logged in on DISPLAY your X-Windows display (if X is being used) PATH Your execution path
Table 3.1. Environment Shell Variables.
For example: The variable PATH holds the list of directories (separated by :) in which the shell will search for executable programs when you enter a command. $ echo $PATH /usr/ucb:/usr/bin:/bin:/usr/local/bin You can change this path simply be changing the value of the variable PATH $ PATH="/bin" or $ PATH=`echo $PATH`:/home/jonesd/bin add the directory /home/jonesd/bin onto the end of the existing path
Shell Variables and Parameter Passing
When you enter a command (be it an executable program or a shell program) you enter the command name and possibly then some arguments or parameters. The arguments or parameters generally effect the way in which the program works. The Bourne shell has a number of predefined shell variables that are used from within shell scripts to access these parameters.
For example: Assume there is a shell program args that contains the following code echo The first parameter was $1 echo the second was $2 args uses two shell variables $1 and $2. These are predefined shell variables that will contain the value of the first and second parameters passed to the shell program. The following are some example runs of args $ args hello there The first parameter was hello the second was there $ args goodbye The first parameter was goodbye the second was
David Jones (11/30/94) Page 53 Shell Programming Chapter 3
$ args The first parameter was the second was
Table 3.2. lists some of the predefined shell variables (for the Bourne shell) that can be used in shell programs to access the program’s parameters.
Predefined Shell Purpose Variable $0 the name of the shell procedure being executed $1 thru $9 the first through ninth parameter passed to the shell program (see shift) $# the number of parameters passed $* used to represent all the parameters passed $$ contains the process id number of the current process, often used to create unique temporary files $? used to hold the exit status of the last command, a value of 0 means the last command succeeded, 1 means it failed "$@" used to represent all the parameters passed, especially if the parameters contain spaces, must use the surrounding ""
Table 3.2. Predefined Shell Variables (Bourne Shell)
Exercises.
3.5. Write a shell program called arguments that tells the user how many command line arguments were passed to it and what they are. For example: arguments hello there arguments accepted 2 command line arguments they were hello there
Your shell program should use shell variables wherever possible.
Using More than 9 Parameters
The obvious way to display the tenth parameter passed to a shell program would be echo $10. The problem here is that the shell sees $1 and immediately substitutes in the value of the first parameter.
David Jones (11/30/94) Page 54 Shell Programming Chapter 3
Instead you must use the shell command shift. shift moves all the parameters one parameter to the left i.e. the first parameter disappears, the second parameter becomes the first, the third the second and so on. So after executing the shift command the tenth parameter has become the ninth parameter.
For example: The following shell program called args2 demonstrates the use of shift echo parameters are $* shift echo parameters are $* shift Example use of the script $ args2 hello there friend parameters are hello there friend parameters are there friend
The exit Command
Every Unix program when it finishes execution returns an exit status. The exit status is used to indicate whether or not the program succeeded.
An exit status of 0 indicates success. An exit status of anything else indicates failure.
The exit command is used by a shell procedure to change its exit status.
exit Command Format.
exit number
number should be 0 if the shell procedure succeeded. Any other value indicates failure.
Figure 3.1. The exit Command.
In a Bourne shell program the predefined variable $* is used to hold the exit status of the last command executed.
For example: The following is code for a shell program called search !#/bin/sh # use the -s (silent) option of grep # first parameter should be a username # see if the username exists grep -s $1 /etc/passwd echo $?
David Jones (11/30/94) Page 55 Shell Programming Chapter 3
Generally speaking this program will display 0 if the grep found the username in the password file and 1 if it didn't.
The Conditional Commands
A conditional command is a command that makes a choice depending on the outcome of some condition. Shells offer two conditional commands if and case. Both of which are similar to constructs you will have met in other programming languages.
The if Command
if Command Format.
if commandt then command-list fi
if commandt then command-list1 elif command1 command-list2 fi
command-list is a list of 1 or more Unix commands. commandt is any Unix command.
Figure 3.2. The if Command.
The main difference with the Unix if is that it waits on the outcome of a Unix command.
For example: if grep -s $1 /etc/passwd then echo user $1 was found in password file else echo user $1 NOT FOUND fi
The command grep -s $1 /etc/passwd will search the password file for the word entered as the first parameter. If the command is successful the first echo command will be executed.
David Jones (11/30/94) Page 56 Shell Programming Chapter 3
The case Command
The other type of conditional command supported by the Bourne shell is the case command. The case command allows the user to compare a single value against multiple other values and when a match is found execute the associated command list.
case Command Format.
case value in pat1) command command;; pat2) command-list;; *) command-list;; esac
Figure 3.3. case Command Format.
value is compared against each of the patterns signified by pat1.. one by one. If there is a match then the command list associated with that pattern is executed. The ;; symbol is used to signify the end of a command list for a particular pattern.
Once the end of the appropriate command list is reached, execution is restarted at the command directly after the esac. Only the first pattern matched will ever be executed.
The * pattern is optional and is used to match any value and so it acts as the default. (The * should never be the first choice. * matches everything and so the other choices will never get a look in.)
The * is an example of using wildcard characters in the patterns. You can also use the ? and [] wildcard symbols in the patterns. The | symbol can be used as an OR operator for patterns. i.e. pat1 | pat2 will be matched if at least one pattern is a match
For example: case $1 in hello ) echo hello there;; goodbye ) echo why goodbye echo you haven't said hello;; T* | *T) echo that word started or ended with a T;; [a-z]?) echo that word started with a echo lowercase letter and was echo two letters long;; *) echo sorry I don't know that word;; esac
David Jones (11/30/94) Page 57 Shell Programming Chapter 3
case $# in 0) echo no parameters;; 1) echo one parameter;; 2) echo two parameters;; 3) echo three parameters;; esac
The test Command
Almost every person who has ever written a shell procedure has written one called test and then wondered why it wouldn't work. The reason is that there is already a Unix program called test. When you enter test at the command line it will typical find the Unix command before it finds your shell procedure.
The test command is generally used in if statements for testing one or more conditions.
test Command Format.
test expr [ expr ]
The [ is actually the name of an executable program (Try the command which [ or whereis [). When it is executed you are actually running the program called test.
Figure 3.4. test Command Format.
The symbol expr can be one of the many expressions understood by the test command. test recognises expressions that test for the existence of files, compare the values of integers, compares strings etc. Tables 3.4-3.7 provide a sample of some of the available formats of expr.
Expression Result -z string true if the length of string is 0 -n string true if the length of string is not zero string1 = string2 true if the two strings are identical string1 != string2 true if the two strings are NOT identical string true if string is NOT the null string
Table 3.4. Expressions for testing String Conditions
David Jones (11/30/94) Page 58 Shell Programming Chapter 3
Expression Result int1 -eq int2 true if the two integers are equal int1 -ne int2 true if the two integers are not equal int1 -gt int2 true if int1 is greater than int2 int1 -ge int2 true if int1 is greater than or equal to int2 int1 -lt int2 true if int1 is less than int2 int1 -le int2 true if int1 is less than or equal to int2
Table 3.5. Expressions for testing Integer Conditions.
Expression Result -r file true if file exists and is readable -w file true if file exists and is writable -x file true if file exists and is executable -f file true if file exists and is a regular file, or true if file exists and is not a directory -d file true if file exists and is a directory -h file true if file exists and is a symbolic link -c file true if file exists and is a character special file -b file true if file exists and is a block special file -p file true if file exists and is a named pipe -u file true if file exists and its set user ID bit is set -g file true if file exists and its set group ID bit is set -k file true if the file exists and its sticky bit is set -s file true if file exists and has a size greater than 0
Table 3.6. Expressions for testing File Conditions.
David Jones (11/30/94) Page 59 Shell Programming Chapter 3
Expression Result ! the unary negation operator, reverses the result of an expression -a binary AND operator -o binary OR operator ( expr ) parentheses used for grouping, parentheses have special meaning to the shell so to use them in the test command you must quote them
Table 3.7. Logical Operators.
Example uses of the test Command
If the first parameter in a shell procedure is hello display a greeting message. if test $1 = hello then echo hello there fi
If the first parameter is greater than or equal to the second value (and both are integers) display an appropriate message. if [ $1 -gt $2 -o $1 -eq $2 ] then echo $1 is greater than or equal to $2 fi
If there are no parameters to this shell procedure display the appropriate message. if [ $# -eq 0 ] then echo You didn\'t enter any parameters fi
Exercises.
3.6. Rewrite the shell program arguments from exercise 3.5. so that it tests to see if there were any command line arguments. If there weren't any it should display the message No parameters passed. Otherwise it should carry out the actions specified in exercise 3.5.
David Jones (11/30/94) Page 60 Shell Programming Chapter 3
Exercises.
3.7. Write a shell program exists that takes a filename as a command line parameters. If there are no command line parameters it should display a message explaining how to use it. If there are parameters it should test to see if a file with that name exists and display an appropriate message. For example: exists /etc/passwd The file /etc/passwd exists.
Repeated Action Commands
The Bourne shell provides three repeated action commands each of which corresponds to constructs you may have met before in other programming languages • for • while • until
The for command
Execute a list of commands for a specified list of values.
for Command Format.
for variable in word1 word2 word3 ... wordn do command-list done
Figure 3.5. for Command Format.
variable will take on each value in the list word1 word2 word3 ... wordn one value at a time. For each value the commands in command-list will be executed.
For example: count=0 for param in $* # for every parameter do count=`expr $count + 1` # expr talked about later echo parameter $count is $param # display the number parameter and its value done
David Jones (11/30/94) Page 61 Shell Programming Chapter 3
echo days of the week are for today in sun mon tue wed thu fri sat do echo $today done
Exercises.
3.8. Modify the shell procedure exists written for exercise 3.7 so that it allows the user to enter multiple filenames. For example exists /etc/passwd /home/jonesd temp.dat
The while Command
While the exit status of a command is true continually execute a list of commands.
while Command Format.
while command1 do command-list done
Figure 3.6. while Command Format.
command1 is executed and if its exit status is true then the command- list is executed. This is repeated until command1 returns a false exit status.
For example: display a countdown from 10 to 0 count=10 while [ $count -ge 0 ] do echo $count $count=`expr $count - 1` done
David Jones (11/30/94) Page 62 Shell Programming Chapter 3
The until Command
Repeat a command list until a command returns an exit status of TRUE.
until Command Format.
until command1 do command-list done
Figure 3.7. until Command Format.
command1 is executed and while it returns an exit status of FALSE the command-list is executed. This continues until command1 returns an exit status of TRUE.
For example: display a countdown from 10 to 0 count=10 until [ $count -lt 0 ] do echo $count $count=`expr $count - 1` done
Some Other Useful Commands
Table 3.8. lists some of the simple Unix commands you are likely to use when writing shell scripts. The next chapter talks about some of the more complex ones in detail. With the following use the combination of the examples below and the man pages for each command to work out what it does.
David Jones (11/30/94) Page 63 Shell Programming Chapter 3
Command Purpose expr evaluates logical and arithmetic expressions cut used to extract fields of data from the lines of a file or the output of a command sort sorts its standard input into order head display the first X number of lines of standard input tail display the last X number of lines of standard output file attempts to describe the type of a file grep search for lines containing a character string uniq remove duplicated lines from standard input strings displays any ASCII characters contained in a binary file tr translates specified character(s) into other character(s) paste the opposite to cut, puts lines back together
Table 3.8. Some other useful commands.
Examples
expr 5 + 6 expr 5 \* 10 + 5 echo "105 - 5 = " `expr 105 \- 5`
cut -d: -f1,6 /etc/passwd cut -c1-8 /etc/passwd cut -d: -f1 /etc/passwd t -d: -f1 /etc/passwd | sort
cat /etc/group sort /etc/group sort -r /etc/group
cut -d: -f1,6 /etc/passwd | tr : ' ' cut -d: -f1 /etc/passwd | tr [a-z] [A-Z]
Conclusion
As a Unix Systems Administrator (and throughout the rest of this guide) will be called on often to write, debug or change shell scripts. It is important that you understand the concepts and can put them into practice. The more you program in a language the easier it becomes.
David Jones (11/30/94) Page 64 Shell Programming Chapter 3
Review Questions
3.1. Write a shell procedure called my_sum which accepts a list of numbers and then calculates the sum of all the numbers.
bash$ my_sum 1 2 3 4 5 Total = 15
3.2. Modify the my_sum shell script so that it produces output resembling the following.
csh> my_sum 1 2 3 4 5 1 + 2 + 3 + 4 + 5 = 15
3.3. Write a shell script called file_type that accepts as parameters the names of files. The script should decide what type of file the file is and inform the user. The shell script should display a help message if no parameters were passed and be able to handle multiple filenames as input.
File types are listed in table 2.1.
David Jones (11/30/94) Page 65 Advanced Unix Use Chapter 4
Chapter 4
ADVANCED UNIX USE
The distance is nothing; it is only the first step that is difficult. -- Marie de Vichy-Chamrond.
Objectives
At the end of this chapter you will • be able to write shell programs that make use of shell functions, • become aware of how to use input/output redirection from within shells, • be introduced to the concepts of processes, programs and signals, • understand and use regular expressions to perform a number of tasks, • be able to make use of awk and understand some of its intricacies, and • be able to make use of sed.
Introduction
In chapter 3 you were provided with an introduction to some simple UNIX comamnds and to the art of writing shell scripts. In this chapter you will encounter some of the more advanced commands and some of the advanced techniques that can be used in writing shell scripts.
Before getting started lets be certain about the terminology that is to be used in relation to shell programs. Table 4.1 lists the terminology that will be used throughout this book in relation to shell programs. Other people may use slightly different terms.
Term Explanation Shell program an executable file that contains UNIX and shell commands and can be interpreted by a UNIX shell Shell script same as a shell program Shell procedure same as a shell program shell function similar to a function in C or Pascal, only used within a shell program (introduced below)
Table 4.1. Terminology for shell scripts and functions.
David Jones (11/30/94) Page 66 Advanced Unix Use Chapter 4
Shell Functions
Any good programming langauges provides support for functions. Functions serve to group collections of commands that carry out an often required task. The Bourne shell programming language is no exception.
Actually that isn't quite true. Older versions of the Bourne shell actually didn't support functions. On most System V based Unix operating systems this is no longer a problem as the Bourne shell has been updated. However some BSD based machines may have old versions of the Bourne shell that don’t recognise shell functions. (bash supports them)
Bourne Shell Function Syntax
name() { command-list }
Figure 4.1. Bourne Shell Function Syntax.
Chapter 3 introduced the predefined shell variables $0 $1...$9 and the special meaning that shell programs give them. The meaning of these variables change when they are used inside of a shell function.
Some of the differences include: • $1 $2... are used to represent any parameters passed to the function (not those passed to the shell program), • $0 still contains the name of the shell program, • $* is the list of all the parameters passed to the shell function.
There is only one pool of shell variables in a shell program. The Bourne shell does not support any notion of local variables. Any variable created in a shell function can be accessed from outside of that function.
For example: Create a shell procedure containing the following code #!/bin/sh
# an example shell function called hello hello() { # display the function's 1st param echo hello there $1 # display all function parameters echo $*
David Jones (11/30/94) Page 67 Advanced Unix Use Chapter 4
echo $0 local_var="hello" }
# call the function and pass to it all the # programs parameters hello $* # show that there is no scope rules echo "Local variable value = " $local_var # call the fucntion but pass it no parameters hello
In keeping with the standard concept of functions, shell functions also have a return status. The return command indicates the shell function’s return status. A successful shell function should return a value of 0 while a shell function that fails should return a non zero value.
For example: temp() { return 1 }
if temp then echo succeeded else echo failed fi
Exercises.
4.1. Convert the shell script file_type from review question three from chapter 3 into a function. The function should be called file_type and it should set a variable type to the type of the file passed in (the function should only take one parameter).
An example use of the function would be file_type /etc/passwd echo "/etc/passwd is a $type."
Input/Output for Shell Programs
This section shows you how to • obtain interactive input from the user while a shell program is executing, • read input from a file while the shell program is running, and • introduce you to some of the advanced features of the echo command.
David Jones (11/30/94) Page 68 Advanced Unix Use Chapter 4
The read Command
The read command is used to take a line from standard input and place it into a list of shell variables. On versions of Unix System V Release 2 or later the input to the read command can be redirected to come from a file.
read Command Format.
read variable-list
variable-list is a list of one or more shell variables.
The first word read is placed into the first variable, the second into the second and so on. Any excess words are placed into the last shell variable.
read always returns an exit status of zero unless an end of file condition is met. With interactive input from the keyboard this is when the
Figure 4.2. read Command Format.
For example: Using the following shell program called input read first second echo "Variable first contains " $first echo "Variable second contains " $second Some example runs of input input run the shell script hello there my friend input some data Variable first contains hello Variable second contains there my friend input hello Variable first contains hello Variable second contains Assume there exists a file call input_file with the following contents hello there The following command line runs the script with the above file as input rather than using standard input (the keyboard) input < input_file Variable first contains hello Variable second contains there Assume the following is in a shell procedure while read whole_line do echo I just got the line $whole_line
David Jones (11/30/94) Page 69 Advanced Unix Use Chapter 4
done < /etc/passwd This shell procedure will go through the /etc/passwd file a line at a time. The read command continues to succeed until end of file is reached.
Exercises.
4.2. It is often the case that you need to ask a user whether or not to do something. Write a shell function called ok that displays the following message (y,n) ==> then waits for the user to enter a value. The function should set the value of the shell variable response using the following guides • if the user enters y (or yes Yes YES etc) it should contain y • if the user enters n (or no No nO etc) it should contain n • if the user enters anything else it should contain ERROR
4.3. Assume the existence of a file called amounts in which each line contains two numbers (you don't know how many lines it will have). For example, 5 15 65 40 Write a shell procedure calculate that uses the file amounts as input and produces a file totals with the following format 5 + 15 = 20 65 + 40 = 105
Extensions to the echo Command
The echo command by default always displays a terminating new line at the end of each command. There is a way of circumventing this. As of the System III version of UNIX, special echo escape characters were added. These escape characters are listed in Table 4.2.
Using \c to supress the terminating newline character will only work on some versions of UNIX. The alternative to using \c on most systems is a - n switch for the echo command.
For example: Use the \c character to surpress the newline character (this won’t work on Linux) echo "Enter a number ==> \c" read number Use the -n switch to surpress the newline character echo -n "Enter a number ==> " read number
David Jones (11/30/94) Page 70 Advanced Unix Use Chapter 4
Character Purpose \b display the backspace character \c display the line without the terminating newline \f do a form feed \n display the newline character \r display the carriage return character \t display the tab character \\ display the backslash character \nnn display any character with the ASCII value signified by nnn (a 1 to 3 digit octal number starting with 0)
Table 4.2. echo Escape Characters.
Trapping Signals
When you hit the CTRL-C combination to stop the execution of a shell program the shell sends a software termination (sometimes referred to as the TERM signal) signal to the program's process. By default all shell programs when they receive this signal will terminate immediately.
The UNIX operating system generates a number of different signals (Table 4.3. lists some of them). Each signal has an associated unique identifying number and a symbolic name. Table 4.2 lists the signals defined by a UNIX machine using a SVR4 version of UNIX.
Each process has an a default behaviour for every signal. When it receives a signal it carries out this default behaviour. The most common default actions are • terminate, or In this case the process will cease to exist. Signal number 9 (SIGKILL is an example of one signal that causes this behaviour.) • ignore. Many processes will simply ignore some signals and carry on processing. Some signals cannot be ignored, SIGKILL for example.
There are mechanisms available that allow programmers or Systems Administrators to change the default behaviour associated with some signals. The trap command listed below is one of them.
The kill Command
There are a number of methods by which you can send a signal to a process, for example hitting the key combination CTRL-C will usually send the SIGTERM signal to a process. Another more general method is to use the kill command.
David Jones (11/30/94) Page 71 Advanced Unix Use Chapter 4
kill Command Format.
kill [ -signal ] pid ...
Sends the signal specified by the number signal to the process identified with process identifier pid. The kill command will handle a list of process identifiers.
By default kill sends signal number 15 (the TERM signal).
Figure 4.3. kill Command Format.
The ps command lists processes, their process identifiers and a variety of other information.
For example: ps PID TTY STAT TIME COMMAND 26796 pp0 S 0:01 -bash the user’s login shell 30434 pp0 R 0:00 ps the process for ps kill 26796 this will have the affect of logging the user out as the TERM signal has been sent to the login shell which will terminate on receiving it.
The trap Command
There are situations in which the default behaviour associated with a signal needs to be overridden.
For example: It is common to write shell scripts the create temporary files. In most cases it is good practice to delete these temporary files when the shell script exits. However what happens if a user sends a shell program process the SIGTERM signal while it is running? By default the process will cease immediately. It will NOT delete any temporary files. In this case it would be nice to be able to delete the temporary files first, before exiting.
Another example is the case where you don’t want the user to be able to quit out of a shell procedure. In this case you want signals like SIGTERM to be ignored.
The trap command allows the default behaviour of signals to be overridden.
David Jones (11/30/94) Page 72 Advanced Unix Use Chapter 4
Symbolic Name Number Purpose SIGHUP 1 hangup SIGINT 2 interrupt (rubout) SIGQUIT 3 quit (ASCII FS) SIGILL 4 illegal instruction (not reset when caught) SIGTRAP 5 trace trap (not reset when caught) SIGIOT 6 IOT instruction SIGABRT 6 used by abort, replace SIGIOT in the future SIGEMT 7 EMT instruction SIGFPE 8 floating point exception SIGKILL 9 kill (cannot be caught or ignored) SIGBUS 10 bus error SIGSEGV 11 segmentation violation SIGSYS 12 bad argument to system call SIGPIPE 13 write on a pipe with no one to read it SIGALRM 14 alarm clock SIGTERM 15 software termination signal from kill SIGUSR1 16 user defined signal 1 SIGUSR2 17 user defined signal 2 SIGCLD 18 child status change SIGCHLD 18 child status change alias (POSIX) SIGPWR 19 power-fail restart SIGWINCH 20 window size change SIGURG 21 urgent socket condition SIGPOLL 22 pollable event occured SIGIO 22 socket I/O possible (SIGPOLL alias) SIGSTOP 23 stop (cannot be caught or ignored) SIGTSTP 24 user stop requested from tty SIGCONT 25 stopped process has been continued SIGTTIN 26 background tty read attempted SIGTTOU 27 background tty write attempted SIGVTALRM 28 virtual timer expired SIGPROF 29 profiling timer expired SIGXCPU 30 exceeded cpu limit SIGXFSZ 31 exceeded file size limit
Table 4.3. Signals on a SYSVR4 Box.
David Jones (11/30/94) Page 73 Advanced Unix Use Chapter 4
trap Command Format.
trap [commands] [signals]
trap with no parameters displays a list of current trap assignments.
signals a list of signals to change the default action for commands is one or more commands that will be executed when ever one of the listed signals is received
If commands is the null string then the specified signals are ignored.
If only signals is used the list of signals will be reset back to the original default actions.
Figure 4.4. kill Command Format.
For example: ps execute ps to find out the pid of login shell PID TTY STAT TIME COMMAND 26796 pp0 S 0:01 -bash 30434 pp0 R 0:00 ps trap "echo I received signal 2; echo goodbye" 2 everytime the shell receives signal 2 it should perform the echo commands kill -2 26796 send signal 2 to the login shell I received signal 2 goodbye trap view the default actions trap -- 'echo you sent signal 2' SIGINT trap 2 Reset the default actions for signal 2. trap 'echo logged off at `date` >> $HOME/logoffs' 0 When the user logs out append a message to the logoffs file. Type the command in at the shell prompt and then logout.
David Jones (11/30/94) Page 74 Advanced Unix Use Chapter 4
Exercises.
4.4. Write a shell procedure called signals that loops forever doing nothing. However if it should receive the signal 15 it should display an appropriate message.
For example: signals& run the shell script in the background kill 34256 send signal 15 to the signals process Signals received signal 15
Regular Expressions
It has been mentioned that the shell has built in support for character matching with the * ? and [] constructs. However some Unix commands understand a more powerful method for character matching called regular expressions.
Regular expressions are recognised, in one form or another, by the following Unix commands ed, ex, sed, awk, grep, egrep, expr and even vi. Regular expressions are used by these commands to match a sequence of characters.
Regular expressions are sometimes referred to as REs.
Regular expressions provide more powerful and varied mechanisms for matching and modifying text. Table 4.3 lists the basic symbols recognised by REs and what they will match.
The power in regular expressions comes when they are combined. The primitives from Table 4.4 can be combined together to perform complex matches. Table 4.5 provides a number of examples.
Symbols Purpose c any character other than \ [ . * ^ ] $ (or some subset of these depending on implementation) matches itself \ remove any special meaning from a character . match any one character ^ match the start of the line when it is the first character in the RE $ matches the end of the line when it is the last character in the RE * match zero or more matches of previous character or expression [chars] match any one character from the string chars [^chars] match any one character NOT in the string chars
Table 4.4. Summary of Regular Expressions.
David Jones (11/30/94) Page 75 Advanced Unix Use Chapter 4
Example RE What it matches unix matches the work unix and nothing else [Uu]nix matches any word starting with a u or a U followed by nix [^aeiouAEIOU]* matches any sequence of characters that does not contain a vowel ^abc$ matches lines that contain abc only hel. match any word starting with hel followed by one letter e.g. help hell hel<
Table 4.5. Example Regular Expressions.
Exercises.
4.5. Describe the text that the following REs will match. a) abc.def b) [aA][bB][cC]dD c) $ d) [^sS]*[aeiou]*[a-z]* e) hello^there$friend f) [abc]\* g) \[hello there\.*$
One of the difficulties with regular expressions is that there is more than one variety. Each different variety has its own extensions and limitations. The varieties of REs include • limited regular expressions, As used by the commands ex, ed, sed and grep. • full regular expressions, and As used by egrep, awk and some versions of vi • System V extensions to regular expressions. Versions of System V support commands that recognise extensions to limited REs
Limited Regular Expressions
Limited regular expressions recognise the basic symbols from Table 4.3 and in addition support the storing of matched expressions into registers. The system maintains a set of registers starting from register one. The act of storing an RE into a register is sometimes referred to as tagging.
A regular expression surrounded by \( \) is considered tagged (the \ character is included because the () characters have special meaning to the shell) and is stored into the next register. The first tagged RE will be placed into register 1, the second into register 2 and so on. The value stored in each register is accessed by \n where n is the number of the register.
David Jones (11/30/94) Page 76 Advanced Unix Use Chapter 4
For example: \(tick\) tock \1 will match the strings tick tock tick \(aa\)bb\1* will match any string that starts with aabb and then has any number of aa characters following
Tagging is especially powerful when used in combination with some of the ed commands discussed later in this chapter.
Full Regular Expressions
Full regular expressions include the symbols outlined in table 4.3 and add the additional constructs from table 4.6. They do not support the notion of tagging supported by limited regular expressions.
Symbol Meaning + matches one or more occurrences of the previous RE ? matches zero or one occurrences of the previous RE | matches either of two REs separated by a | or both REs separated by a new line () used to group an RE so that * + ? | can be applied to it
Table 4.6. Extensions for Full REs.
System V Extensions
System V provides an extension to limited REs that allows you to specify the exact number of matches you require.
Symbol Meaning \{n\} match exactly n occurrences of a single character RE \{n,\} match at least n occurrences of a single character RE \{n,m\} match between n and m occurrences of a single character RE
Table 4.7. System V Extensions to Limited REs.
For example: abc\{5\} matches a string starting with abc followed by 5 occurrences of c
David Jones (11/30/94) Page 77 Advanced Unix Use Chapter 4
Exercises.
4.6. Write grep commands that use REs to carry out the following. a)find any line starting with j in the file /etc/passwd (equivalent to asking find any username that starts with j) b)find any username that starts with j and uses bash as their login shell c) find any user that belongs to a group with a group number between 0 and 99
ex, vi and Regular Expressions
When you use commands like :w or :q from within the vi editor you are actually executing an ex command. ex is a powerful line oriented text editor that will accept commands either interactively or from a ex script file.
Our main interest in ex here is its ability to use regular expressions to perform complex manipulation of text. Particularly interesting is the ability to use these ex commands from vi. Similar commands can also be used under ed and sed.
You will probably already have used a number of ex commands during your normal use of vi. Whenever you enter : and then some command (e.g. :w :wq :q) you are entering an ex command.
Specifying Addresses
ex commands support the notion of addresses that allow you to specify the range of lines on which you want the command to be applied to. Table 4.8 specifies the variety of methods by which addresses can be specified.
For example: :1,5w!hello.dat Save lines 1 through to 5 into the file hello.dat :/hello/,/goodbye/w!inbetween Save to the file inbetween all the lines between the next line that contains hello and the next line after that that contains goodbye :.,$w!rest Save all the lines from the current line to the last line into the file rest
David Jones (11/30/94) Page 78 Advanced Unix Use Chapter 4
Address Purpose . addresses the current line $ addresses the last line of the file number addresses the line which is number lines from the start of the file letter addresses the line which has been tagged with the label corresponding with the lower case alphabetical character letter * /expression/ addresses the first line which matches the regular expression expression by searching FORWARD from the current line ?expression? addresses the first line which matches the regular expression expression by searching BACKWARD from the current line address+number addresses the line which is number lines FORWARD from address address-number addresses the line which is number lines BACKWARD from address address1,address2 specifies the range from address1 to address2 , stands for the pair 1,$ i.e. the whole file, from the first line to the last line ; stands for the pair .,$ i.e. from the current line to the end of the file * lines are tagged using the k command outlined below Table 4.8. Specifying Addresses for Regular Expressions.
Other Commands
w and q (write and quit) are just two of a wide arrange of ex commands. Some of the others are listed in table 4.8. (This is not a complete list.)
ex Command Format.
address command count
address as specified in table 4.6 command as specified in table 4.7 count how many times to perform the command count defaults to 1
Figure 4.5. ex Command Format.
The buffer referred to in Table 4.9 (for the d command) could be one of the 27 buffers maintained by ex and vi for the purpose of storing text.
David Jones (11/30/94) Page 79 Advanced Unix Use Chapter 4
There is an unnamed buffer that is used to store text from a normal delete operation. There are then 26 named buffers (referred to by the names a through to z).
For example: 4d3 Starting from line 4 delete 3 lines 1,$s/hello/HELLO/ replace the first occurence of hello with HELLO from every line in the file 1,$s/hello/HELLO/g replace all occurrences of hello with HELLO from every line in the file 5d+++ delete the 5th line and then move on three lines da2 delete two lines starting from the current line and put them into the a buffer 5pa insert the contents of the a buffer after line 5
Command Purpose line a append text after line range co line copy lines specified by range to just after line range d buffer count delete lines specified by range and count. Place them into buffer if specfied range j count join text from specifed lines into one line n edit the next file specified in the command line q quit line r file read the contents of file and insert it after line sh start up another shell range s/pat1/rep1/options replace all patterns matching the RE count pat1 in the specified range with rep1 (will only work on the first occurence of pat1 for each line unless g is used as an option u undo the previous change range w file write the specified lines to the file. w>>file appends the lines to the file
Table 4.9. Sample of ex commands.
David Jones (11/30/94) Page 80 Advanced Unix Use Chapter 4
Exercises.
4.7. Do the following using both vi and sed a)replace all occurrences of Unix with Unix[tm] b)replace all occurrences of Write with Writeln for all lines between the next occurence of BEGIN and the next occurence of END c) replace all occurrences of > that start a line with nothing
4.8. What do the following ex commands do? a) .+1,$d b)1,$s/OSF/Open Software Foundation/g c) 1,/end/s/\([a-z]*\) \([0-9]*\)/\2 \1/
awk
awk (named for its inventors Aho, Weinberger and Kernighan) is a very useful Unix tool used by Systems Administrators to perform a number of tasks. awk's primary tasks are involved with searching, modifying and creating reports based on the contents of a file.
The following reading provides you with an introduction to this powerful tool. You will notice that the syntax of awk has much in common with that of the C programming language.
Reading.
Resource Materials Chapter 4.1
Purpose.
Provide you with an overiew of the awk command.
For example: Place the following into a shell script, it will find any line that starts with the first parameter to the shell script in the file indicated by the second parameter. To use a shell variable in an awk script you need to surround it by ' '. awk 'BEGIN { FS=":"; print "Usernames matching '$1' are.." } /^'$1'/ { print $1; X = X + 1 } END {
David Jones (11/30/94) Page 81 Advanced Unix Use Chapter 4
print "There were", X, " of them." }' $2
Exercises.
4.9. A computing academic stores the marks for his students in a colon delimited file called results.dat. The file has the following format surname:firstname:a1Mark:examMark
For example: jones:david:5:55 blow:joe:10:100
Write an awk script to produce a file called final.dat that displays the information taken from results.dat in the following format: jones, david 60 blow, joe 110
Assume: a1Mark is out of 10 and is worth 10% of the final result. examMark is out of 90 and is worth 90% of the mark.
4.10. Write an shell script called user that accepts a list of group identifiers and uses awk to produce a list of users who belong to that group and displays the total number of users who belong to that group.
user 0 1 2 Group 0 has no users root daemon uucp uucpa Group 1 has 4 users Group 2 has no users
Perl
The following description of perl is taken from the frequently asked questions list of the Usenet newsgroup comp.lang.perl. Perl is becoming an increasingly important tool in toolkit of a Systems Administrator. We do not cover it in any detail in this guide because of a lack of space.
David Jones (11/30/94) Page 82 Advanced Unix Use Chapter 4
What is Perl?
A programming language, by Larry Wall
Here's the beginning of the description from the Perl man page:
Perl is an interpreted language optimized for scanning arbitrary text files, extracting information from those text files, and printing reports based on that information. It's also a good language for many system management tasks. The language is intended to be practical (easy to use, efficient, complete) rather than beautiful (tiny, elegant, minimal). It combines (in the author's opinion, anyway) some of the best features of C, sed, awk, and sh, so people familiar with those languages should have little difficulty with it. (Language historians will also note some vestiges of csh, Pascal, and even BASIC-PLUS.) Expression syntax corresponds quite closely to C expression syntax. Unlike most Unix utilities, Perl does not arbitrarily limit the size of your data--if you've got the memory, Perl can slurp in your whole file as a single string. Recursion is of unlimited depth. And the hash tables used by associative arrays grow as necessary to prevent degraded performance.
Perl uses sophisticated pattern matching techniques to scan large amounts of data very quickly. Although optimized for scanning text, Perl can also deal with binary data, and can make dbm files look like associative arrays (where dbm is available). Setuid Perl scripts are safer than C programs through a dataflow tracing mechanism which prevents many stupid security holes. If you have a problem that would ordinarily use sed or awk or sh, but it exceeds their capabilities or must run a little faster, and you don't want to write the silly thing in C, then Perl may be for you. There are also translators to turn your sed and awk scripts into Perl scripts.
Conclusions
Throughout this chapter you have been introduced to some of the more advanced commands that form some of the tools of the trade for a Systems Administrator. These are tools you will use again and again as a Unix Systems Administrator. It is important that you become familiar with them.
David Jones (11/30/94) Page 83 Advanced Unix Use Chapter 4
Review Questions
4.1. Write a shell program called mycp. It is to be a replacement for the cp command but instead of simply copying the file it should first check to see if the destination file exists. If it does it should ask the user whether or not they • wish to copy over the existing destination file, • move the existing destination file to filename.old and then copy the new destination, or • not copy at all.
4.2. The existing cp program supports the following type of command. cp file1 file2 file3 directory That is copying more than one file into a destination directory. Modify your mycp command from the previous question to allow this type of syntax.
4.3. It is often the case that specific users on a system continually use too much disk space. There are a number of solutions to this problem including quotas (talked about in a later chapter). Implement another solution to this problem that has the following characteristics. • maintain a file called disk.hog which contains the usernames (one per line) of the offending users and the amount of disk space they are allowed to have. For example jonesd 50000 okellys 10 • maintain a script find_hog that is run once a day and performs the following tasks • for each user in disk.hog discover how much disk space they are using • if the amount of disk space exceeds the allowed amount write their username to a file offender
HINTS. The command du -s directoryname can be used to find out how much disk space the directory directoryname and all its child directories use. The file /etc/passwd records the home directory for each user.
David Jones (11/30/94) Page 84 Systems Administration Theory Chapter 5
Chapter 5
SYSTEMS ADMINISTRATION THEORY
No theory is good except on condition that one use it to go beyond. -- Andre Gide
Objectives
By the end of this chapter you should • have an idea about the non-technical responsibilities of a Systems Administrator, • have gained an understanding of why policies, procedures and penalties are important, • be aware of the importance of keeping the user population informed and happy, and • know the importance and type of documentation that should be kept.
Introduction
Not every responsibility of a Systems Administrator is dependent on the type of machine or operating system being used. There are some tasks that have to be undertaken whether Unix, VMS or any other operating system is being used. These tasks can be as, if not more, important than the technical portion of the job.
These machine independent tasks include • implementing and enforcing a set of policies and penalties, It is important that a written document exists that specifies what can and can't be done on the system and by whom. The document should be available and all users should be aware of it. It must include mention of what happens when policy is broken. • procedures and automation, Where ever possible human intervention should be minimised and they should exist set, standard procedures for performing tasks. • maintaining documentation, Recording everything that has ever been done to your system and how it was done. • maintenance and observation, and Keeping track of the state of the system. Is the system still up? Is everyone still behaving? • user liaison. Letting the users know what you are doing and what is happening with the system. The communication mechanism should be bi-directional allowing the users tell you what they want to do.
David Jones (11/30/94) Page 85 Systems Administration Theory Chapter 5
Policy and Penalty
Think of the computer systems you manage as an environment in which humans live and work. Like any environment, if anarchy is not to reign supreme then there must exist some type of behavioural code that everyone lives by. In a computer system this code is liable to include (amongst others) such things as • a single person shall not hog all the resources (disk, cpu etc), • users who work for accounting have xyz access, those who work for research have zyx access, and • no-one should endeavour to access areas in which they are not allowed.
A set of rules by themselves is not enough. There must also exist • a set of penalties to be applied if one of the policies is broken, • a person(s) charged with detecting the breaking of policy, • a person(s) charged with deciding the appropriate policy, • a mechanism for the change of policy and penalties, and • a mechanism for informing users of the policy and the penalties. If any one of these necessary components is missing the system may not work to the best of its ability.
Types of Policy
On a computer system there should be set policies (of which everyone is aware) for a whole range of issues including (but not limited to) • network access, Who can go where on the net? • resource utilisation, How much of a resource can each category of user use? How much does it cost? • format of usernames, Should all usernames be surnames or first names or something else? What happens when there are clashes. • communication methods for users to contact Systems Administrators and vice versa, Should users come and visit the Systems Administrator in person, should they use the phone, is there a help desk? • methods of obtaining a new account, • methods for obtaining help, and • methods for complaining.
Depending on the site the Systems Administrator will take on responsibilities in any one or all of these components. At the very least he or she must be aware of them and make sure that they all exist. At most sites the Systems Administrator will be responsible for enforcing policy. It is very rare for a Systems Administrator to be charged with creating policy.
David Jones (11/30/94) Page 86 Systems Administration Theory Chapter 5
Exercises.
5-1. What factors do you think would affect the setting up of a sites policy and penalty components? e.g. the type of work being carried out the company might affect security procedures.
5.2. For what other issues might it be important to have policies on?
Reading.
Resource Materials Chapter 5.1.
Purpose.
Provide a look at a real life policy implementation including a number of key lessons on policy.
Records
It is not unusual for a Systems Administrator to spend two to three days trying to fix some problem that requires minor changes to obscure files hidden away in the dim, dark recesses of the file hierarchy. It is not unusual for a problem of this sort to crop up unexpectedly every six to twelve months.
What happens if the Systems Administrator didn't record the solution? Unless he or she is blessed with a photographic memory there is liable to be another two to three days lost trying to fix the problem.
Records of everything done to the system must be kept and they must be accessible at all times.
It is typical for a Systems Administrator and/or a computer site to maintain some type of log book. There is no set format to follow in keeping a log book.
Log Book Format
There are two basic types of log books that are used. • electronic, or Log information is stored using some type of program or by simply creating a file. • paper based. Some form of book or folder in which entries are written by hand.
Table 5.1. lists compares these two forms of log book.
David Jones (11/30/94) Page 87 Systems Administration Theory Chapter 5
Electronic Paper For Against For Against easy to update if the machine is less prone to harder to update and and search down there is no machine down search access time easy to include can be hard to can be carried can become messy command output include diagrams around and hard to read
Table 5-1. Comparison of electronic versus paper log books.
What Information should be Logged?
Anything that might be necessary to reconstruct the current state of the computing system should be stored. Examples of necessary information includes • copies of policy regarding usernames, directory structure etc, Your site might have a set way of assigning usernames or particular locations in which certain types of files need to be placed. • diagrams of the physical connections and layout of the machines and network, Any information required to reconstruct your system, for example CMOS settings on an IBM PC. • a copy of a full listing of the device directory, The /dev directory is liable to contain information specific to your machine. If this directory is trashed having a copy in your log book will enable you to reconstruct it. • copies of major configuration files, • daily modifications to configuration or other files, • lists of useful commands, and • solutions to common problems.
Example Log Book Layout
The type of information recorded will depend on your responsibilities and the capabilities of your site. There might be someone else who looks after the physical layout of the network leaving you to worry about your machine.
It is possible that a log book might be divided into separate sections. The sections might include • configuration information, Listings of the device directory, maps of network and cabling information, and any other static information about the system • policy and procedure, A section describing the policy and procedures of the particular machine (usernames, directory locations etc).
David Jones (11/30/94) Page 88 Systems Administration Theory Chapter 5
• useful commands, and A list of commands or hints that you've come across that are useful and you would like to remember. • daily modifications. The daily modifications made to the system in the normal course of events. The main reason to store this information is so that you have a record of what is being done to your system.
Each entry in a log book should contain information about time, date, reason for the change, and who made the change.
If you intend using a paper based log book then one suggestion is to use a ring binder. Using a ring binder you can add pages to various sections if they are starting to fill up.
User Liaison
Systems Administration is like being a policeman. The only time the users come to see you is when they are in trouble. If the computer system is working without a problem they won't give the Systems Administrator a second thought.
The following reading was first published in "The Australian Systems Administrator" (Vol 1, Issue 2, June/July 1994) the bi-monthly newsletter of the Systems Administrators Guild of Australia (SAGE-AU). It provides an example of how a real-life System Administrator handles user liaison.
Communicating with Users
Copyright Janet Jackson.
Next to balancing conflicting demands, communicating with users is the hardest part of my job. I tend to make a great effort for little gain, whereas in technical endeavours a little effort can produce a major, long-lasting improvement (for example, taking ten minutes to set up regular, automated scratch area cleanups has saved me hours of tedious work and the users a lot of frustration).
Also, with users there are emotions to take into account. It doesn't matter whether the computer respects you, but if the users respect you life is a lot easier.
My aim in communicating with users is to make life (my job and those of the users) easier by: • getting them to respect me (my judgment; my abilities; my integrity and professionalism). • teaching them all sorts of things, such as how to remove jobs from the printer queue; what they have to do to keep the systems secure; and when not to interrupt me with questions.
David Jones (11/30/94) Page 89 Systems Administration Theory Chapter 5
In this column I'm going to describe some of the communication vehicles I've tried, and how effective they've been for me. I'll start with those I've found least effective overall, and work my way up.
Probably the method most useless with the general user community is the policy statement. The typical user just isn't going to read it. However, it can be a good way of communicating with management. Drafting a good policy statement (based on discussions with everyone, but especially with them) shows you mean business and understand how your work fits into the organisation. It should cover the responsibilities of the systems administrator as well as those of the users.
Group meetings, whether of the users in general or of a committee of representatives, can help people -- again, especially senior people -- feel more confident that things are going OK, but aren't much use for disseminating information. If a meeting is run well you can have a productive discussion of major issues, but if run badly it is likely to turn into a gripe session.
Paper memos are to be avoided, because they encourage stiffness and formality. I use them only to answer other people's paper memos (which are usually complaints) and then only when I don't think the person will read it if I do it by email. Replying by email to a memo has the effect of saying "There's no need to be so formal".
There are a number of leading-the-horse-to-water methods, which only work if the user makes an effort. You can use electronic information services, such as bulletin boards, newsgroups, Gopher, or online manuals; and you can get together a library of printed manuals and books. If you provide easy access to high-quality information, the interested user can learn a lot. Unfortunately it's often the disinterested user that you really want to reach.
People often come to my office to ask me things. You'd think that face-to- face communication would work the best, but in this particular setting it doesn't because I am not comfortable. It's not so much that I resent interruptions -- it's that I don't have an office, only a desk. There's no room for a visitor's chair; to talk to anyone I have to swivel round and face backwards; and people make a habit of sneaking up on me. Hopefully, one day my campaign for proper accommodation will be successful, and it will be interesting to see how much difference it makes.
Talking on the phone is only good for emergencies. Someone is always interrupted; there's no body language; and you tend to forget half of what you wanted to say.
I write a column, "Computer Corner", in our staff newsletter. I sometimes write about issues (such as what I'm trying to achieve) and sometimes about technical tips. This column isn't as useful as I'd hoped. The first problem is
David Jones (11/30/94) Page 90 Systems Administration Theory Chapter 5
that there isn't room to say much, because the newsletter is short and a bit, shall we say, irregular. The second problem is that the rest of the newsletter tends to be kind of dull (lists of visitors; dry field-trip reports; the occasional births and deaths) so people aren't so eager to read it. When I pointed this out I was told that it is deliberately impersonal and non-funloving because some of the more senior readers are rather easily offended. Sigh.
Next on the scale are signs (on doors, noticeboards, etc) and electronic messages-of-the-day. People have a strong tendency to miss the former and ignore the latter. It may help to make them more interesting with graphics, pictures and human-interest items.
Seminars and workshops are worthwhile if you can get people to attend, but they're a lot of work. If not many turn up, you don't get much return on your investment. Students can sometimes be induced to attend by making it count towards their marks. In other situations, offering food, door prizes, alcohol, sex, drugs or rock-n-roll may help.
For explaining specific information (how to pick a good password; how Unix file permissions work) I've found paper handouts reasonably effective. Some users take them quite seriously, even filing them for later reference. Unfortunately, others toss them straight in the bin.
After about 3 months in my current job I emailed everyone a questionnaire, asking such things as what they used the systems for, what new services they would like to see, and how often they did backups. I offered a chocolate frog to each person who replied. The subject line "Apply here for your FREE chocolate frog" caused some of the more pokerfaced members of staff to delete the mail without reading it, but otherwise the response was surprisingly good. In hindsight, I guess the questionnaire generated more PR than information, although it did confirm my suspicion that most people did not back up their data even though they were supposed to.
For me, the second most effective communication vehicle is email. Email is as informal as a personal visit or phone call, but you can get in a lot more information. It is also asynchronous: no-one has to be interrupted, and you don't have to wait for people to be available.
I often use email broadcasts for notification -- to tell people about impending downtime, for example. Email is quick, convenient, and reaches people who are working offsite. It is also informal and I think people feel more at ease with it than they do with paper memos and printed signs.
1-to-1 email gives people a sense of personal service without much of the hassle that normally entails. At my site people can email problem reports and questions to a special address, "computerhelp". Our stated aim is to respond within 2 working days. We don't always make it. But it does give people a point of contact at all times, even after hours, and it means we get a few less interruptions.
David Jones (11/30/94) Page 91 Systems Administration Theory Chapter 5
You'd think all of that might be enough, but no. My boss said, "You need to communicate more with the users, to tell them about what you're doing". I agreed with him. So I now produce a fortnightly emailed bulletin. It is longer and more formal than a typical email message, with headings and a table of contents. Most of the information in it is positive -- new software that we've installed, and updates on our programme of systems improvements. I also include a brief greeting and a couple of witty quotations. Judging by the feedback I've received, this seems to be working remarkably well -- much better than the staff newsletter column.
The only thing that works better than email is personal visits where I am in their office, usually leaning over their screen showing them how to do something. Taking an interest in their work helps a lot. I find this easy where they are graphing the temperature of a lake in glorious colour, but more difficult where they are typing up letters. I don't do enough personal visiting, partly because I'm so busy and partly because I'm not keen on interrupting people. It usually happens only when they've asked a question that requires a "show me" approach.
A disadvantage of personal visits is that they help only one person at once, whereas with email you can reach all your users.
To sum up: in communicating with users, I aim to teach them things and get them to respect me. By sending email I can help the most people for the least effort, although personal visits have much more impact. There are other useful methods, such as policy statements, newsletters, handouts and seminars, but they may not reach the ones who need it most.
It's hard. Very hard. If you have any insights or ideas in this area, I'd love to hear them, and I'm sure the rest of the readers would too.
Observation
The major responsibility of a Systems Administrator is to ensure that the computer system keeps working. A major part of this responsibility is simply checking the system regularly to see that everything is okay. You don't want the users telling you that the machine is down, you should already know.
The sort of regular observations required include • resource usage, Are some of the systems resources (CPU, disk space, printers etc) being overused? Is a particular disk partition becoming full? This is especially true of partitions containing spool space for e-mail or news. Is a particular user using too much disk space? Is there a runaway process that is consuming all the CPU time?
David Jones (11/30/94) Page 92 Systems Administration Theory Chapter 5
• security, There should be a regular check for some obvious security holes. These will be covered in more detail in the security chapter but include checking for changes in the protections on various files and directories, accounts with no passwords set and repeated failed attempts to log in using a system account. • checking logs, and Every Unix system provides some sort of logging mechanism that records various happenings about the system including errors. These logs should be checked regularly to ensure that nothing out of the ordinary has occurred. • networks. Is the network still up? Are particular users abusing the network?
These tasks are repetitive and can be time consuming. This is especially so if you are in charge of tens or even hundreds of machines. So like many things in Systems Administration these tasks can, and should be automated. There are various collections of shell scripts and programs that perform these tasks.
Another important reason for making AND recording these observations is to use them in negotiations with management when it becomes time to lobby for better equipment. It is easier to convince management that you need another disk drive if you can show them reports that demonstrate that disk usage on the system has consistently run at 95% or higher.
We will examine the methods used on Unix system to do this logging and observation in chapter 16.
Conclusion
The topics covered in this chapter can be issues that are overlooked or treated as poorer cousins by some Systems Administrators (the author included). They are however probably the most important issues covered in this text. It is essential that they are done well. If not, the computer system can quickly degenerate into anarchy.
The importance of • keeping accurate, complete and up to date records, • having in place a set and well known collection of policy and penalties, • effective and friendly relationships and communication between users and the Systems Administrators, and • keeping regular and close observations on the machines cannot be over emphasised.
David Jones (11/30/94) Page 93 Systems Administration Theory Chapter 5
Review Question
5-1. Choose one of the following Unix commands. • mail • ls • rm
Print out the manual page for the command. Write a simple explanation of the command. Give to somebody who has never used Unix and see if they can follow the documentation.
5-2. Decide on a format for your logbook (something you may already have done). Collect all the information that is mentioned in this chapter and anything else you consider important.
David Jones (11/30/94) Page 94 Chapter 6 Users and Accounts
Chapter 6
USERS AND ACCOUNTS
I've been with the best and the worst in my life, and the secret is they're all people. -- Helen Drazenovich Berklich.
Objectives
By the end of this chapter you should • be aware of the processes involved in creating and removing user accounts from a Unix system, • have learnt about the various files that hold information about the users of a Unix system, and • be familiar with how the processes of adding and deleting a user account can be automated.
Introduction
It doesn't take long for a new Systems Administrator to start thinking of users as the enemy (sentiments like "The computer would run much better without any users" are not uncommon). However, users are the reason the computer system exists in the first place. The fact that users don't understand everything about the computer system and require assistance is the main reason why Systems Administrators are needed. Keeping this in mind sometimes helps with the frustration, anger and depression that some users can inspire in their Systems Administrator.
Reading. (optional, but funny)
Warning!! Some might find the language included here offensive. Exercise your own judgement about whether or not you should read this material.
Resource Materials Chapter 6.1
Purpose.
Provide a bit of comic relief and demonstrate the wrong type of attitude to have.
David Jones (11/30/94) Page 95 Chapter 6 Users and Accounts
Users present two major problem areas to Systems Administrators • the technical, and Modifying configuration files, changing permissions and setting up the system so that the users can actually log onto the machine. • the non-technical. Responding to pleas for help, criticism and demands, and informing users of changes in the system.
A UNIX Account
Every user that intends to make use of a UNIX machine must first have an account. A UNIX account is a collection of logical characteristics that specify who the user is, what the user is allowed to do and where the user is allowed to do it. Important concepts related to a UNIX account include • login names (also called a username), • passwords, • a user identifier number (referred to as a UID), • a group identifier number (referred to as a GID), • home directories, • mail aliases, • a mail file, and • a variety of startup files.
The following section introduces you to each of these concepts.
Login names
The account of every user is assigned a unique login (or user) name. The login name is only used by the users. The operating system uses the user identifier number (UID) to uniquely identify an account. The translation between UID and the username is carried out by the operating system using the /etc/passwd file (/etc/passwd is discussed later in this chapter).
On a small system the format of login names is generally not a problem since with a small user population it is unlikely that there will be duplicates. However on a large site with hundreds or thousands of users and multiple computers assigning a login name can be a major problem. With a higher number of users it is likely that you may get a number of jonesds.
Some guidelines for the creation of usernames include • login names should be unique, This includes across different machines at the same site. A login name should identify the same person and only one person on every machine on the site. This can be very hard to achieve at a site with a large user population. • they should be no more than eight characters long, Unix ignores or disallows login names that are longer.
David Jones (11/30/94) Page 96 Chapter 6 Users and Accounts
• login names are usually all lower case letters, Numbers and upper case letters can be used. Login names that are all upper case should be avoided as UNIX can assume this to mean your terminal doesn't recognise lower case letters and every piece of text subsequently sent to your display is in uppercase. • login names should be easy to remember, A random sequence of letters and numbers is hard to remember and so the user will be continually have to ask the Systems Administrator “what’s my username?” • do not use pseudonyms or nicknames, and One of the purposes of login names is to enable other users to identify who is on the system. Users may not know about nicknames. • they should follow a fixed system format. There should be a specified system for creating a username. Some combination of first name, last name and initials is usually the best. Setting a policy allows you to automate the procedure of adding new users. It also makes it easy for other users to work out what the username for a person might be.
Passwords
An account’s password is the key that lets someone in to use the account. A password should be a secret collection of characters known only by the owner of the account.
Poor choice of passwords is the single biggest security hole on any multi- user computer system. As a Systems Administrator you should follow a strict set of guidelines for passwords (after all if someone can break the root account’s password, your system is going bye, bye). In addition you should promote the use of these guidelines amongst your users. (Chapter 14 on security discusses passwords and breaking them in more detail).
An example set of password guidelines might include • use combinations of upper and lower case characters, numbers and punctuation characters, • don't use random combinations of characters if they break the following two rules, • be easy to remember, If a user forgets their password they can't use the system and guess who they come and see. Also the user SHOULD NOT have to write their password down. • be quick to type, One of the easiest and most used methods for breaking into a system is simply watching someone type in their password. It is harder to do if the password is typed in quickly. • a password should be at least 6 characters long, The shorter a password is the easier it is to break. Some systems will not allow passwords shorter than a specified length.
David Jones (11/30/94) Page 97 Chapter 6 Users and Accounts
• a password should not be any longer than 8 to 10 characters, Most systems will look as if they are accepting longer passwords but they simply ignore the extra characters. The actual size is system specific but between eight and ten characters is generally the limit. • do not use words from ANY language, Passwords that are words can be cracked (see Chapter 14). • do not use combinations of just words and numbers, Passwords like hello1 are just as easy to crack. • use combinations of words separated by punctuation characters or acronyms of uncommon phrases/song lines, They should be easy to remember but hard to crack. e.g. a.big1p • change passwords regularly, and Not too often that you forget which password is currently set. • never reuse passwords. On a UNIX system a user changes the password for an account using the passwd command. A normal passwd command performs the following steps • asks for the old password, To make sure you are the user who owns the account. • asks for the new password, and • asks for the new password again. Just to check that you didn’t make a typing mistake.
Many systems today come with a pro-active passwd program. A pro- active passwd program has a set of rules built into it. If a new password breaks any of those rules it will refuse to change the password (more on proactive password programs in Chapter 14). The User Id Number (UID)
Every account on a Unix system has a unique user or login name that is used by users to identify that account. The operating system does not use this name to identify the account. Instead each account must be assigned a unique user identifier number (UID) when it is created. The UID is used by the operating system to identify the account.
In choosing a UID for a new user there are a number of considerations to take into account including • choose a UID number between 100 and 32767 (or 60000), Numbers between 0 and 99 are usually reserved by some systems for use by system accounts. Different systems will have different possible maximum values for UID numbers. • UIDs for a user should be the same across machines, Some network systems (e.g. NFS) require that users have the same UID across all machines in the network. Otherwise they will not work properly.
David Jones (11/30/94) Page 98 Chapter 6 Users and Accounts
• you may not want to reuse a number. Not a hard and fast rule. Every file is owned by a particular user id. Problems arise where a user has left and a new user has been assigned the UID of the old user. What happens when you restore from backups some files that were created by the old user? The file thinks it is owned by the user with a particular UID. The new user will now own those files even though the username has changed.
The Mail File
When someone sends mail to a specific user that mail message has to be stored somewhere so that the recipient can read the message later on. Under Unix each user is assigned a mail file that is located in a specific directory. When they receive a mail message it is appended onto the end of that mail file.
The mail files for all the users on a machine are kept in the same directory. The location of this directory can change depending on the operating system being used. Common locations are /usr/spool/mail /var/spool/mail /usr/mail /var/mail.
Aliases
If you send an e-mail message that cannot be delivered (e.g. wrong address) typically the postmaster for your machine will receive the message. There is usually no account called postmaster. Postmaster is a mail alias.
When the mail delivery program gets mail for postmaster it will not be able to find a matching username. It will then look up a specific file, usually /etc/aliases or /etc/mail/names. This file will typically have an entry postmaster: root This tells the delivery program that anything addressed postmaster should actually be delivered to the user root.
Some companies will have a set policy for e-mail aliases for all staff. This means that when you add a new user you also have to update the aliases file.
For example: The Central Queensland University has aliases set up for all staff. An e-mail sent to [email protected] will be delivered to that staff member’s real mail address. In my case the alias is [email protected]. The main on-campus mail host has an aliases file that translates this alias into my actual e- mail address [email protected].
David Jones (11/30/94) Page 99 Chapter 6 Users and Accounts
Exercise.
6.1. Create an alias for yourself that uses your full name separated by a full stop. In my case the alias would be david.jones. Test your alias. (You will need to send some mail. Typical mail programs include elm, pine and mail.
Home Directories
Every user must be assigned a home directory. When the user logs in it is this home directory that becomes the current directory. Typically all user home directories are stored under the one directory (e.g. /home or /usr/users) and are given names that match the username for the account.
For example: A user jonesd might have a home directory /home/jonesd
In some instances it might be decided to further divide users by placing users from different categories into different sub-directories.
For example: All staff accounts may go under /home/staff while students are placed under /home/students. These separate directories may even be on separate partitions.
The Startup Files
When a user logs into a machine they are presented with a specific type of environment. Most of that environment is created by special files (called startup files or dot files) that modify the default environment of the system.
All these files work by modifying the execution of specific commands. Many of them .cshrc, .profile and /etc/profile (for example) work by modifying the behaviour of the login shell. These shell startup files are responsible for • setting up command aliases, Some shells allow the specification of aliases for various commands. A common command alias is dir, usually set to mean the same as ls -l. • setting terminal types, and • search paths.
Other dot files modify the execution of other programs including editors, mail and news readers and many others. Table 6.1. summarises some of these files.
David Jones (11/30/94) Page 100 Chapter 6 Users and Accounts
Filename Command Explanation $HOME/.cshrc /bin/csh Executed first when logging in for C shell. Executed every time C shell started. Used to set things needed for each C shell $HOME/.login /bin/csh Executed after .cshrc when logging in for C shell. /etc/profile /bin/sh Global startup file for Bourne shell. Used to set machine standard settings $HOME/.profile /bin/sh Located in users home directory. Contains user specific settings. $HOME/.logout /bin/csh executed just prior to the system logging the user out $HOME/.forward mail Used to forward mail to another address that is contained in the file $HOME/.exrc vi used to set options for vi $HOME/.mailrc mail used to set mail options, aliases etc. $HOME is a shell variable that indicates a users home or login directory. Table 6.1. Sample Startup Files.
It is good policy to provide new users with a standard set of startup files that they can modify if they wish. The easiest way to do this is to create a skeleton directory (usually /etc/skel) that contains all the standard startup files. When a new account is created the standard startup files are copied from the skeleton directory into the new accounts home directory.
Exercise.
6.2. Create a .logout and a .login or .profile that maintains a file account.usage in your home directory. The contents of the file account.usage should look something like this Logged in Thu May 26 11:53:37 EST 1994 Logged out Thu May 26 11:58:37 EST 1994 Logged in Fri May 27 11:00:35 EST 1994
Special Accounts
All UNIX systems come with a number of special accounts. These accounts already exist on a system and are there for a specific purpose. Typically these accounts will all have UIDs that are less than 100 and are used to perform a variety of administrative duties. Table 6.2. lists some of the special accounts that may exist on a machine.
David Jones (11/30/94) Page 101 Chapter 6 Users and Accounts
Username UID Purpose root 0 The super user account used by the Systems Administrator. Can do anything they want, has no restrictions. daemon 1 The owner of many of the system daemons. bin 2 The owner of many of the standard executable programs.
Table 6.2. Special User Accounts.
The passwords for these special system accounts must be kept secret or in some cases some system accounts should have invalid passwords (therefore no-one can login as that user).
Account Configuration Files
The UNIX operating system maintains a number of configuration files that store the information discussed above. This section examines each of these files and their format.
Table 6.3. lists the configuration files examined and their purpose. Not all systems will have the /etc/shadow file. In some cases the shadow file will exist but its filename will be different.
File Purpose /etc/passwd password file, contains a list of users and most of their information /etc/shadow shadow password file, contains encrypted password for users on more secure/modern systems /etc/group group file, contains a list of the systems groups and their members
Table 6.3. Account Configuration Files.
The /etc/passwd File
/etc/passwd is the central account configuration file. It is a text file in which each line contains information about one account. Each line is divided into seven fields by colons (sometimes referred to as a colon delimited).
Table 6.4. summarises each of the fields in the /etc/passwd file.
David Jones (11/30/94) Page 102 Chapter 6 Users and Accounts
Field Name Purpose login name user's login name * encrypted encrypted version of the user's password password UID number the unique User Identification number used by Unix to identify the user default GID number the Group Identifier number used to group users comment field no strict purpose, usually contains full name and phone number of user home directory the directory into which the user is placed on login login shell/program the program that is run for the user as they login (usually a shell) * on systems with the /etc/shadow file there will be no encrypted password Table 6.4. Fields of the /etc/passwd file.
The /etc/shadow File
The /etc/passwd file has to be able to be read by every user on the system. This is because many of the programs (ls for example) need to be able to access the passwd file to perform the translation from UID to username.
The problem with everyone being able to read the /etc/passwd file is that everyone can also read the encrypted passwords contained there. The method used to encrypt the passwords is supposedly a one way encryption algorithm. This means that people trying to break into a system cannot break into the system by running a program that decrypts the passwords.
However what they can do is obtain an on-line dictionary of words and encrypt the whole dictionary. This provides them with a list of encrypted words that can be compared with the encrypted passwords from the /etc/passwd file. If an account has a simple word as the password then the encrypted password will match one of the encrypted words from the dictionary. With a carefully chosen dictionary between 10-20% of passwords can be broken on any machine (discussed in Chapter 14).
To get around this problem recent versions of the Unix operating system will not store the encrypted password in the /etc/passwd file. It has been moved to the /etc/shadow file. The shadow file can only be read by the super user.
The shadow file can also go under a number of different names depending on the version of UNIX being used.
David Jones (11/30/94) Page 103 Chapter 6 Users and Accounts
This file is a recent addition to Unix systems and provides a more secure method for storing the user's encrypted password. Typically the shadow file consists of one line per user containing the encrypted password and some additional information. Figure 6.1 lists some of the fields encountered in a typical shadow file.
• username • encrypted password • the date the password was last changed • minimum number of days before the password can be changed again • maximum number of days before the password must be changed • number of days until age warning is sent to user • number of days of inactivity before account should be removed • absolute date on which the password will expire • an undefined flag
Figure 6.1. Example Fields of an /etc/shadow file. Most systems that support the shadow password file also support password aging. Password aging allows the Systems Administrator to add a number of additional restrictions to passwords including • a minimum time period before a password can be changed, This prevents users from changing passwords every day. • a maximum number of days a password can be used, and Forces users to change passwords after a set time period. Prevents them from using the same password forever. • a number of days before the account will automatically expire. For example: An example shadow file taken from a Linux machine (the encrypted passwords have been modified) ba012:FjmJeOIdfeKcE:8903:0:99999:7:0:0:0 ba015:OsR54f8BVxRTI:8905:0:99999:7:0:0:0 ba014:NPd32s/JvzCRg:8905:0:99999:7:0:0:0
Exercises.
6.3. Does your system support shadow passwords? If so in which file are the encrypted passwords stored?
Hint: try looking in the /etc directory, try the man pages for passwd.
6.4. Does your system support some form of password aging?
David Jones (11/30/94) Page 104 Chapter 6 Users and Accounts
The /etc/group File
A group is a logical collection of users. Users with similar needs or characteristics are usually placed into groups. A group is a collection of user accounts that can be given special permissions. Groups are often used to restrict access to certain files and programs to everyone but those within a certain collection of users.
For example: On the Central Queensland University UNIX machine jasper only certain users are allowed to have full Internet access. All these users belong to the group angels. Any program that provides Internet access has as the group owner the group angels and is owned by the root user. Only the group owner and the owner may execute these files.
The /etc/group file maintains a list of the current groups for the system and the users that belong to each group. /etc/group is like /etc/passwd in that it is colon delimited text file. Figure 6.2 summarises the fields of /etc/group.
• group name A unique name for the group. • encrypted password (this is never used) • group identifying number (GID) GID is similar to the UID only it is used for groups. • list of user login names who belong to the group, the list is separated by commas.
Figure 6.2. Fields of the /etc/group file.
For example: friend::101:davido,david staff::102:carterb
Creating a New Account
Adding a user is a fairly mechanical task and can be automated via shell scripts with little difficulty. In fact most machines come with scripts that do it. The steps involved in creating a new account include • modifying the /etc/passwd file, • set an initial password, • modify the /etc/group file, • create the home directory, • create the new user’s mail file, • create any startup files required for the user, and • test that the addition has worked.
David Jones (11/30/94) Page 105 Chapter 6 Users and Accounts
Prerequisite Information
Before adding a user to your system you will need to know the following information.
Information My Site username format user's personal information user's group location of user's home directory location of mail boxes user's startup shell location of skeleton startup files
Table 6.5. Requisite Knowledge for Adding a User.
Modify the /etc/passwd file
For every new user an entry has to be added to the /etc/passwd file. There are a variety of methods by which this is accomplished including • appending information to the /etc/passwd file, This is a method that is often used. However it can be unsafe and it is generally not a good idea to use it. • the command vipw, or Some systems (usually BSD based) provide this command that invokes an editor so the Sys Admin can edit the passwd file safely. The command performs some additional steps that ensures the editing is performed consistently. • a dedicated adduser program. Many systems provide a program (the name will change from system to system) that accepts a number of command-line parameters that specify the values for each of the fields of the passwd file.
Set an initial password
NEVER LEAVE THE PASSWORD FIELD BLANK.
If you are not going to set a password for a user put a * in the password field of /etc/passwd or the /etc/shadow file. On most systems the * character is considered an invalid password and it prevents anyone from using that account.
David Jones (11/30/94) Page 106 Chapter 6 Users and Accounts
If a password is to be set for the account then the passwd command must be used. The user should be forced to change immediately any password set by the Systems Administrator.
Modify the /etc/group file
Add the user's login name to the correct group entry. If necessary you may have to create a new group entry.
Make the home directory
Not only must the home directory be created but the permissions have to be set correctly so that the user can access the directory.
For example: mkdir home_directory chown login_name home_directory chgrp group_name home_directory
The permissions of a home directory should be set such that • user owner should be the owner of the account, • group owner should be the primary group the account belongs to (the one listed in /etc/passwd), • at the very least the owner of the directory should have rwx, and • the group and other permissions should be set as restrictive as possible (possibly to none at all).
Creating the Mail File.
One of the responsibilities that must be carried out when creating a new user is to create an empty mail file in the correct location. It is important that the ownership and the permissions on the mail file are set correctly.
The permissions of a mail file should allow • the user needs to be able to read and write the file so the mail can be read and deleted (the user should own the mail file), • the group owner should be group mail and it should be able to read and write the file The programs that deliver mail to the file are owned by the group mail. It needs to be able to write to the file to deliver the mail. • no-one else should have any access to the file No-one wants anyone else peeking at their private mail.
For example: Creating a mail file for the user david touch /usr/spool/mail/david chown david /usr/spool/mail/david chgrp mail /usr/spool/mail/david chmod 660 /usr/spool/mail/david
David Jones (11/30/94) Page 107 Chapter 6 Users and Accounts
Exercises.
6.5. Write a shell function create_mail that accepts the login name of the new user, creates their mail file and sets the appropriate permissions. Remember that not every site is going to have the same location for mail files. Define a shell variable that holds the location of the mail folders for the current system.
Copy any startup files
Place in the user's home directory the appropriate startup files. Once again make sure to use chown so that the user owns the files.
Verify the addition
This is an optional procedure to check that the new account works. Login as the user and examine the current working directory and the various permissions.
The super user does not need the user's password to login as that user, they can use the su command.
For example: su - david Simulates a full login as the user david. This actually forces all the startup files to be executed. If this command is executed as the root user no password will be prompted for.
Inform the user
Tell the user their account is ready, what their password is and what the various local customs are. It is best to do this in person.
Deleting an Account
Deleting an account involves reversing the steps carried out when the account was created. It is a destructive process and whenever something destructive is performed care must always be taken.
Change the password field
The account should be automatically disabled so as to prevent someone using the account. One method is to insert into the password field (in /etc/passwd or /etc/shadow) an invalid character. An example would is the * character. * EXPIRED 1/1/93* is one format that can be used.
David Jones (11/30/94) Page 108 Chapter 6 Users and Accounts
Another method is to simply remove the entry from the /etc/passwd and /etc/shadow files all together.
Back up the user's home directory and contents
It is possible that this user may have some files that need to be used by other people. So back everything up, just in case.
Remove the user's files
All the files owned by the account should be removed from whereever they are in the file hierarchy. It is unlikely for a user to own files that are located outside of their home directory (except for the mail file). However it is a good idea to search for them. The command find / -user UID - print will display full path names of any file owned by the user with the UID UID.
Depending on the system it may be a good idea to take a backup copy of the user’s files before removing them.
Mail for Old Users
In most cases once you’ve deleted a user’s mail box any mail sent to that user will bounce (that is be returned to the sender as undeliverable). In some cases this might be the desirable outcome.
However what happens if the user has moved onto a new machine and would like their mail forwarded? One solution is to create a mail alias for the user that points to their new address.
Automating the Procedure
One of the first processes that many Systems Administrators automate is the creation of new accounts. The following shell script is used on the CQ-PAN system. CQ-PAN is a research project within the Department of Maths & Computing at the Central Queensland University that is supplying Internet access to the general public. It is included here to provide an example of a shell script used to create new accounts.
David Jones (11/30/94) Page 109 Chapter 6 Users and Accounts
#!/bin/bash # FILE: /usr/local/adm/bin/makeeuser # FORMAT: makeeuser [username] # [username "comment"] # [username "comment" home_dir] # username is the new username # "comment" is the GCOS information for the account # home_dir is the home directory of the account # AUTHOR: David Jones (based on a script by Chris Parry) # PURPOSE: Creates a new user account. # The script does very little checking as it relies # on the supplied useradd command to do much of it for us. # HISTORY: makesuser Created 27/03/94 #makeeuser started 2/6/94 home_dir='/home/os' email='/tmp/asemail' init_group='os' login_shell='/bin/bash' admin_address='david@bertha' # addres to send notification welcome_mail='/usr/local/extern/welcome' trap "" 1 2 3 15 if [ `whoami` != "root" ]; then echo "Only root may add users to the system."; exit 1 fi
# Get the login name of the user. if [ "$#" -lt 1 ]; then echo -n 'Enter login name: ' read username junk else username="$1" fi # Convert extern name to lower case. username=`echo $username | tr "[A-Z]" "[a-z]"`
# Get the account description field. if [ "$#" -le 1 ]; then echo -n 'Please input the Users full name==> ' read comment echo -n 'Please enter the Users student number==> ' read student_num else comment="$2" fi
# Get the location of the home directory of the account. home_dir=$home_dir/$username
# Create the new extern user account. # Add a new extern account and the first extern administration account. /usr/sbin/useradd -c"$comment" -d$home_dir -m -k/etc/skel -g$init_group -s$login_shell $username
# Make sure the home directory of the account exists. if [ -d "$home_dir" ]; then # Create a mail file for the account. touch /usr/spool/mail/$username chmod 660 /usr/spool/mail/$username chown $username /usr/spool/mail/$username chgrp mail /usr/spool/mail/$username # Set the password life of the account. passwd -x 365 -n 0 $username # Record the account creation details into a temporary account to be # mailed to administration when completed processing. fi if [ $# -eq 0 ] then passwd $username fi
David Jones (11/30/94) Page 110 Chapter 6 Users and Accounts
Allocating Super User Privilege
There are two problems that the CQ-PAN script does not address • how do you set a password without having to type it in yourself, and If you are creating 300 new user accounts you don't want to have to think of new passwords for each user and then enter them yourself. The best solution would be to have a program generate and set them for you. The problem here is that the passwd command requires input from the user and redirecting input from another program won’t work. This means you can’t automate on the passwd command so you can't do it easily from a shell script. • how do you let an offsider handle the menial task of adding a user without giving them root access? Creating new accounts is a fairly mundane job especially when it is automated. A Systems Administrator is typically better occupied doing something more complex. It would be nice to be able to let someone with less responsibility do this task. But how do you let them do this without giving them the root password?
Both these problems have already been solved. Table 6.6. describes to public domain utilities that solve these problems.
Program Name Purpose Location sudo Allocate specific root ftp.connect.com.au privileges on a user by user /pub/security/sudo.v1.2. basis tar.Z expect provide the ability to use ftp.connect.com.au interactive commands from a /pub/misc/expect.tar.Z shell script without user intervention
Table 6.6. Sudo and expect.
expect and sudo are NOT standard Unix commands. They are programs that have been written by Systems Administrators to solve these problems and then made available to the general public free of charge. The locations specified in Table 6.6. are FTP sites and are only useful to you if you can gain access to the Internet.
Using existing software instead of writing your own is a good example of another Systems Administration maxim, "Never reinvent the wheel". If someone else has already solved a problem it is stupid for you to waste your time trying to solve it again.
David Jones (11/30/94) Page 111 Chapter 6 Users and Accounts
Exercises.
6.6. Write a shell function get_uid that returns the next free UID number. (Some systems by default have a user with a very high UID. This means you can’t just sort the passwd file on UID and get the last entry.)
6.7. Write a shell function make_username that accepts a user's firstname and lastname as parameters and then sets a variable username. username is to contain a username for this user using the format, first seven letters of the surname followed by the first initial. For example: David Jones turns into jonesd Georgina Pickering turns into pickerig
Conclusions
Hopefully by now you are familiar with the process, commands and configuration files associated with adding and deleting a user from a Unix system. You should also be aware of mail aliases, password aging, the ownership and location of mail spool files and the various start up files executed when a user logs in.
Review Questions
6.1. Write an adduser script that uses some of the scripts written for this chapter. Make sure it makes use of any commands that may be specific to your system.
David Jones (11/30/94) Page 112 File Systems and Disk Drives Chapter 7
Chapter 7
FILE SYSTEMS AND DISK DRIVES
Ask, and it shall be given you; seek, and ye shall find; knock an it shall be opened unto you. -- Matthew 2:5-7
Objectives
By the end of this chapter you will • understand how Unix handles the storage and retrieval of information onto disks, • be familiar with the terms logical disk, partition, blocks, superblocks and i- nodes, • have used or be aware of the purpose of the commands mkfs, fsck, mount, umount, sync, ncheck, ls -i, • be able to explain the different meanings applied to the word file system, • have an overview of the structure and components of the system Unix uses to store information onto disk drives, and • be able to create, mount and repair file systems.
Introduction
The primary job of any operating system is to store information. Unix (like most operating systems) makes use of a structured upside down, tree-like hierarchy of files and directories. A Systems Administrator must know a number of things about how this information is stored including • where and why are certain files are located, and Most Unix operating systems have a number of standard locations for specific files and directories. The first section of this chapter will introduce you to some of the more common locations. • how to manipulate, create and repair the file hierarchy and file system. The Systems Administrator has to know more than where to find things. If the disk drive becomes corrupted it is the responsibility of the Systems Administrator to attempt to recover the data from that drive. To accomplish this, sometimes difficult task, the Systems Administrator needs to know about the data structures and algorithms used by the operating system to store information onto the disk.
By now you should have an inkling that the unified directory hierarchy you see as a Unix user is a logical abstraction. The illusion of all the directories and files of a Unix machine being all linked together under the one directory hierarchy is created by a part of the operating system referred to as the file
David Jones (11/30/94) Page 113 File Systems and Disk Drives Chapter 7
system. In reality the files and directories that are seen as being joined together can in fact be located on separate partitions, drives and even on separate machines.
Diagram 7-1 demonstrates this. The directory hierarchy as seen by the user is all joined together. In fact the /usr/bin directory and the / directory are both stored on different local partitions (partition is explained below) and the /home directory is actually stored on a drive connected to a remote machine. The file system is responsible from hiding these facts from the user and the commands the used by the users.
An Example Unix File System.
/
usr bin etc home
bin jonesd balsys Partition 1 Local Machine
Partition 2 Local Machine Partition 5 Remote Machine mounted over a network
Diagram 7.1. Logical and Physical File System Organisation.
A file system is a collection of algorithms and data structures that are responsible for translating and structuring data stored on disk into the standard format used by UNIX commands.
The File Hierarchy
Each user of a Unix system is provided their own home directory in which they can create, delete and modify files and directories at will. Most users will have little need to move outside their home directory.
The system administrator on the other hand will have to manipulate files from throughout the entire directory hierarchy. To do this efficiently you will need a good working knowledge of the directory structure. One of the responsibilities of a System Administrator is management of the directory hierarchy that includes • controlling what files go where, and It is important that there are system guidelines for the placement of different types of files. • controlling which users can access certain parts of the file hierarchy, Users should be restricted to the parts of the file hierarchy to which they must have acccess. Access to other sections of the directory hierarchy should be prevented. This is achieved by using file permissions.
David Jones (11/30/94) Page 114 File Systems and Disk Drives Chapter 7
Table 7.1 provides a summary of the more important, relevant or useful directories you may find on a Unix machine. These will change in some small way from machine to machine. It is by no means an exhaustive list.
Path Purpose / the root directory, the basis for the whole structure /bin the binary directory, contains all the essential commands needed when the system first boots /dev the device directory, contains special files associated with hardware devices (explained later in this chapter) /home location of user’s home directories, usually kept in one /usr/users directory, /home is the more common /etc contains machine-specific configuration files and system administration databases, some systems place administration executable programs here /sbin some systems take the administration executable programs /usr/sbin out of /etc and place them here /lost+found used by fsck to save disconnected files and directories (each partition will have its own lost+found) /tmp temporary directory used to store temporary files /usr usually where almost everything else goes /usr/local the local directory, used to store all local additions to the file system /usr/adm administration, logfiles and record keeping /usr/ucb used on some systems to store commands that were originally developed at the University of California at Berkeley /usr/bin more executable programs, those not strictly needed when the system first boots /usr/etc some administrative commands on some BSD systems /usr/lib library files /lib /usr/mail mailboxes for users, could be under /usr/spool/mail or a number of other locations /usr/man directory containing manual pages /usr/spool the spool directory, used by mail, print and other systems to /var/spool temporarily store information that is being spooled /usr/src source code for utilities and the system /proc mount point of proc file system which maintains information about running processes, SVR4 /sbin SVR4, executables used in the booting process and in manual recovery from failure /var on some systems the contents of /usr/spool are moved here
Table 7.1. Special Purpose Directories.
David Jones (11/30/94) Page 115 File Systems and Disk Drives Chapter 7
/bin and /usr/bin
Both these directories are used to store executable (binary) programs and commands. The distinction between the two directories is the assumption that /usr/bin will not always be present on the root partition.
The next chapter will introduce the concept of single user mode. Occasionally the Systems Administrator will have to bring up a UNIX machine in single user mode. Single user mode is basically a maintenance mode where only the root account is able to login and the Systems Administrator may perform various maintenance tasks. (A Systems Administrator should always ensure that there is some way of bringing a system up in single user mode.)
When the system comes up in single user mode only the root partition will be mounted (the root partition is the partition that contains the / directory). This may mean that /usr/bin is not present (as it would be in the system depicted in Diagram 7-1).
The /bin directory should contain all the commands that are necessary to bring a system up into single user mode and once there perform the necessary tasks. /usr/bin is used to hold commands that are not strictly essential in single user mode. For example it would be rare for the command yacc to be needed in single user modem so it will not be in /bin.
/usr/local
As a system is being used it will become necessary to add various data files, source programs and executable commands. Where should this information be kept?
One option is to add these files throughout the entire file hierarchy. Executable programs under /usr/bin, configuration files under /etc, library files under /usr/lib, and so on. However this causes some problems, especially when the system must be upgraded.
When a UNIX operating system is upgraded what usually happens is the vendor will ship a new tape or CD-ROM containing the new, updated operating system. The Systems Administrator will install the new operating system onto the machine drives and the system will be ready to go.
The problem is that installing a new operating system will generally overwrite any previous information, including any local programs and files the Systems Administrator has installed.
The solution to this problem is to backup any local files before installing the new operating system and once it is installed restoring the local files.
David Jones (11/30/94) Page 116 File Systems and Disk Drives Chapter 7
Backing up local files is very difficult if they are spread throughout the file hierarchy.
/usr/local is used to store all local files under one directory. On most system /usr/local will contain sub-directories including (amongst others) • doc, Used to store local documentation. • bin, Containing additional local executable commands and programs. • src, Where the source code to local programs and commands are stored. • lib and include. Used for local library and include files.
Exercises.
7.1. Explore the directory hierarchy of your machine. Attempt to identify each of the locations listed in Table 7.1. on your machine. This is the type of information that would be useful in your log book.
7.2. Examine your systems’ file hierarchy for the /usr/local, /bin and /usr/bin directories. What do they contain?
The Components of the UNIX File I/O System
The following sections aim to provide you with an overview and definitiion of all the different components that work together to make the Unix file I/O system work. Diagram 7.3 provides a representation of how all the different components fit together.
The Physical Disk
The information ends up (or starts life depending on how you look at it) as binary information stored on the surface of a disk drive. The smallest accessible unit of measure on a disk is a block. A disk block is usually between 512-4096 bytes depending on the device.
A modern disk drive will consist of a number of circular surfaces. Each surface is divided by • a collection of concentric circles, the area between each concentric circle are tracks, and • a collection of lines across the diameter that divide the surface into sectors.
David Jones (11/30/94) Page 117 File Systems and Disk Drives Chapter 7
The collection of tracks on all surfaces that are the same distance from the centre form a cylinder.
re a d / writ e se ctor blo ck
tr a ck
Diagram 7.2. The Physical Structure of a Disk Drive.
Disk Controller
A disk controller is an electronic device that tells the disk drive what to do, which block to read from where, and when. A single disk controller will typically control a number of separate disk drives.
Most Unix boxes will rely on SCSI (pronounced scuzzy) disk controllers, though a few PC based Unices will support IDE controllers.
Device Drivers
Every single physical device has a different command language that it understands including disk controllers. Different types of disk controllers and other devices expect to be told to do things in different ways.
The UNIX operating system defines a standard method that is used to perform I/O regardless of the type of device that is being used. All commands and other parts of the operating system will use this standard method to perform I/O.
However because all the physical I/O devices talk a different language something must perform the translation between how UNIX performs I/O and the commands that I/O devices understand.
The part of the operating system that performs this task are the device drivers.
Every physical device connected to a UNIX machine must have an appropriate device driver installed into the kernel of the operating system before the operating system can use the device.
It is possible for one device driver to service a number of different devices, provided that the devices are all of the same type.
David Jones (11/30/94) Page 118 File Systems and Disk Drives Chapter 7
Device Files
A device file is basically an entry point into a device driver and are usually located under the /dev directory. Device files are used by many parts of the operating system to send or retrieve information from a device.
Device files can even be used in conjunction with the I/O redirection performed by shells to send the output of commands directly to devices.
For example: Send the a long directory listing of the current directory to the terminal used by the Systems Administrator. ls -l > /dev/console /dev/console is explained in Table 7.2.
The entry point used by a device file is made up of • a major device number, and The major device number is used to indicate the particular device driver that the device file is pointing to. • a minor device number The minor device number indicates which entrance into the device driver that this particular device file will use. A device driver is liable to have many entrances each used to access a different device.
A major difference between device files and normal files is that when ls -l is used on a device file the size of the file is not shown. Instead the ls command displays the major and minor device numbers of the device file.
For example: ls -l /dev/null crw-rw-rw- 1 root root 1, 3 Aug 6 00:34 /dev/null This device has a major device number of 1 and a minor device number of 3.
There are usually two types of device file, characterised by the method that is used to transfer information • character device file, and Information is sent to/from the device one character at a time. Examples of a character device include serial ports, printers, terminals and modems. • block device file. Information is sent to/from the device a block at a time. One block is usually 512 or 1024 bytes. An example of a block device is a disk drive.
David Jones (11/30/94) Page 119 File Systems and Disk Drives Chapter 7
Device File Usual Purpose /dev/console the system console, the terminal used by the Systems Administrator /dev/ttyp0 a pseudo terminal, used by logins over a network /dev/tty0 a dumb terminal connected via serial /dev/ser a serial port /dev/par a parallel port /dev/null the null device (ignores everything sent to it)
Table 7.2. Example Special Device Files.
It is important that the permissions on a device file are set correctly. Incorrectly set permissions will result in users being able to read or send information straight to the a physical device. In many cases (hard disk drives for example) this is not a good idea.
When a Device File is Deleted
It is often suggested that a physical copy of the output of the ls -l /dev command should be made so that a record of the major and minor device numbers associated with every device file can be kept.
The reason being is that if the entries in /dev become corrupted or are deleted then big parts of the system will stop working. It is important that the device files can be reconstructed.
A device file is created using the mknod command. The mknod command takes as parameters • the name of the device file to create, • whether the device file is a character device or a block device, • the major device number, and • the minor device number.
The name of a device file is not as important as the major and minor device numbers. Two device files can have completely different names and yet they will still point to the same device if the major and minor number are identical.
For example: The /dev/null device file is called the null device. Anything sent to the null device is simply thrown away. The following sequence of commands first removes and then recreates the /dev/null device file. ls -l /dev/null crw-rw-rw- 1 root root 1, 3 Aug 6 00:34 /dev/null rm /dev/null cat /etc/passwd > /dev/null ls -l /dev/null
David Jones (11/30/94) Page 120 File Systems and Disk Drives Chapter 7
crw-rw-rw- 1 root root 10356 Aug 6 05:22 /dev/null rm /dev/null mknod /dev/null c 1 3 ls -l /dev/null crw-rw-rw- 1 root root 1, 3 Aug 6 00:34 /dev/null
Exercises.
7.3. The tty command displays the device file associated with the terminal the user is currently logged in at. Use the ls command to discover whether it is a character device or a block device.
7.4. Examine the permissions of the terminal device file. Which users can write and read to the device file? Change the permissions so another user can write to your terminal. Have them redirect the output of a command onto your terminal.
7.5. Make a hard copy of a long listing of your systems device files. (You will use it for the next question)
7.6. Log onto two different terminals or consoles as the root user. Determine which device files are associated with one of the terminals by using the tty command. Delete that device file. How does it affect the session? Log out on that terminal. What happens? Why? (use mknod to recreate the device before leaving).
File Systems
There are many different methods that can be used to structure the way in which information is stored onto a disk drive. MS-DOS uses one method, OS/2 uses another and different versions of Unix each use different methods. Under UNIX the part of the operating system responsible for structuring and understanding the information that is stored on disk is called the file system.
Some operating systems (MS-DOS for example) can only handle one file system. The UNIX operating system can manage many different file systems at the same time. Each of the partitions in Diagram 7.1. could be using completely different file system types.
This is possible because the code associated with each file system translates how the information is structured on the disk into a standard Unix programming interface. This standard programming interface is used by the majority of UNIX commands to access the services provided by the file system kernel code.
This means that under most versions of UNIX operating system, files can be copied (using the normal UNIX command cp) from a partition formatted using MS-DOS onto a partition formatted using a UNIX file system.
David Jones (11/30/94) Page 121 File Systems and Disk Drives Chapter 7
Different file systems have different characteristics. Table 7.3 lists some of the different file systems you'll come across and some of their obvious differences.
In Table 7.3 the label ext2 file system refers to the collection of code and data used to implement a file system called ext2 (extended file system 2, that is the major file system used on Linux machines).
File System Description Characteristics MS-DOS standard MS-DOS file limit of 8 character filenames systems with 3 letter extensions ext2 standard Linux file allows 255 character system filenames s5 System V file system allows 14 character filenames
Table 7.3. Brief comparison of different file systems.
(Most versions of Unix supply a file system capable of reading MS-DOS floppies and in some cases hard disk drives. Linux is one of these.)
System Calls
The standard programming interface mentioned in the previous section consists of a number of system calls. A system call is a special type of function called used by programs to access services provided by the operating system. These services can include • opening, closing, reading and writing from files, • changing the priorities or killing a process, and • allocating sections of memory and others.
You should remember from a previous chapter that there is a section of the manual pages set aside for system calls.
Exercises.
7.7. There is a system call called kill. Examine the manual page for this command. (HINT: Remember there is also a user command called kill.)
David Jones (11/30/94) Page 122 File Systems and Disk Drives Chapter 7
User Programs
Libraries User Level Kernel Level
System Call Interface
Process
File Subsystem Control
Subsystem
Buffer Cache
Character Block Device Drivers
Hardware Control
Kernel Level Hardware Level
Hardware
Diagram 7.3. An Overview of File I/O.
User Commands & Programs
The normal user interface with the file system consists of commands or programs like cp, ls and rm. A Systems Administrator will use additional commands like fsck, mount and mkfs (explained later in this chapter).
These commands make use of system calls that define the standard interface that all file systems must fulfil. By using the standard programming interface the commands will work with files stored on different file systems. The interface is used to hide the implementation detail.
File System Data Structures
The following section provides a basic introduction the internal data structures used by a standard UNIX file system. The actual implementation of most modern Unix file systems is actually much more complex.
David Jones (11/30/94) Page 123 File Systems and Disk Drives Chapter 7
The following structures are based on the s5 file system. The s5 files system was the old UNIX file system standard developed as part of SysV UNIX. Most modern UNIX file systems are based on the Fast File System (FFS) developed at the University of California at Berkeley. The FFS extended the basic structure of the s5 file system to make it more efficient.
Blocks The smallest unit of information that can be read from or written to a disk is a block. Some file systems specify a logical block size that may be different from the physical block size of the disk drive.
A block can only be used by one file. That means if you have a block size of 1024 bytes and a file that is 1030 bytes long. That file will need to use two blocks.
The Boot Block
The first block in all partitions are reserved to act as a boot block. The boot block on the boot partition is used to hold the operating system boot strap (or loader) program. This small program (it needs to fit on one disk block) has enough code to locate the operating system kernel and to place it into memory so that it can start executing.
The SuperBlock
The second block in any partition will tend to be the superblock. The superblock is basically the master index for the file system. It is used to maintain information about the entire file system.
Information kept in the superblock includes • the size of the file system (partition), It is likely that there will be multiple partitions on one disk. This records the size of partition that contains the superblock. • address of the first data block, • the number of free blocks, • a list of the free blocks, When new files are created the file system code must locate free blocks that can be allocated to the new file. • the time and date the superblock was last updated, and • the file system's name.
The I-Node
To read the information from a file the programs need to know where on the disk the blocks that hold the information in the file are stored. Under Unix it is the i-node that points the way to the data.
David Jones (11/30/94) Page 124 File Systems and Disk Drives Chapter 7
The information kept on the i-node also includes • the type of file, Is it a normal file, a directory, a device file or any of the other special types recognised by the operating system? • access permissions, Which users can perform what operations? • the number of links, The number of hard links making using of the i-node (links are explained in more detail below). • user and group owner identifiers, The UID and GID of the user and group owners of the file. • size of the file in bytes, • time the file was last used, • time the file was last modified, • time the i-node was last changed, and The i-node is changed whenever any of this information is changed about the file. • the addresses of thirteen disk blocks. The file system must be able to find all the data that is contained by a file. UNIX uses these disk block addresses to point to the data associated with this i-node.
Every file in a file system must have an i-node. Note that the i-node does not store the name of the file. The name of the file is stored in the directory. A directory in Unix is simply a file that contains filenames and pointers to the i-nodes of the files within the directory.
The number of i-nodes in a partition is set and must be decided upon when creating the file system on that partition. Once this decision is made it limits the number of files that can be created on that file system (since every file must have an i-node). A general rule is that about ten percent of a file system will be taken up by i-nodes.
However, if you know a particular partition is going to be used for a few very large files you may decide to modify the number of i-nodes (the same applies if the file system will contain very small files.)
The concept of an i-node is central to the Unix file system. Diagram 7-4 provides a representation of how an i-node points to all the data blocks related to a particular file and a summary of the other information the i-node stores about a file.
An i-node contains a number of pointers that, in some way, point to the disk blocks owned by that file. Some of these pointers point directly at data blocks. Others point to indirect block.
David Jones (11/30/94) Page 125 File Systems and Disk Drives Chapter 7
I-Node.
Other Info
Block Addresses
direct
single indirect double indirect triple indirect
Legend
Data block, contains actual file data
Indirect blocks, contain about 256 block pointers
Diagram 7.4. I-Node structure.
An indirect block, instead of containing data, contain the pointers to other blocks. A single indirect block can contain pointers to at least another 256 blocks (how many depends on the particular version of UNIX being used and how it is configured).
A UNIX i-node can point to • single, The pointers in a single indirect block point to actual data blocks. • double, and The pointers in a double indirect block point to a single indirect block. Each of the single indirect blocks then point to actual data blocks • triple indirect blocks. The pointers in a triple indirect block point to double indirect blocks which then point to single indirect blocks......
Depending on size and other factors a UNIX i-node can point to enough data blocks for a file in excess of two gigabytes in length.
David Jones (11/30/94) Page 126 File Systems and Disk Drives Chapter 7
Exercises.
7.8. If a single data block contains 512 bytes of information and the i- node contains 10 direct blocks, 1 indirect block, a double indirect block and a triple indirect block and each indirect block contains pointers to another 256 blocks, what is the size of the largest possible file?
I-Nodes and Links
The -i option of the ls command can be used to view the i-node numbers that are associated with individual files. Some system will also provide a program called ncheck that can be used to display the name of the file associated with a particular i-node.
Executing the command cp /etc/passwd $HOME/passwd will cause • a new i-node to be created, • a new entry (pointing to the new i-node) to be added to the directory $HOME for the file passwd, • each of the data blocks of the file /etc/passwd to be read and copied to new data blocks, and • the block addresses of the new i-node are updated to point to the new data blocks.
Once the cp command has completed you have two completely separate files with no relation between them. Executing a command on one of the files will not effect the other file in any way.
Hard Links
The command ln /etc/passwd $HOME/passwd creates a hard link called $HOME/passwd that points to /etc/passwd file. The process carried out as a result of this command is • a new entry is added to the $HOME directory, and • the new entry is set to point to the same i-node used by /etc/passwd.
With a hard link the two files share the same i-node and thus the same data blocks. Any change made to one file also appears to be made to the other.
A hard link can only be used to join two files that are within the same partition. This is because the i-node pointers contained in directory entries can only point to i-nodes that reside within the same partition.
David Jones (11/30/94) Page 127 File Systems and Disk Drives Chapter 7
Soft Links
The command ln -s /etc/passwd $HOME/passwd creates a soft or symbolic link. The process involved in creating a soft link includes • creating a new i-node and setting the file system to soft link, • adding a new entry to the $HOME directory to point to the i-node, • creating new data blocks for the new file pointed to by the i-node, and • placing into the data blocks the name of the file the soft link is pointing to (in this case /etc/passwd).
When a program tries to access the contents of the symbolic link the file system notices that the file type is a symbolic link and reads the name of the file the link points to. The file system then carries out the requested command on this new file.
For example: The difference between a soft and a hard link can be demonstrated by a number of simple commands. bash$ cat > hello hello there
Exercises.
7.9. Perform the steps outline in the above example.
7.10. Change the permissions on the file hello. What happens to the permissions for the file hello2? Try the same procedure on the file hello2. Why is this so?
7.11. Repeat the procedure from exercise 7-11 for the files hello and hello3. What happens? Why is it different?
7.12. What are the i-node numbers for the files in the root directory? Why do you think the i-nodes have these numbers?
David Jones (11/30/94) Page 128 File Systems and Disk Drives Chapter 7
Device and Partition Names
Most UNIX systems use a standard format for names given to device files associated with disk drives and partitions. The format of these device filenames depends on the version of UNIX being used. Table 7.4. summaries some of the different formats being used.
Every disk partition will usually have two device files associated with it. One device file will be a character device file and the other will be a block device file. The majority of commands will use the block device file to communicate with the disk. The character file is used by commands to create a new file system (format) or fix an old one.
Unix Version Partition Explanation Example Name System V cCdDsS C - controller number c0d0s2 DsS D - disk number c3d0s0 S - partition number 0s0 BSD (Linux) ccDp cc - two letter name hd0a D - drive number rd0g p - partition letter hd0b
Table 7-4. A Comparison of Disk Device Naming Schemes.
When a system is supplied from the vendors there will be a number of device files associated with each disk drive including • a drive device file, and The drive device file is used to refer to the entire disk drive. • a number of partition device files. The partition device files are used to access the various partitions into which a disk may be divided.
System manufacturers do not know how many partitions may be created when the system is shipped. Generally they will set aside a number of device files for use by the one drive. There is no need for all of the supplied device files to be used.
David Jones (11/30/94) Page 129 File Systems and Disk Drives Chapter 7
For example: The following example is taken from a Linux machine. /dev/hda is the device file associated with the entire disk. The other device files are associated with particular partitions. brw-rw---- 1 root disk 3, 0 Jan 29 1994 /dev/hda brw-rw---- 1 root disk 3, 1 Jan 29 1994 /dev/hda1 brw-rw---- 1 root disk 3, 2 Jan 29 1994 /dev/hda2 brw-rw---- 1 root disk 3, 3 Jan 29 1994 /dev/hda3 brw-rw---- 1 root disk 3, 4 Jan 29 1994 /dev/hda4 brw-rw---- 1 root disk 3, 5 Jan 29 1994 /dev/hda5 brw-rw---- 1 root disk 3, 6 Jan 29 1994 /dev/hda6 brw-rw---- 1 root disk 3, 7 Jan 29 1994 /dev/hda7 brw-rw---- 1 root disk 3, 8 Jan 29 1994 /dev/hda8 brw-rw---- 1 root disk 3, 9 Jan 29 1994 /dev/hda9 Notice that the major device number is the same for all the device files.
Examine the permissions and ownership of the device files for the disk drive and its partitions? What would happen if the permissions were brw-rw-rw instead of brw-rw----?
By default if users wish to access the information on a disk they must use the system calls supplied by the kernel. These system calls then access the file systems algorithms within the kernel to access the information on the disk. The file system also implements the file protection mechanism. It is the file system that checks whether or not a particular user can access the file they are asking for.
If anyone could read/write direct to the device files then the file system could be bypassed. This in turns bypasses the file protection scheme. With a little bit of work a program could be written to obtain and evaluate all the information on the disk.
Always make sure that the permissions on a device file are set correctly.
Operations on File Systems
There are a number of basic operations that a Systems Administrator can carry out on a file system including (some of the following commands will be system specific) • create a new file system (the mkfs and newfs commands), Creating a new file system initialises all the data structures talked about in the next section. This operation makes the file system ready to be used to store information. • mount a file system (the mount command), Mounting a file system connects it into the entire file hierarchy as seen by the user. Each of the different partitions from Diagram 7.1. would have been previously mounted. • unmount a file system (the umount command), Once a particular file system is finished with it must be removed from the file hierarchy.
David Jones (11/30/94) Page 130 File Systems and Disk Drives Chapter 7
• monitor a file system (the find, fstat commands), In some cases you will wish to monitor the state and contents of a file system. • check the integrity and repair a file system (the fsck and fsdb commands), and In some instances the file system will be corrupted and need to be fixed. • flush a file system’s buffers (the sync command). Unix doesn't automatically write all changes made to a file straight to disk. It buffers some of that information in memory. This means that if the power gets turned off unexpectedly the information in memory will be lost and it is possible that the file system will be in a corrupt state. This is the reason why you should never just turn off a Unix computer (if you can possibly avoid it).
Creating a New File System
A brand new 5 Gigabyte drive has arrived waiting to be connected to the system. The process to be undertaken includes the following steps • low level formatting of the disk, • partitioning of the disk, • creating a file system on each of the partitions, and • mount the file system.
Low Level Format
Most modern disks are pre-formatted by the manufacturer so a low level format may not be necessary. If a low level format must be performed then the command will be machine and operating system dependent.
Related to this step is identifying any bad blocks that may exist on the disk surface. There are no standard Unix commands to format a drive or check for bad blocks.
For example: Linux provides a command fdformat to perform a low level format of floppy disks. (examine the man page)
Partition the drive
It is rare for a disk drive to be used as one large partition. In most cases a disk drive will be divided into a number of partitions. The Systems Administrator must make a number of decisions about partitioning the disk drive • how many partitions will there be? • how big will the partitions be? • what type of file system will the partition be formatted as? and • what portion of the file hierarchy will be stored on each partition?
David Jones (11/30/94) Page 131 File Systems and Disk Drives Chapter 7
Under UNIX there is no set command that is used to divide a disk drive into partitions. Each system will implement its own command. Examine the system documentation to discover the appropriate command. apropos partition is always a good place to start.
Create the file system
Once the disk has been partitioned each partition must have the data structures (superblock, i-nodes etc) mentioned in a previous section initialised. The mkfs or the newfs commands are used to initialise a partition depending on the system.
Mount the file system
Mounting the file system will usually entail two steps • create or choose a mount point, and • mount the directory.
The new file system needs to be mounted either manually using the mount command or automatically when the system boots up.
Reasons to Partition a Drive
The smallest drives that can be purchased today for a Unix system would be at least 100 to 200 Mbytes in size. It is possible to use the entire disk space as one file system. This is generally not a good idea. Reasons for partitioning a disk are explained below.
Separation
Different sections of the disk hierarchy have different characteristics that make keeping them on separate partitions a good idea. These characteristics include • rate of change, The users’ home directories and those used to store incoming mail or news are likely to change almost continually, where as others like the root directory, /bin, /usr/bin and others are unlikely to change at all. • size and number of files, and See the discussion above about the number of i-nodes being fixed. • the possibility of runaway processes. Some partitions like tmp and spool directories are very likely to have runaway processes chew up lots of disk space. It a file system fills up it ceases working. Keeping these directories in separate partitions reduces the problems this can cause.
David Jones (11/30/94) Page 132 File Systems and Disk Drives Chapter 7
In addition the software on most machines can put into one of two categories • system software, and Software distributed by the operating system vendor. • local software. Software installed by the owners of the machine.
When an operating system is updated the install script will usually just replace everything in certain directories. It is suggested that local software be stored in a separate directory and possibly partition so that it can be kept safe.
Size of the Backup Media
Backups are considerably easier if you can achieve at least one partition per backup media (tape, optical disk). If you are using 150Mb tapes and have a 500Mb partition it will be more difficult to backup.
Backups should be as easy as possible to do.
Performance
Certain sections of the whole file system will be busier than others. By distributing the busy sections amongst different partitions on different drives you can spread the load around and improve the performance of the system.
Mounting and Unmounting a File System
Look back to Diagram 7.1. The full file specification of the adm directory is /usr/adm but this is only true because partition 2 is mounted in the /usr directory. The /usr directory is referred to as the mount point for partition 2 (before mounting adm the /usr directory was empty).
A mount point must be an empty directory and is the place in which a new partition joins the rest of the directory hierarchy. If a new partition is mounted in a directory that is not empty one of the following may happen • if there is already another mounted file system using the mount point the mount command will report an error, • if there are any normal files already there they will disappear (they will reappear when the partition is unmounted), or • you may mount new file systems in directories that are part of other mounted file systems (as long as they satisfy the above criteria).
There are two basic methods that can be used to mount a file system • file system configuration files, or Either the /etc/vfstab or /etc/fstab file is examined during the operating system’s boot process. • the mount command.
David Jones (11/30/94) Page 133 File Systems and Disk Drives Chapter 7
When mounting a file system the following information is required • the type of the file system, Remember UNIX can support many different types of file system. The mount command must be told the type of file system this new partition is so that it can use the correct algorithms. • the device file associated with the partition, • the directory name of the mount point, and • a variety of options that control how the file system is mounted.
File System Configuration Files
All UNIX operating systems have a configuration file that controls which partitions are automatically mounted when the system boots. BSD based systems use the file /etc/fstab while SysV based machines use the file /etc/vfstab.
Both files are text based with one line per partition separated into a number of columns. The major difference between the two files (apart from the name) is the format of the columns.
For example: The following is an example of an /etc/fstab file taken from a Linux machine. partition name mount point file system type options /dev/hdb1 swap swap defaults /dev/hda2 / ext2 defaults /dev/hdb2 /home ext2 defaults,usrquota none /proc proc defaults
The mount Command
While the fstab and vfstab files are used to automatically mount a file system at boot time, the mount command is used to mount a file system without having to reboot the computer.
The format of the mount command is similar to mount -t type device_file mount_point
The -t switch is used to indicate the type of file system. The actual switch used on some systems will be different. type is a unique identifier that indicates the type of file system on the partition. Most systems will also take other command line arguments that can be used to specify further options.
David Jones (11/30/94) Page 134 File Systems and Disk Drives Chapter 7
For example: The following mount command is taken from a Linux box mount -t ext2 /dev/hda2 /bin It results in the ext2 partition associated with the device /dev/hda2 to be mounted under the /bin directory.
Without any parameters the mount command will display the currently mounted file systems.
For example: pol# mount /dev/hda3 on / type ext2 (rw) /dev/hda1 on /home type ext2 (rw) none on /proc type proc (rw)
The umount Command
The umount command is used to remove a file system from the file heirarchy. The umount command can take as a parameter either the device file of the partition or the mount point.
For example: To unmount the file system mounted in the example from the above section, either of the following commands can be used. umount /dev/hda1 umount /bin
If the umount command doesn't work, reasons may include • the user performing the umount doesn't have write permission on the device file on which the file system resides (typically only the root user can unmount a file system), or • the file system is being used.
The file system is being used if • the current working directory of any user is within the file system, • there is another mounted file system mounted within the file system, or • there is a process running that is accessing a part of the file system.
Exercises.
7.13. Discover what partitions are currently mounted on your machine. Which partitions are automatically mounted on startup?
7.14. Use the manual pages to discover the format of your system's disk configuration files. Discover the different types of file system supported by your system.
David Jones (11/30/94) Page 135 File Systems and Disk Drives Chapter 7
Exercises.
7.15. Create a new file system and mount it somewhere in your directory hierarchy. (Simplest method might be to use a floppy disk if your system supports it.)
7.16. All versions of Unix provide a directory /mnt. This directory is set aside to be used as a temporary mount point for file systems. If your system supports it (Linux does) mount an MS-DOS floppy under the /mnt directory (make sure it has something on it). Refer to the manual pages for mount for information how to do it.
Copy a file of at least a couple of hundred kilobytes onto the floppy. Once it is copied do nothing but observe the drive lights (wait for between 30 seconds and 2 minutes). What do you observe? What do you think causes this?
Perform the previous task again but this time instead of waiting type the command sync. What happens? Why?
Repeat the process again but this time unmount the floppy straight away. What happens? Why?
Checking a File System
There will come a time in every System Administrator's working life when it is discovered that a file system has become corrupt. When this occurs it is best to follow the advice of Douglas Adams and DON'T PANIC!!! Panic at such a time invariably causes more complex problems to arise.
Unix provides a very powerful command that can usually be used to recover most damaged file systems. Even if it doesn't work, as a professional Systems Administrator you should have recent backups of the file system anyway.
fsck stands for file system check. The type of problems fsck can generally repair include • one data block being pointed to by several i-nodes (one block owned by multiple files), • blocks marked as free but actually being used, • blocks marked as used but actually free, • incorrect link counts in the i-node , • differences between the size field of the i-node and the number of data blocks actually pointed to by the i-node, • illegal blocks within files, • i-nodes that contain information but don't occur in a directory entry (these are placed into the lost+found directory in the root directory of the file system, and
David Jones (11/30/94) Page 136 File Systems and Disk Drives Chapter 7
• directory entries that point to illegal or unallocated i-nodes.
Guidelines on Using Fsck
• Where possible fsck should be run on a unmounted file system using the character device file and not the block device file. • If you receive any particularly drastic messages from fsck first thing you should do when it is finished is to run fsck again. • Write down any device file, i-node and block numbers fsck reports. Using the ncheck command allows you to identify which file belongs to an i-node. This information can be used to discover whether you have to recover something from your backups. • If fsck reports it Can't stat, open, read... check to see that the disk is connected properly, not write protected and that you have permission on the character and block device files. • If fsck asks if it can SALVAGE? FIX? CONTINUE? RECONNECT? or ADJUST? it is usually safe to let it. • If fsck asks to REMOVE? or CLEAR? be more picky. Be ready to recover material from your backups.
Exercises.
7.17. Select a floppy disk you can afford to lose. Mount the floppy disk and copy to it a large file. After the first phase of writing to the disk has finished, remove the disk (don't unmount it just remove it). Now attempt to unmount the floppy as if it were still in the drive (there will be errors). Once all the hubbub has died down attempt to remount the floppy. What happens? Try to fix the floppy.
Conclusions
Hopefully by now you have an inkling of how the Unix operating system stores and retrieves information from disk drives. You should be aware • of the separate components that make up the Unix file sub system • of the various commands that are useful when manipulating, creating and using file systems • the format of configuration files for file systems
David Jones (11/30/94) Page 137 File Systems and Disk Drives Chapter 7
Review Questions
7.1. What is the purpose of the following directories • /etc • /usr/spool/mail • /home • / • /bin
7.2. Define the following terms with reference to this chapter • block, • cylinder, • file system, • i-node, • device file, • major and minor device number.
7.3. What are the two methods that can be used to mount a file system.
7.4. What are the steps used to prepare a new disk drive for use on a UNIX system?
7.5. What is the difference between a hard and soft link?
David Jones (11/30/94) Page 138 Backups Chapter 8
Chapter 8
BACKUPS
Like most of those who study history, he (Napoleon III) learned from the mistakes of the past how to make new ones. A.J.P. Taylor.
Objectives
This chapter aims to • discuss the components of a backup strategy, • introduce you to the characteristics of a good backup strategy, • discuss the factors that affect a backup strategy, • introduce the Unix commands tar dump restore dd gzip compress, and • offer suggestions on creating a backup strategy for your system.
Introduction
This is THE MOST IMPORTANT responsibility of the System Administrator. Backups MUST be made of all the data on the system. It is inevitable that equipment will fail and that users will "accidentally" delete files. There should be a safety net so that important information can be recovered.
It isn't just users who accidentally delete files
A friend of mine who was once the administrator of a Unix machine (and shall remain nameless, but is now a respected Academic at CQU) committed one of the great no-nos of Unix Administration.
Early on in his career he was carefully removing numerous old files for some obscure reason when he entered commands resembling the following (he was logged in as root when doing this).
cd / usr/user/panea notice the mistake rm -r *
The first command contained a typing mistake (the extra space) that means that instead of being in the directory /usr/user/panea he was now in the / directory. The second command says delete everything in the current directory and any directories below it. Result: a great many files removed.
David Jones (11/30/94) Page 139 Backups Chapter 8
The moral of this story is that everyone makes mistakes. Root users, normal users, hardware and software all make mistakes, break down or have faults. This means you must keep backups of any system.
The Components of Backups
There are basically three components to a backup strategy the • scheduler • transport, and This is the program that is used to move the data to be backed up from the disk on which it resides to the media on which it will be saved. There are a number of different programs available on a Unix system and you will be introduced to most of the more popular ones in this chapter. • media The actual physical device on which the backup is made. Media might be floppy disks (very painful on anything but a trival system), magnetic tapes (ranging from 80Mb up to 8 Gigabytes and beyond), write once optical disks or even another hard disk drive.
Scheduler
The scheduler is the object that decides when backups should be performed and how much should be backed up. The scheduler could be the root user or a program, usually cron (discussed in a later chapter).
The amount of information that the scheduler backs up can have the following categories • full backups, All the information on the entire system is backed up. This is the safest type but also the most expensive in machine and operator time and the amount of media required. • partial backups, or Only the busier and more important file systems are backed up. One example of a partial backup might include configuration files (like /etc/passwd), user home directories and the mail and news spool directories. The reasoning is that these files change the most and are the most important to keep a track of. In most instances this can still take substantial resources to perform. • incremental backups. Only those files that have been modified since the last backup are backed up. This method requires less resources but a large amount of incremental backups make it more difficult to locate the version of a particular file you may desire.
David Jones (11/30/94) Page 140 Backups Chapter 8
Transport
The transport is a program that is responsible for placing the backed up data onto the media. There are quite a number of different programs that can be used as transports. Some of the standard UNIX transport programs are examined later in this chapter.
There are two basic mechanisms that are used by transport programs to obtain the information from the disk • image, and • through the file system.
Image Transports
An image transport program bypasses the file system and reads the information straight off the disk using the raw device file. To do this the transport program needs to understand how the information is structured on the disk. This means that transport programs are linked very closely to exact file systems since different file system structure information differently.
Once read off the disk the data is written byte by byte from disk onto tape. This method generally means that backups are usually quicker than the file by file method. However restoration of individual files generally takes much more time.
Transport programs that use the method include dd, volcopy and dump.
File by File
Commands performing backups using this method use the system calls provided by the operating system to read the information. Since almost any UNIX system uses the same system calls a transport program that uses the file by file method (and the data it saves) is more portable.
File by file backups generally take more time but it is generally easier to restore individual files. Commands that use this method include tar and cpio.
Media
Backups are usually made to tape based media. There are different types of tape. Tape media can differ in • physical size and shape, and • amount of information that can be stored. From a 100Mb up to 8Gb.
Different types of media can also be more reliable and efficient. The most common type of backup media used today are 4 millimetre DAT tapes.
David Jones (11/30/94) Page 141 Backups Chapter 8
Characteristics of a Good Backup Strategy
Backup strategies change from site to site. What works on one machine may not be possible on another. There is no standard backup strategy. There are however a number of characteristics that need to be considered including • ease of use, • time efficiency, • easy to restore files, • ability to verify backups, • tolerant of faulty media, and • portable to a range of machines.
Easy To Use
If backups are easy to use, you will use them. AUTOMATE!! It should be as easy as placing a tape in a drive, typing a command and waiting for it to complete. The reading for this chapter contains a story about what happens if backups are seen to be too much work.
Time Efficiency
Obtain a balance to minimize the amount of operator, real and CPU time taken to carry out the backup and to restore files. The typical tradeoff is that a quick backup implies a longer time to restore files. Keep in mind that you will in general perform more backups than restores.
On some large sites particular backup strategies fail because there aren’t enough hours in a day. Backups scheduled to occur every 24 hours fail because the previous backup still hasn't finished.
Easy to Restore Files
The reason for doing backups is so you can get information back. You will have to be able to restore information ranging from a single file to an entire file system. You need to know on which media the required file is and you need to be able to get to it quickly.
This means that you will need to maintain a table of contents and label media carefully.
Verified Backups
YOU MUST VERIFY YOUR BACKUPS. The safest method is once the backup is complete, read the information back from the media and compare it with the information stored on the disk. If it isn’t the same then the backup is not correct.
David Jones (11/30/94) Page 142 Backups Chapter 8
Well that is a nice theory but it rarely works in practice. This method is only valid if the information on the disk hasn't changed since the backup started. This means the file system cannot be used by users while a backup is being performed or during the verification. Keeping a file system unused for this amount of time is not often an option.
Other quicker methods include • restoring selected files from the start, middle and end of the backup, If these particular files are retrieved correctly the assumption is that all of the files are valid. • create a table of contents during the backup, afterwards read the contents of the tape and compare the two.
These methods also do not always work. Under some conditions and with some commands the two methods will not guarantee that your backup is correct.
Fault Tolerant
A backup strategy should be able to handle • faults in the media, and • physical dangers.
There are situations where it is important that • there exist at least two copies of full backups of a system, and • that at least one set should be stored at another site.
Consider the following situation.
A site has one set of full backups stored on tapes. They are currently performing another full backup of the system onto the same tapes. What happens when the backup system is happily churning away when it gets about halfway and crashes (the power goes off, the tape drive fails etc). This could result in the both the tape and the disk drive being corrupted. Always maintain duplicate copies of full backups.
The Pauls ice-cream factory in Brisbane is located right on the river bank. During the early 1970s Brisbane suffered a major flood. Pauls’ computer room was in the basement of their factory and was completely washed out. All the backups were kept in the computer room.
Portable
There may be situations where the data stored on backups must be retrieved onto a different type of machine. The ability for backups to be portable to different types of machine is often an important characteristic.
David Jones (11/30/94) Page 143 Backups Chapter 8
For example: The computer currently being used by a company is the last in its line. The manufacturer is bankrupt and no-one else uses the machine. Due to unforeseen circumstances the machine burns to the ground. The Systems Administrator has recent backups available and they contain essential data for this business. How are the backups to be used to reconstruct the system?
Considerations for Developing a Backup Strategy
Apart from the above characteristics, factors that may affect the type of backup strategy implemented will include • the available commands The characteristics of the available commands limit what can be done. • available hardware The capacity of the backup media to be used also limits how backups are performed. In particular how much information can the media hold? • maximum expected size of file systems The amount of information required to be backed up and whether or not the combination of the available software and hardware can handle it. A suggestion is that individual file systems should never contain more information than can fit easily onto the backup media. • importance of the data The more important the data is the more important that it be backed up regularly and safely. • level of data modification The more data being created and modified the more often it should be backed up. For example the directories /bin and /usr/bin will hardly ever change so they rarely need backing up. On the other hand directories under /home are likely to change drastically every day.
Reading.
Resource Materials Chapter 8.1
Purpose.
To provide an overview of backup systems.
Commands
As with most things the different versions of Unix provide a plethora of commands that could possibly act as the transport in a backup system. The following table provides a summary of the characteristics of the more common programs that are used for this purpose.
David Jones (11/30/94) Page 144 Backups Chapter 8
Command Availability Characteristics dump/restore BSD systems image backup, allows multiple volumes, not included on most AT&T systems tar almost all file by file, most versions do not support systems multiple volumes, intolerant of errors cpio AT&T systems file by file, can support multiple volumes some versions don't,
Table 8.1. The Different Backup Commands.
There are a number of other public domain and commercial backup utilities available which are not listed here.
dump and restore
A favourite amongst many Systems Administrators, dump is used to perform backups and restore is used to retrieve information from the backups.
These programs are of BSD Unix origin and have not made the jump across to SysV systems. Most SysV systems do not come with dump and restore. The main reason is since dump and restore bypass the file system they must know how the particular file system is structured. So you simply can't recompile a version of dump from one machine onto another (unless they use the same file system structure).
Many recent versions of systems based on SVR4 (the latest version of System V UNIX) come with versions of dump and restore.
dump
The command line format for dump is
dump [ options [ arguments ] ] file system dump [ options [ arguments ] ] filename
Arguments must appear after all options and must appear in a set order.
dump is generally used to backup an entire partition (file system). If given a list of filenames dump will back up the individual files.
dump works on the concept of levels (it uses 9 levels). A dump level of 0 means that all files will be backed up. A dump level of 1...9 means that all files that have changed since the last dump of a lower level will be backed up.
David Jones (11/30/94) Page 145 Backups Chapter 8
Options Purpose 0-9 dump level a archive-file archive-file will be a table of contents of the archive. f dump-file specify the file (usually a device file) to write the dump to, a - specifies standard output u update the dump record (/etc/dumpdates) v after writing each volume rewind the tape and verify. The file system must not be used during dump or the verification.
Table 8.2. Arguments for dump
There are other options. Refer to the manual page for the system for more information.
Linux does not currently support the dump or restore commands.
For example: dump 0dsbfu 54000 6000 126 /dev/rst2 /usr full backup of /usr file system on a 2.3 Gig 8mm tape connected to device rst2 The numbers here are special information about the tape drive the backup is being written on.
The restore/rrestore Command
The purpose of the restore command is to extract files archived using the dump command. restore provides the ability to extract single individual files, directories and their contents and even an entire file system.
restore -irRtx [ modifiers ] [ filenames ]
The restore command has an interactive mode where commands like ls etc can be used to search through the backup.
David Jones (11/30/94) Page 146 Backups Chapter 8
Arguments Purpose i interactive, directory information is read from the tape after which you can browse through the directory hierarchy and select files to be extracted. r restore the entire tape. Should only be used to restore an entire file system or to restore an incremental tape after a full level 0 restore. t table of contents, if no filename provided root directory is listed including all subdirectories (unless the h modifier is in effect) x extract named files. If a directory is specified it and all its sub- directories are extracted.
Table 8.3. Arguments for the restore Command.
Modifiers Purpose a archive-file use an archive file to search for a file's location. Convert contents of the dump tape to the new file system format d turn on debugging h prevent hierarchical restoration of sub- directories v verbose mode f dump-file specify dump-file to use, - refers to standard input s n skip to the nth dump file on the tape
Table 8.4. Argument modifiers for the restore Command.
The tar Command
tar is a general purpose command used for archiving files. It takes multiple files and directories and combines them into one large file. By default the resulting file is written to a default device (usually a tape drive). However the resulting file can be placed onto a disk drive.
tar -function[modifier] device [files]
When using tar each individual file stored in the final archive is preceded by a header that contains approximately 512 bytes of information. Also the end of the file is always padded so that it occurs on an even block boundary. For this reason every file added into the tape archive has on average an extra .75Kb of padding per file.
David Jones (11/30/94) Page 147 Backups Chapter 8
Arguments Purpose function A single letter specifying what should be done, values listed in Table 8.6 modifier Letters that modify the action of the specified function, values listed in Table 8.7 files The names of the files and directories to be restored or archived. If it is a directory then EVERYTHING in that directory is restored or archived
Table 8.5. Arguments to tar.
Function Purpose c create a new tape, do not write after last file r replace, the named files are written onto the end of the tape t table, information about specified files is listed, similar in output to the command ls -l, if no files specified all files listed u * update, named files are added to the tape if they are not already there or they have been modified since being previously written x extract, named files restored from the tape, if the named file matches a directory all the contents are extracted recursively * the u function can be very slow Table 8.6. Values of the function argument for tar.
Modifier Purpose v verbose, tar reports what it is doing and to what w tar prints the action to be taken, the name of the file and waits for user confirmation f file, causes the device parameter to be treated as a file m modify, tells tar not to restore the modification times as they were archived but instead to use the time of extraction o ownership, use the UID and GID of the user running tar not those stored on the tape
Table 8.7. Values of the modifier argument for tar.
If the f modifier is used it must be the last modifier used. Also tar is an example of a UNIX command where the - character is not required to specify modifiers.
David Jones (11/30/94) Page 148 Backups Chapter 8
For example: tar -xvf temp.tar tar xvf temp.tar extract all the contents of the tar file temp.tar tar -xf temp.tar hello.dat extract the file hello.dat from the tar file temp.tar tar -cv /dev/rmt0 /home archive all the contents of the /home directory onto tape, overwriting whatever is there
Exercises
8.1. Create a file called temp.dat under a directory tmp that is within your home directory. Use tar to create an archive containing the contents of your home directory.
8.2. Delete the $HOME/tmp/temp.dat created in the previous question. Extract the copy of the file that is stored in the tape archive (the term tape archive is used to refer to a file created by tar) created in the previous question.
The dd Command
The man page for dd lists its purpose as being "copy and convert data". Basically dd takes input from one source and sends it to a different destination. The source and destination can be device files for disk and tape drives or normal files.
The basic format of dd is
dd [option = value ....]
Table 8.8. lists some of the different options available.
David Jones (11/30/94) Page 149 Backups Chapter 8
Option Purpose if=name input file name (default is standard input) of=name output file name (default is standard output) ibs=num the input block size in num bytes (default is 512) obs=num the output block size in num bytes (default is 512) bs=num set both input and output block size skip=num skip num input records before starting to copy files=num copy num files before stopping (used when input is from magnetic tape) conv=ascii convert EBCDIC to ASCII conv=ebcdic convert ASCII to EBCDIC conv=lcase make all letters lowercase conv=ucase make all letters uppercase conv=swab swap every pair of bytes
Table 8.8. Options for dd.
For example: dd if=/dev/hda1 of=/dev/rmt4 with all the default settings copy the contents of hda1 (the first partition on disk a) to the tape drive for the system
Exercises.
8.3. Use dd to copy the contents of a floppy disk to a single file to be stored under your home directory.
The mt Command
The usual media used in backups is magnetic tape. Magnetic tape is a sequential media. That means that to access a particular file you must pass over all the tape containing files that come before the file you want. The mt command is used to send commands to a magnetic tape drive that control the location of the read/write head of the drive.
mt [-f tapename] command [count]
Arguments Purpose tapename raw device name of the tape device command one of the commands specified in table 8.9. Not all commands are recognised by all tape drives. count number of times to carry out command
Table 8.9. Parameters for the mt Command.
David Jones (11/30/94) Page 150 Backups Chapter 8
Commands Action fsf move forward the number of files specified by the count argument asf move forward to file number count rewind rewind the tape retension wind the tape out to the end and then rewind erase erase the entire tape offline eject the tape
Table 8.10. Commands Possible using the mt Command.
For example: mt -f /dev/nrst0 asf 3 move to the third file on the tape mt -f /dev/nrst0 rewind mt -f /dev/nrst0 fsf 3 same as the first command
The mt command can be used to put multiple dump/tar archive files onto the one tape. Each time dump/tar is used one file is written to the tape. The mt command can be used to move the read/write head of the tape drive to the end of that file, at which time dump/tar can be used to add another file.
For example: mt -f /dev/rmt/4 rewind rewind the tape drive to the start of the tape tar -cvf /dev/rmt/4 /home/jonesd backup my home directory, after this command the tape will be automatically rewound mt -f /dev/rmt/4 asf 1 move the read/write head forward to the end of the first file tar -cvf /dev/rmt/4a /home/thorleym backup the home directory of thorleym onto the end of the tape drive
There are now two tar files on the tape, the first containing all the files and directories from the directory /home/jonesd and the second containing all the files and directories from the directory /home/thorleym.
Compression Programs
Various compression programs are sometimes used in conjunction with transport programs to reduce the size of backups. This is not always a good idea. Adding compression to a backup adds extra complexity to the backup and as such increases the chances of something going wrong.
David Jones (11/30/94) Page 151 Backups Chapter 8
compress
compress is the standard Unix compression program and is found on every Unix machine (well, I don't know of one that doesn't have it). The basic format of the compress command is compress filename
The file with the name filename will be replaced with a file with the same name but with an extension of .Z added and that is smaller than the orginal (it has been compressed).
A compressed file is uncompressed using the uncompress command or the -d switch of compress. uncompress filename or compress -d filename
For example: bash$ ls -l ext349* -rw-r----- 1 jonesd 17340 Jul 16 14:28 ext349 bash$ compress ext349 bash$ ls -l ext349* -rw-r----- 1 jonesd 5572 Jul 16 14:28 ext349.Z bash$ uncompress ext349 bash$ ls -l ext349* -rw-r----- 1 jonesd 17340 Jul 16 14:28 ext349
gzip
gzip is a new addition to the Unix compression family. It works in basically the same way as compress but uses a different (and better) compression algorithm. It uses an extension of .z and the program to uncompress a gzip archive is gunzip.
For example: bash$ gzip ext349 bash$ ls -l ext349* -rw-r----- 1 jonesd 4029 Jul 16 14:28 ext349.z bash$ gunzip ext349
Exercises.
8.4. Modify your solution to exercise 8-2 so that instead of writing the contents of your floppy straight to a file on your hard disk it first compresses the file using either compress or gzip and then saves to a file.
David Jones (11/30/94) Page 152 Backups Chapter 8
Conclusions
In this chapter you have • been introduced to the components of a backup strategy scheduler, transport, and media • been shown some of the Unix commands that can be used as the transport in a backup strategy • examined some of the characteristics of a good backup strategy and some of the factors that affect a backup strategy
Review Questions
8.1. Design a backup strategy for your system. List the components of your backup strategy and explain how these components affect your backup strategy.
8.2. Explain the terms media, scheduler and transport.
8.3. Outline the difference between file by file and image transport programs.
8.4. If possible contact a local business and discuss the backup strategy that they use. If possible identify whether that strategy meets some of the guidelines discussed in this chapter. (It is not necessary for them to be using UNIX.)
David Jones (11/30/94) Page 153 Starting Up and Shutting Down Chapter 9
Chapter 9
STARTING UP AND SHUTTING DOWN
God alone can finish. -- John Ruskin
Objectives
By the end of this chapter you should be • able to describe the process of booting a UNIX computer, • able to explain the different formats of the initialisation scripts used by different versions of the UNIX operating system, • able to explain the purpose of the init process, • able to describe the format, contents and purpose of the file /etc/inittab, • able to list and explain the different run levels a UNIX computer may be in, • aware of the daemons that may be started, • aware of the reasons and methods of shutting a UNIX computer down, • able to explain why you shouldn't just turn a UNIX computer off, and • aware of some problems and their solutions as to why a UNIX computer will not boot.
Introduction
Being a multi-tasking, multi-user operating system means that UNIX is a great deal more complex than an operating system like MS-DOS. Before the UNIX operating system can perform correctly there are a number of steps that must be followed and procedures executed. The failure of any one of these can mean that the system will not start, or if it does it will not work correctly. It is important for the Systems Administrator to be aware of what happens during system startup so that any problems that occur can be remedied.
It is also important for the Systems Administrator to understand what is the correct mechanism to use to shut a UNIX machine down. A UNIX machine should (almost) never be just turned off. There are a number of steps to carry out to ensure that the operating system and many of its support functions remain in a consistent state.
By the end of this chapter you should be familiar with the startup and shutdown procedures for a UNIX machine and all the related concepts.
David Jones (11/30/94) Page 154 Starting Up and Shutting Down Chapter 9
The UNIX Boot Process
The startup procedure of a UNIX machine is much more complex than that of a single user operating system. Different UNIX systems will have slightly different methods of coming up. The following discussion provides a general overview of the UNIX boot process. Diagram 9-1 provides a summary of the process.
ROM
Most machines have a section of read only memory (ROM) that contains a program the machine executes when the power first comes on. What is programmed into ROM will depend on the hardware platform.
For example, on an IBM PC the ROM program typically does some hardware probing and then looks in a number of predefined locations (the first floppy drive and the primary hard drive partition) for a bootstrap program.
On hardware designed specifically for the UNIX operating system (machines from DEC, SUN etc) the ROM program will be a little more complex. Many will present some form of prompt. Generally this prompt will accept a number of commands that allow the Systems Administrator to specify • where to boot the machine from, and Sometimes the standard root partition will be corrupt and the system will have to be booted from another device. Examples include another hard drive, a CD-ROM, floppy disk or even a tape drive. • whether to come up in single user or multi-user mode.
As a bare minimum the ROM program must be smart enough to work out where the bootstrap program is stored and how to start executing it.
The Bootstrap Program
At some stage the ROM program will execute the code stored in the boot block of a device (typically a hard disk drive). The code stored in the boot block is referred to as a bootstrap program.
The bootstrap program is responsible for locating and loading the kernel of the UNIX operating system into memory. The kernel of a UNIX operating system is usually stored in the root directory of the file system under some system defined filename. Table 9.1. provides examples of some of the kernel filenames used by some systems.
David Jones (11/30/94) Page 155 Starting Up and Shutting Down Chapter 9
System Filename /vmunix BSD /unix SysV /vmlinuz Linux
Table 9.1. Kernel Names.
On some systems the bootstrap program may also perform some additional hardware probing.
Kernel Initialisation
Once the bootstrap program has installed the kernel into memory the kernel will • initialise its internal data structures, • perform some further hardware checking (some systems check for every major device that is supposed to be connected), • verify the integrity of the root file system and then mount it, and • create the process 0 (swapper) and process 1 (init).
The swapper process is actually part of the kernel and is not a "real" process. The init process is the ultimate parent of all processes that will execute on a UNIX system.
The only way in which a process can be created is by an existing process performing a fork. A fork creates a brand new process that contains copies of the code and data structures of the original process. In most cases the new process will then perform an exec that replaces the old code and data structures with that of a new program.
Once the kernel has initialised itself init will perform the remainder of the startup procedure.
init
init is the process that is the ultimate ancestor of all user processes on a UNIX system. It always has a PID of 1. Depending on its configuration init will place the system into either single user mode or multi-user mode (the normal state of a UNIX machine).
Going into multi-user mode the init process must execute the various system startup scripts. These startup scripts are Bourne shell scripts stored under the /etc directory. Different versions of UNIX have slightly different formats but the responsibilities are basically the same (they are outlined below).
David Jones (11/30/94) Page 156 Starting Up and Shutting Down Chapter 9
.
. ROM
Boot strap program
Kernel Initialization, Hardware probing. Single User Mode
Creation of swapper and init init executes rc single or scripts multi-user?
/etc/ttysinit reads terminal /etc/inittab (BSD) configuration file (Sys V)
getty process started getty process started
Diagram 9.1. UNIX Startup Procedure.
User Logins
One of the last steps performed by init is to enable user logins. A user is able to login using a terminal because a getty process has been run for that terminal. What happens after the getty process will be discussed in the next chapter on terminals.
Single User, Multi-User and Different Run-Levels
All UNIX machine can be in two basic states • multi-user mode, and This is the standard mode for a UNIX machine. Multiple users are allowed to log in, all the daemons and all the services provided by the machine are available. • single user mode. This is basically the system maintenance mode. In single user mode only the bare minimum of services are available. Only one user (the root user) will be able to log in, only the root file system will be mounted automatically (others may be able to be mounted manually) and most of the daemons and services will not be available.
Whether or not the system comes up into single-user or multi-user mode depends on how the system was started up. On some systems the ROM program will have a switch that allows the Systems Administrator to specify single or multi-user mode.
David Jones (11/30/94) Page 157 Starting Up and Shutting Down Chapter 9
When going into single user mode init forks to create a new process running the Bourne shell with root privilege. It does this prior to executing any of the initialisation scripts. This means that in single-user mode very few of the normal services on the machine are running, including normal logins, networking, the print services and many others.
The system will typically come up in single user mode for two reasons • the root user wanted it to There are times when you will want to bring the system up single user mode to perform some system maintenance duties. Typically the root user will bring the system up single user mode by entering a particular command at the ROM program prompt or by specifying a particular option at shutdown. • the boot procedure has failed Typically caused by some errors in the initialisation files or by fsck detecting some errors that it could not fix by itself.
When the Systems Administrator logs out of the shell prompt provided in single user mode the system will normally attempt to enter multi-user mode.
System V Run Levels
Later versions of System V based Unices added a number of different run levels that the machine could be in. Table 9.2 summarises the different run levels. At any one time the system must be in one of these run levels. There are various administrative commands that can take the system from one level to the other (the command who -r can be used to display the run level).
State Function 0prepare the machine for turning off power, if the machine can turn the power off tell it to do so 1system administrator mode, all file systems mounted, only small set of kernel processes running, single user mode 2multi-user mode 3multi-user mode with remote file sharing, processes, and daemons 4user definable system state 5shutdown to ROM 6shutdown and reboot s,S single-user mode, only root file system mounted
Table 9.2. init states for SVR4 UNIX Systems
As the system boots it will move through the various run levels (s, 1, 2, 3) under the control of init. Each run level has associated with it various initialisation scripts that will be executed as the machine enters that run level.
David Jones (11/30/94) Page 158 Starting Up and Shutting Down Chapter 9
On SysV based machines the procedures followed by init as it moves through these run levels is controlled by the file /etc/inittab.
/etc/inittab
Each line of the inittab file is a separate entry and uses the following format
id:run_state:action:process
Label Explanation id one or two characters to uniquely identify the entry run_state indicates the run level at which the process should be executed action this tells init how to execute process process the full path of a program to execute for this entry
Table 9.3. Explanation of inittab field entries.
When init receives notification that a certain event has happened it will examine the /etc/inittab file and execute the process specified for that event. How it should execute the process is controlled by the action field.
Example values for action include • respawn If the process does not already exist create it but don't wait for it to terminate, carry on immediately. If at any stage the process terminates, restart it (the getty processes for terminals are started this way). • wait When init receives notification of the event for the first time it will execute the process and wait for the process to finish. The process is never executed again until the system receives notification of the event again (This is how the system will execute the initialisation scripts.) • bootwait Execute the process only when the system first goes multi-user and wait for the process to finish. • powerfail Execute only when init receives a power fail signal.
Figure 9.1 is an example of an /etc/inittab taken from a System V Release 4 machine.
David Jones (11/30/94) Page 159 Starting Up and Shutting Down Chapter 9
mt:23:bootwait:/etc/brc
Figure 9.1. Example /etc/inittab.
The Startup Scripts
As mentioned above one of the major responsibilities of the init process is to execute the systems startup scripts. These scripts are executed after the kernel has been initialised but before normal users are allowed to log on. These scripts will typically • check the integrity of the machine’s file systems using fsck, • mount the file systems, • designate paging and swap areas, • check disk quotas, • clear out temporary files in /tmp and other locations, • start up system daemons for printing, mail, accounting, system logging, networking and cron, • enable user logins by running getty processes, and • a number of other tasks.
Each basic version of UNIX has a different format for the startup scripts • the BSD format, and init will run predefined shell scripts usually called /etc/rc and perhaps /etc/rc.local. • the System V format. init reads the file /etc/inittab and as the system enters each associated run level it runs a specified shell script.
BSD Startup Scripts
Most BSD based systems will have at least the files • /etc/rc, and The system startup script that is executed as the system goes multi-user. It will typically run /etc/rc.local. • /etc/rc.local. The startup script that contains procedures deemed to be specific to your local site.
Some systems will add additional scripts /etc/rc.boot, /etc/rc.single that are run under various circumstances.
David Jones (11/30/94) Page 160 Starting Up and Shutting Down Chapter 9
SysV Startup Scripts
Under SysV the /etc/inittab file is used to inform the init process which startup scripts it should execute. Each run level will have associated with it a particular startup script.
The filenames for these scripts generally follow the format /etc/rcL (these files may be located in the /sbin directory on later versions). Where L is the single character that denotes the run level (taken from table 9.2).
The purpose of these scripts is to execute all the shell scripts stored in a directory called with the name /etc/rcL.d. Where L is again the run level.
For example: When the system enters run level 3 init will execute the script /etc/rc3 (/sbin/rc3 on later machines). This script will in turn execute all the scripts in the directory /etc/rc3.d.
The rcL.d directories will contain scripts with filenames that either start with • an uppercase K, or The "K files" are used to kill processes. • an uppercase S The "S files" are used to start processes and other initialisation procedures.
init will execute all the "K files" in a directory in alphabetical order first and then execute all the "S files" in alphabetical order.
Exercises.
9-1. Examine the startup scripts of your system. Which format does your machine use? List the names of all the startup scripts used. Add echo commands to each of the startup scripts to display an informative message when the scripts start and finish.
9-2. Modify your systems startup files to so that it displays a message.
Why Won't My System Boot?
There will be times when you have to reboot your machine in a nasty manner. One rule of thumb used by Systems Administration to solve some problems is "When in doubt, turn the power off, count to ten slowly, and turn the power back on". There will be times when the system won't come back to you, DON'T PANIC!
David Jones (11/30/94) Page 161 Starting Up and Shutting Down Chapter 9
Possible reasons why the system won't reboot include • hardware problems, Both hardware failure and problems caused by human error (e.g. the power cord isn't plugged in, the drive cable is the wrong way around) • defective boot floppies, drives or tapes, • damaged file systems, • improperly configured kernels, A kernel configured to use SCSI drives won't boot on a system that uses an IDE drive controller. • errors in the rc scripts.
Solutions
The following is a Systems Administration maxim
Always keep a separate working method for booting the machine into at least single user mode.
This method might be a boot floppy, CD-ROM or tape. The format doesn't matter. What does matter that at anytime you can bring the system up in at least single user mode so you can perform some repairs.
A separate mechanism to bring the system up single user mode will enable you to solve most problems involved with damaged file systems, improperly configured kernels and errors in the rc scripts.
Hardware Problems
Some guidelines to solving hardware problems • check the power supply and its connections, Don’t laugh, there are many cases I know of in which the whole problem was caused by the equipment not being plugged in properly or not at all. • check the cables and plugs on the devices, • check any fault lights on the hardware, • power cycle the equipment (power off, power on), There is an old Systems Administration maxim. If something doesn’t work turn it off, count to 10 very slowly and turn it back on again (usually with the fingers crossed). • try rebooting the system without selected pieces of hardware, It may be only one faulty device that is causing the problem. Try isolating the problem device. • use any diagnostic programs that are available, or as a last resort • call a technician or a vendor.
David Jones (11/30/94) Page 162 Starting Up and Shutting Down Chapter 9
Damaged File Systems
First always have backups of all file systems so that you can quickly recover some information. Try using fsck to fix the problem. If worse comes to worse resort to your backups.
Improperly Configured Kernels
Reasons why you might change the kernel will be discussed in a later chapter. When you do change the kernel you should always keep a backup working version of kernel that you can use to reboot the system.
Daemons
A daemon is a process that spends much of its time waiting for some event to occur. Once the event occurs the daemon wakes up and performs some predefined action. The action is sometimes controlled by a configuration file. init itself can be classed as a daemon.
A standard UNIX system has a large number of daemons running at any one time. Most of these daemons have to be started by the initialisation scripts as the system first boots. Table 9.4. lists some of the common daemons on a UNIX machine.
Daemon Purpose inetd the main network server named Name server, provides dynamic hostname data for TCP/IP networking timed Time daemon used to synchronise different system clocks sendmail the mail daemon, responsible for delivering mail locally and to remote hosts nfsd nfs file exporting daemon ypbind NIS (yellow pages) daemons ypserv syslogd System logging daemon, records various events.
Table 9.4. Various Daemons.
Exercises.
9-3. Using a combination of the ps command and by examining your systems startup scripts discover which daemons are being run on your system.
David Jones (11/30/94) Page 163 Starting Up and Shutting Down Chapter 9
Shutting the System Down
You should not just simply turn a UNIX computer off or reboot it. Doing so will usually cause some sort of damage to the system especially to the file system. Most of the time the operating system may be able to recover from such a situation (but NOT always).
There are a number of tasks that have to be performed for a UNIX system to be shutdown cleanly • tell the users the system is going down, Telling them 5 seconds before pulling the plug is not a good way of promoting good feeling amongst your users. Wherever possible the users should know at least a couple of days in advance that the system is going down (there is always one user who never knows about it and complains). • signal all the currently executing processes that it is time for them to die, Hopefully these processes will all die gracefully (given some time) and will not do anything nasty to the system in the process. • place the system into single user mode, and • perform sync to flush the file systems buffers so that the physical state of the file system matches the logical state.
All UNIX systems will supply a command or two that perform these tasks for you. System V and BSD based systems use a slightly different format.
Reasons for Shutting the System Down
In general, you should try to limit the number of times you turn a computer on or off as doing so involves some wear and tear. It is often better to simply leave the computer on 24 hours a day. In the case of a UNIX system being used for a mission critical application by some business it may have to be up 24 hours a day.
Some of the reasons why you may wish to shut a UNIX system down include • general housekeeping, Everytime you reboot a UNIX computer it will perform some important housekeeping tasks, including deleting files from the temporary directories and performing checks on the machines file systems. Rebooting will also get rid of any zombie processes. • general failures, and Occasionally problems will arise for which there is only one resort, shutdown. These problems can include • hanging logins, • unsuccessful mount requests, • dazed devices, • runaway processes filling up disk space or CPU time and preventing any useful work being done.
David Jones (11/30/94) Page 164 Starting Up and Shutting Down Chapter 9
• system maintenance and additions. There are some operations that only work if the system is rebooted or if the system is in single user mode, for example adding a new device.
Being Nice to the Users
Knowing of the existence of the appropriate command is the first step in bringing your UNIX computer down. The other step is outlined in the heading for this section. The following command is an example of what not to do.
shutdown -g0
On a SVR4 box this results in a message somewhat like this appearing on the user's terminal
THE SYSTEM IS BEING SHUT DOWN NOW ! ! ! Log off now or risk your files being damaged.
Not a method inclined to win friends and influence people.
The following is a list of guidelines of how and when to perform system shutdowns • shutdowns should be scheduled, If users know the system is coming down at specified times they can organise their computer time around those times. • perform a regular shutdown once a week, and A guideline, so that the housekeeping tasks discussed above can be performed. If it's regular the users get to know when the system will be going down. • use /etc/motd. This is the message users see when they first log onto a system use it to inform users of the next scheduled shutdown.
Commands for Shutting Down and Rebooting
There are a number of different methods for shutting down and rebooting a system including
• the shutdown command The most used method for shutting the system down. The command can display messages at preset intervals warning the users that the system is coming down. • the BSD halt command Logs the shutdown, kills the system processes, executes sync and halts the processor. • the BSD reboot command Similar to halt but causes the machine to reboot rather than halting.
David Jones (11/30/94) Page 165 Starting Up and Shutting Down Chapter 9
• sending init a TERM signal init will usually interpret a TERM signal (signal number 15) as a command to go into single user mode. It will kill of user processes and daemons. The command is kill -15 1 (init is always process number 1). It may not work or be safe on all machines. • the BSD fasthalt or fastboot commands Shell scripts which create a file /fastboot before calling halt or reboot. When the system reboots and it finds a file /fastboot it will not perform a fsck on the file systems.
The most used method will normally be the shutdown command. It provides users with warnings and is the safest method to use. AT&T and BSD have different shutdown commands.
The following sections talk about the format, purpose and options for various commands associated with the shutdown procedure. It is by no means an exhaustive discussion and many systems will have modified or made additions to the way in which these commands work.
The AT&T shutdown Command
The format of the command is
shutdown -ggrace_period -iinit_state [-y]
Parameter Purpose grace_period The number of seconds to wait before beginning the shutdown (default of 60) init_state The init state (run level) to put the system into (one of the appropriate levels listed in Table 9.2.
Table 9.5. Parameters of the SysV Shutdown Command.
In some cases the command will ask for confirmation just before performing the shutdown. This question can be pre-answered by using the -y option.
The BSD shutdown Command
The format of the command is
shutdown [-fhknr]time [warning-message]
The command actually calls on the halt command to perform the actual work of halting the system. shutdown just displays messages etc. before calling halt.
David Jones (11/30/94) Page 166 Starting Up and Shutting Down Chapter 9
Parameters Meaning -f flag to specify that file systems will not be checked on system restart (create the /fastboot file) -k simulate shutdown of the system, WON'T actually do it -h simply halt the system (the same as using the halt command) -r reboot the system (same as using the reboot command) -n don't execute the sync command before shutting down time has two formats either +number system down in number minutes hour:min system down at time indicated in 24 hour format Warning messages are displayed at periodic intervals and logins are disabled five minutes before shutdown warning-message message to display to the users
Table 9.6. Format of the BSD Shutdown Command.
The BSD halt Command
The halt and reboot commands are actually used by the shutdown command. The halt command performs a sync on the disks and stops the processors. The format of the halt command is.
halt [-nqy]
Parameters Purpose -n don't sync the disks before stopping -q do a quick halt (create the /fastboot file so that the file systems are not fscked on reboot) -y halt the system
Table 9.7. Parameters of the BSD halt Command.
David Jones (11/30/94) Page 167 Starting Up and Shutting Down Chapter 9
The reboot Command
The reboot command shuts the system down and then restarts it. Its format is
reboot [-dnq][boot arguments]
Parameters Meaning -d dump a copy of the kernels memory (referred to as system core) before rebooting (some machines recognise this option but don't do anything) -n avoid the sync call. -q reboot quickly and ungracefully boot arguments Use to specify how the system should restart (single user mode for example). Some machines ignore these arguments.
Table 9.8. Parameters of the BSD reboot Command.
Conclusions
This chapter has provided • an introduction to the processes performed when starting up and shutting down a UNIX computer, • a knowledge of the commands, shell scripts and configuration files involved in both processes, • an introduction to the problems that might cause a UNIX computer not to reboot and some of the possible solutions, and • reasons why you would want to shut a UNIX computer down.
David Jones (11/30/94) Page 168 Starting Up and Shutting Down Chapter 9
Review Questions
9.1. Describe (with diagrams) the process of starting a UNIX computer.
9.2. List three reasons why you might want to shut a UNIX computer down.
9.3. List four reasons why a UNIX computer may not reboot.
9.4. What steps have to be completed to shut a UNIX system down properly?
9.5. On a Linux machine there is usually a shell script /etc/rc.d/rc.0 that is executed when the machine is shutdown. However there is no entry in the /etc/inittab that tells init to run the script. How is the script executed?
Hints.
Think about the programs that are executed when a machine is being shutdown. There is a command strings that displays any text that may appear in an executable program.
David Jones (11/30/94) Page 169 Terminals Chapter 10
Chapter 10
TERMINALS
Many complain of their looks, but not of their brains. -- Yiddish Proverb
Objectives
By the end of this chapter you should • have been introduced to procedures for connecting terminals to a UNIX computer, • be able to describe the login procedure for a terminal, • be able to list and explain the configuration files used by terminals, • be able to explain the purpose of the commands tty stty, • explain the purpose and relation between termcap and terminfo,
Introduction
UNIX is a multi-user operating system. To make use of this attribute multiple users must be able to connect to the system at the same time. This implies that there must be multiple access points. Dumb terminals are the cheapest method for providing multiple access points to a UNIX machine.
In most cases a dumb terminal is connected to a UNIX machine using a serial line. A dumb terminal does little more than present text to the user and transfer keystrokes from the terminal back to the central computer. It is dumb because the terminal does no processing on the data.
Even though the interface on such beasts is primitive they are still one of the most used methods for adding extra access points to a UNIX computer. Businesses wanting to use dumb terminals have two options • purpose built dumb terminals, or • modern personal computers using communications programs to simulate dumb terminals.
Hardware
The process of connecting terminals to a UNIX machine follows a fairly generic set of steps that can be divided into two separate sections, hardware and software. Figure 10.1 lists the steps in the hardware process.
David Jones (11/30/94) Page 170 Terminals Chapter 10
1. Choose a port on the computer. The device has to be connected to the computer . To do so an I/O port and the associated device file must be chosen. 2. Obtain the correct cable. Find a cable of the right type that has the right plugs for your device and computer. 3. Configure the device. The device will have to be configured to work correctly with your machine. 4. Make the connection. Connect the device and prepare it for operation. 5. Test the connection. Send output of some description to the device file to test whether or not the hardware connection is working.
Figure 10.1. Generic Hardware Process for Adding a Device.
Ports and Devices
There are two steps to choosing a port to which to connect a device • choosing the physical port, and A typical UNIX computer is likely to have many different ports of various configurations. Some computers will be connected to multi-port cards that supply multiple (4, 20 and more) ports. • finding the corresponding logical device file. Each physical connector has a corresponding device file through which the operating system passes information to the port.
Table 10.1 lists some of the common device files on a UNIX machine.
Device File Purpose /dev/ttyn Generic device file used for terminals, usually corresponds to a serial port (n is a number e.g. /dev/tty0) /dev/modem modem device file /dev/lp line printer device file /dev/par parallel port /dev/ser serial port
Table 10.1. Example Device Filenames.
David Jones (11/30/94) Page 171. Terminals Chapter 10
Terminals are generally connected using serial ports. The number, type and location of the serial ports will be machine specific. Some machines will have multi-port serial cards that provide from four or more serial ports.
Most serial ports will be associated with device files of the format /dev/ttyn, where n is some number.
Cables
The next step is obtaining the correct cable to connect the device to the computer. To do so the cable must be correctly configured and have connectors that are the • right sex, and Plugs are either female (small holes) or male (small prongs). • the correct size Serial plugs for example can be either 9 pin, 25 pin or a couple of other configurations.
Even though serial cables are meant to follow the RS-232 standard there are a number of differences that mean no two serial cables are the same. Serial cables are used to connect two different types of device • DTE, data terminal equipment Most terminals and computers fall into this category. • DCE, data communications equipment Modems and serial printers fall into this category.
The type of serial cable you use depends on the type of equipment you are connecting. Table 10.2 explains.
Connection Cable Type DTE <-> DCE straight modem cable DTE <-> DTE null modem cable
Table 10.2. Types of Serial Cable.
Reading.
Resource Materials Chapter 10.1.
Purpose.
An optional reading that explains in more detail the ins and outs of RS-232 cables. This section will not be examined and is simply included for reference.
David Jones (11/30/94) Page 172. Terminals Chapter 10
Terminal Configuration
For a dumb terminal to work correctly it must be configured properly. Characteristics of a dumb terminal that need to be configured include • bits per second, The speed at which information can be transferred. Typical values (for today) range from 9600 bps up to 38,400 bps. • parity, Typically be set off. • duplex, This should be set to full and signifies that data can be transferred in both directions simultaneously. • auto linefeed, Should be turned off. The end of a line under UNIX is signified by a newline character. MS-DOS and other systems use a combination of a newline and a carriage return character. • data bits, and Suggested values are either 7 or 8 with 7 being the preference. • stop bits. With 7 data bits use 2 stop bits. With 8 data bits use 1 stop bit. In the case of purpose built dumb terminals, configuration will generally be performed by setting dip switches. In the case of a personal computer and a communications package these settings are set using the options within the communications program.
Testing the Connection
Once all the hardware is configured, connected, and turned on, the next step is to test whether or not you can actually transmit data through the connection. The simplest method to do this is to send some information directly to the device file associated with the device.
For example: ls -l / > /dev/tty1 If the connection is correct and working you should see the output appear on the device.
Be careful when you are choosing device files to send output to. Sending output to the wrong device can be disastrous.
For example, one of the previous students for this subject tried the command ls > /dev/hda on a Linux box. Under Linux /dev/hda is the device file for the first hard disk. Since he was the root user at the time the operating system happily sent the data to the hard disk.
David Jones (11/30/94) Page 173. Terminals Chapter 10
By writing directly to the device file he bypassed the file system code that places the data on the drive into the correct format. As a result the data on the disk was lost.
Reasons Why It Won't Work
There are a number or reasons why a connection may not work including • incorrect permissions, For the test to work you must have write permission on the device file. Check the permissions. Typically you have to be the root user to perform the test. Don't change the permissions on the device file to something like world write. This can be a security hole. • the wrong device file, You've picked the wrong device file and the information isn't being sent to the new device. • incorrect configuration, and The hardware configuration on the device is not correct. Don't expect the output you send to the device to appear picture perfect. Since you are bypassing the normal mechanism for using the device it may not work 100%. • incorrect cabling. Is the device turned on? Are you sure you have the correct type of cable.
Exercise.
10.1. Beg, borrow or steal a dumb terminal (another PC with a communications program will suffice). Perform all the steps listed above for connecting the terminal to your UNIX machine.
Software
Terminal configuration files is one area in which the diversity of UNIX platforms rears its ugly head. System V based machines will use different configuration files than BSD based systems. Early BSD systems use different configuration files again.
Terminal configuration files can be divided along the lines of their purpose • enabling the login process, For a terminal to work users have to login. For users to login particular processes have to be executed for each terminal. There are configuration files that control which device files have the login process enabled. • setting line configuration, and The operating system has to know about and set the characteristics of the serial line that the terminal is connected to, speed, data bits, parity etc.
David Jones (11/30/94) Page 174. Terminals Chapter 10
• terminal characteristics. Different terminals have different keyboard layouts, different capabilities (colour etc) and different special character codes to do things like clear the screen. In order to use a particular type of terminal to its fullest ability UNIX must know about its capabilities. To do this the terminal must have an entry in the database of terminal characteristics that UNIX maintains.
The Login Process
For a user to actually be able to use a terminal it is necessary for the UNIX login process to be running for that terminal.
The login process on a terminal involves • /etc/init, The first process run on a UNIX system. It is init's responsibility to ensure that each terminal has a getty process running for it. • getty, Is responsible for setting up the characteristics of the line, displaying the login prompt, getting a username, and executing the login process with the username as an argument. • login, and Is responsible for logging the user into the system, obtaining and validating the password, and executing the user's login shell. • a login shell. The user's entry in the /etc/passwd file lists the program to execute as the user's login shell. The shell will in turn execute the appropriate startup scripts.
When the user quits their login shell the init process picks up this event and will rerun the getty process for that terminal.
The SysV Login Process
Under SysV the init process is controlled by the /etc/inittab configuration file (the format of /etc/inittab is discussed in the previous chapter). The inittab file must have an entry for each terminal that requires a getty process. Typical entries look like c6:23:respawn:/etc/getty 9600 tty6 c7:23:respawn:/etc/getty 9600 ttys1
The getty process here takes two arguments • a label used as a lookup value for the file /etc/gettydefs, and • the device name for which the getty process is being executed.
The /etc/gettydefs file is used for setting various line options. /etc/gettydefs has the following format label#initial flags#final flags#login prompt#next label
David Jones (11/30/94) Page 175. Terminals Chapter 10
Table 10.3 explains each field from the gettydefs file.
Field Purpose label the "key" to the particular entry initial flags to set before running the login process flags final flags flags to set after running the login process login the prompt to be displayed by getty before prompt running login next label points to the next label to use if this one is not valid (used to cycle through baud rates for modems)
Table 10.3. Fields for the /etc/gettydefs file.
The process for enabling logins for a new terminal on a SysV system includes • have an entry in gettydefs file, The system should come with most of the standard settings required for normal use so a new addition is usually not required. • add a new entry to the inittab file to start the getty process for the terminal, and • force init to reread the inittab file. The new getty process will not be started until init reenters the specified run levels. Under SysV this is typically done using the command init -q
The BSD Login Process
UNIX versions based on pre 4.3 BSD use a different format than subsequent versions. In this section discussion will be restricted to the later versions. From 4.3 on BSD uses a single configuration file /etc/ttys with the following format.
device program terminal_type status
David Jones (11/30/94) Page 176. Terminals Chapter 10
Field Purpose device device file terminal is connected through program program to be run if terminal is on (usually getty) terminal_type typically the name of a terminal type listed in /etc/termcap status a list of key words separated by spaces, keywords include on should run program off ignore the entry secure root logins are allowed on this terminal
Table 10.4. Format of the BSD /etc/ttys File.
Line Configuration
Every terminal connected to a UNIX machine has an associated terminal driver process. This process maintains • a list of characteristics about the current terminal, and • a list of special characters and how they should be handled.
Probably the most common complaint from users is that when they hit particular keys the terminal doesn't do what is expected. Hitting the backspace key might produce a weird character or the cursor keys might not work under vi. These problems are caused by the terminal driver not being configured properly.
The tty command is sometimes useful. It displays the name of the device file being used by the current terminal.
Initially these settings are set up by the system from the entries in the system’s terminal configuration database. The stty command can be used to view and modify these settings.
Table 10.5 lists some of the terminal characteristics and Table 10.6 lists some of the special characters. To view the current settings try stty -a (the command might be stty all or stty everything depending on your system).
David Jones (11/30/94) Page 177. Terminals Chapter 10
For example: bash > stty -a speed 9600 baud; rows 24; columns 80; line = 0; intr = ^C; quit = ^\; erase = ^?; kill = ^U; eof = ^D; eol =
Option Meaning n bits per second rows n lines to the screen columns n columns on the screen oddp odd parity evenp even parity -parity no parity
Table 10.5. Characteristics affected by stty.
Symbolic SysV BSD Meaning Name default default ERASE # ^H erase one character of input WERASE N/A ^W erase one word of input KILL @ ^U erase entire line EOF ^D ^D end of file INTR ^? ^C interrupt current process QUIT ^\\ ^\\ kill current process with core dump STOP ^S ^S stop output to screen START ^Q ^Q restart output to screen SUSPEND N/A ^Z suspend current process
Table 10.6. Special Characters.
In these tables you will see character combinations like ^H and ^?. The ^ symbol is used in this case to signify the control key. So ^H could be rewritten CTRL-H.
David Jones (11/30/94) Page 178. Terminals Chapter 10
A useful option of the stty command is sane. Entering stty sane when the terminal is behaving strangely will solve many problems.
It is possible to use I/O redirection to affect the settings of terminals other than the one you are currently using. Which form of I/O redirection (input or output) you use depends on your system. For BSD redirect the output of stty. For SysV redirect the input. (This will only work if you have the correct permissions on the device file associated with the terminal.
Exercises
10.2. Use the stty program to examine the current status for your terminal. Change the values for EOF ERASE and INTR experiment using the system with the changes.
10.3. Change the values for rows and columns and observe the difference.
10.4. Complete the process of connecting the terminal you started in question 10.1.
10.5. If you are using a PC and a communications program you should be able to change the type of terminal you are using as part of the programs options. Change the type of terminal you are using. How does it effect using the terminal? Modify the appropriate files so that the system uses the correct entry from the terminal configuration database.
Terminal Characteristics
Different terminals have different keyboard layouts, escape codes and capabilities. For example one terminal will use one combination of characters to signify clearing the screen while another terminal will use another combination of characters.
If programs that wish to be able to clear the screen want to work on different terminals they must be able to find out how each terminal performs the operation. Under the UNIX operating system programs discover this information using • the TERM (term if you’re using csh) environment variable, and • a system specific terminal database.
The shell variable TERM is usually initialised when a user first logs in. It will hold a unique identifier that signifies the type of terminal being used. This identifier is used to access the information about the terminal from the system’s terminal database.
David Jones (11/30/94) Page 179. Terminals Chapter 10
If the TERM variable is set incorrectly or the terminal does not have an entry in the terminal database problems likely to occur include • keys not performing the expected task, Hitting the backspace key doesn't do anything or displays a weird key combination. • screen output not being written properly, or The screen not scrolling properly, unexpected colours or characters are appearing. • programs not working properly. Full screen programs, vi for example, make use of special characteristics offered by most terminals. If the particular terminal you have doesn't have an entry in the terminal characteristics file it can't make use of these special characteristics.
It is the responsibility of various startup files (typically /etc/profile) to make sure that the TERM variable is initialised to the correct value.
For example: The following is an example of how the TERM variable might be set. if [ ‘tty‘ = /dev/tty1 ] then TERM=vt100 elsif [ ‘tty‘ = /dev/tty2 ] then TERM=tvi912b else TERM=console fi On this system the terminal connected to /dev/tty1 is a vt100 so that is the value TERM is set to. The terminal on /dev/tty2 is a tvi912b and it assumes that any other type of terminal is a console. The tty command used here returns the device file used by the current terminal.
Once the TERM variable is set processes can use its value to access information in the terminal database. SysV and BSD based systems use different terminal databases.
Exercises.
10.6. Make up some name for a terminal, e.g. myterm. Set the TERM shell variable to this value. Attempt to use the vi editor. What happens?
The SysV Terminal Database
The SysV terminal database is called terminfo. It is a collection of separate binary files. Each different terminal type will have its own separate binary terminfo file that describes its characteristics.
David Jones (11/30/94) Page 180. Terminals Chapter 10
terminfo files are usually stored under the /usr/lib/terminfo directory. The terminfo sub-directory is further divided into directories with single character names. The actual terminfo files are stored under the separate directory corresponding to the first letter of the terminal identifier.
For example: The terminfo file for the terminal type vt100 is stored in the directory /usr/lib/terminfo/v/vt100 The file for the terminal type tvi912b is in /usr/lib/terminfo/t/tvi912b
terminfo files are compiled from a terminfo source file (an ASCII file) using the command tic. The format of terminfo source files will not be discussed here.
Some of the commands used by the terminfo database are listed in Table 10.7.
Command Purpose tic compile a terminfo source file infocmp list the source for a compiled terminfo entry infocmp -C translate from a terminfo file to the termcap format used by BSD systems captoinfo translate from termcap format to terminfo format
Table 10.7. terminfo Related Commands.
The BSD Terminal Database
BSD systems use one central text based file as the terminal database. The file is /etc/termcap. It contains colon delimited entries for each type of terminal the system recognises.
The first field of every entry is a list of terminal names (separated by |) by which the system recognises a particular terminal. These names are what appears as the value for the TERM variable and is used by the system to look up an entry.
The rest of the entry for a terminal consists of various options that describe the way in which the terminal works. The various options will not be discussed here. They are described in the manual pages for the system if needed.
It is advisable to put the entries for the most used terminals on your site at the front of the termcap file to speed searching.
David Jones (11/30/94) Page 181. Terminals Chapter 10
For example: vt100|dec-vt100|vt100-am|vt100am|dec vt100:\ :do=^J:co#80:li#24:cl=50\E[;H\E[2J:sf=2*\ED:\ :le=^H:bs:am:cm=5\E[%i%d;%dH:nd=2\E[C:up=2\E[A:\ :ce=3\E[K:cd=50\E[J:so=2\E[7m:se=2\E[m:us=2\E[4m :ue=2\E[m:md=2\E[1m:mr=2\E[7m:mb=2\E[5m:me=2\E[m :is=\E[1;24r\E[24;1H:if=/usr/share/tabset/vt100: \ :rs=\E>\E[?3l\E[?4l\E[?5l\E[?7h\E[?8h:ks=\E[?1h\ E=:ke=\E[?1l\E>:\ :ku=\EOA:kd=\EOB:kr=\EOC:kl=\EOD:kb=^H:\ :ho=\E[H:k1=\EOP:k2=\EOQ:k3=\EOR:k4=\EOS:pt:sr=2 *\EM:vt#3:xn:\:sc=\E7:rc=\E8:cs=\E[%i%d;%dr:
Exercises.
10.7. Determine the type of terminal you are using and examine the entry for your terminal that is stored in your system’s terminal database files. (On SysV based machines you'll have to use one of the commands listed in Table 10.5 to obtain the ASCII source.)
Conclusions
The process of adding terminals to a UNIX box is a reasonably straight forward process that can be plagued by strange and mysterious problems. Hopefully this chapter has introduced you to • the hardware and software processes for adding terminals to a UNIX machine, • the BSD and SysV configuration files for terminals, • the commands tty and stty, and • some of the problems that can arise while connecting a terminal.
The steps involved in connecting a terminal are • choose a port on the computer, • obtain the correct cable to connect the terminal to the port, • configure the terminal, • make and test the connection, • ensure that there is a correct entry for the terminal in the systems terminal configuration database, • ensure that the TERM shell variable is set correctly, • start the login process for the port.
David Jones (11/30/94) Page 182. Terminals Chapter 10
Review Questions
10.1. List the different types of configuration files used in controlling a terminal.
10.2. List the specifics of the configuration files for the terminals on your system.
10.3. List and describe the hardware steps involved in connecting a terminal to a UNIX machine.
10.4. Describe the steps involved when a user logs onto a UNIX machine.
David Jones (11/30/94) Page 183. Terminals Chapter 10
Chapter 11
PRINTERS
And I dream of the days when work was scrappy, And rare in our pockets the mark of the mint, When we were angry and poor and happy, And proud of seeing our names in print. -- G.K. Chesterton.
Objectives
By the end of this chapter you should • have been introduced to procedures for connecting printers to a Unix computer, • be able to list and explain the configuration files used by printers, and • have an understanding of how the BSD and SysV print systems work.
Introduction
Printers are a standard peripheral for any computer system. One of the first devices added to a new system will be a printer. The multi-user nature of the UNIX operating system means that the UNIX printer software is more complex than that of a single-user operating system. This makes adding a printer to a UNIX box more than just plugging it in.
UNIX print software performs a number of tasks including • enabling safe use of printers by multiple users, • supporting multiple printers, and • allowing the use of remote (network based) printers.
This chapter will first examine the hardware issues involved in connecting a printer to a UNIX machine before moving on to examine the more complex part of the process, configuring the software.
Hardware
In most situations printers are connected to a UNIX machine using serial connections. Many modern systems provide parallel ports. Generally speaking connecting a printer to a UNIX system follows the same generic process used to connect terminals that was outlined in Figure 10.1.
David Jones (11/30/94) Page 184. Terminals Chapter 10
Choose a Port
Typically you will have two choices with printers, either parallel or serial ports depending on your printer. The details of cabling for serial ports were discussed in the previous chapter.
Test the Connection
Performed the same way as that for terminals. Simply send output to the device file associated with the new device. Figure 11.1 outlines some possible reasons why it might not work as expected.
• Incorrect cabling • Running a getty process on the printer port. Especially on serial ports there might be a getty process running on the port that will cause the login prompt to appear on the paper. It will have to turned off. • Auto line feed Unix uses just the line feed character to separate lines. Most printers have a carriage return/line feed, auto line feed switch that controls what the printer expects to use. The problem can be fixed by using the stty command, by an interface program that performs the necessary translation on all output going to the printer, or by setting the printer dip switch to the appropriate setting. • Incorrect line settings For a serial port check the port settings espcially baud as it might be set too fast for the printer. On older printers the printer might not be configured for the ASCII character set. • Wrong device file or insufficient permissions Using the wrong device file or not having the correct permissions on that file can result in no output to the printer.
Figure 11.1. Possible Hardware Problems with Printer Connections.
Exercises.
11.1. If possible go through the hardware procedure for connecting a printer to your Unix box.
David Jones (11/30/94) Page 185. Terminals Chapter 10
Software
The software that drives the Unix printing process is another area in which the different Unix versions differ greatly. Both versions are based on the concept of spooling (spool stands for Simultaneous Peripheral Operations On-Line).
All UNIX print software has at least the following two components • a print spooler, and The print spooler is the program users execute when they wish to print something (usually the commands lpr or lp). The print spooler takes what the user wishes to print and places it into some pre-defined location, the spooling directory. • a print daemon. The print daemon (usually lpd or lpsched) is responsible for taking output from the spooling directory and sending it to the correct printer.
For every printer there will always be a maximum of one print daemon. This ensures that only one document is being printed on any printer.
Both SysV and BSD print services also support the concept of an interface program. This program acts as a filter through which all output sent to the printer is passed through. Example uses of an interface program include • adding a banner page, Most UNIX systems automatically add a banner page to the front of a print job. The purpose of the page is to identify the owner of the printer output. • adding or removing a line feed character, or • nothing at all.
The SysV Print Software
The SysV print system uses a number of definitions. Table 11.1 summarises some of them.
Term Definition device the actual physical printer printer the interface program/connection to the device class a group of similar printers known by one name
Table 11.1. System V Printer Definitions.
The entire spooling system is a complex system. The following diagram provides a simplification of the systems operation.
David Jones (11/30/94) Page 186. Information to be printed lp
accept / reject
Spooling Area
/usr/spool/lp
enable / disable
lpsched
printer class
device device device
Diagram 11.1. System V Print Spooling System.
lpsched is the name of the daemon that is continually running and provides the "power" for the print service. Only one copy of lpsched should be running at any one time. There should be a file SCHEDLOCK in /usr/spool which is responsible for ensuring only one copy runs.
lpshut turns off the print service by killing the lpsched daemon. Print jobs can still be spooled while lpsched is not running but they won't be printed. You should always run lpshut before using lpadmin.
Command Purpose accept/reject allows/disallows any further requests for a printer or class entering the spooling area cancel allows user to stop the printing of information enable/disable allow/disallow any more output from the spooler to the printer lpmove move print requests between destinations lp THE USER'S PRINT COMMAND, places information to be printed into the spooler lpadmin allows the configuration of the print service lpsched start the print service lpshut stop the print service lpstat display status of the print service
Table 11.2. System V Print Service Commands.
David Jones (11/30/94) Page 187. Steps For Connecting a Printer under SysV
The following summarises the steps involved in adding a printer to a SysV system. • perform the necessary hardware steps, • is the print service running? Execute the command ps -ae. One of the processes displayed should be lpsched. You may have to add lpsched to one of the system rc files. • decide on the interface program to use, Most will use the standard interface program available. Interface programs will probably be in /usr/spool/lp/model • add printer to the print service, and Execute the following commands to add the printer. Replace printer with your name for the printer, model the interface program you are using, device the device file for the port lpshut lpadmin -pprinter -vdevice -mmodel lpsched accept printer enable printer • test the printer. see if the printer works e.g. ls | lp
The BSD Print Software
The major components of the BSD print system are listed in Table 11.3. An overview of the system is provided by Diagram 11.2.
Component Purpose lpc make administrative changes to the print service lpd the daemon, a copy is spawned for each queue, transfers information from spooling area to physical device lpq view the contents of a print queue lpr the user print command, spools information to be printed /etc/printcap system's printer information database lprm removes print jobs from queues
Table 11.3. BSD Print Service Components.
David Jones (11/30/94) Page 188. Information to be printed lpr
Spooling Area Spooling Area lpc (different printers = (different printers = different spool area) different spool area) used to enable/disable spooling and to start lpd daemon
lpd lpd
printer printer
Diagram 11.2. BSD Print Service.
Each printer connected to a BSD system must have its own spooling directory that is serviced by one lpd daemon. Each printer has its own lpd daemon.
lpd accesses the file /etc/printcap for any information it requires concerning the printer. /etc/printcap is the system's printer database file and stores all the necessary information about ALL printers connected to the system. To use a printer, that printer must have an entry in /etc/printcap.
lpr is BSD's equivalent to lp under System V. If a user wishes to print something they use lpr. lpr sends this information to a specified spooling directory.
The process for adding a printer to a BSD system is somewhat similar to that for SysV.
Steps for connecting a printer under BSD
The following summarises the steps involved in adding a printer to a BSD system. • perform the necessary hardware steps, • check that the print service running, If it is running the command ps -xc should show up at least one process called lpd. If none appears then starting lpd must be added to to one of the systems rc files. • find or add an entry in the /etc/printcap file printcap stores specific information about the systems printers. The following section explains the format of the file in more detail. • add printer to the print service The process includes the following commands lpc enable printer_name start printer_name quit
David Jones (11/30/94) Page 189. • test the printer Try printing on the printer lpr -P printer_name /etc/passwd
/etc/printcap
This file contains a colon delimited entry for every printer connected to the system. The entry for each printer contains fields for • printer names, and The first field in an entry is a list of printer names separated by the | character that is used to identify a particular printer. The entry should include at least three names • a short name of no more than four characters, • a full name that specifies the printer type and owner, and • a descriptive name. At least one printer should be called lp. The printer lp is the default printer. • configuration settings
The configuration settings are used to specify a variety of different printer settings. Table 11.4 provides a list of some of the information specified in /etc/printcap.
The configuration settings can take one of three possible formats • XX=string • XX • XX#number
Where XX is a two letter identifier for the particular configuration setting.
For example: Specify the location of the spool directory for a printer sd=/usr/spool/lp/scribe Force a form feed when the device is first opened fo Specify the maximum size (in blocks) of files that can be printed mx#3
David Jones (11/30/94) Page 190. Setting Purpose sd=directory specify spool directory lf=file specify error log file lp=file specify device file af=file specify accounting file rw specify that printer can both read and write infomation (can send status info back to computer) br#number specify baud rate fc#number * specify flag bits to turn off fs#number * specify flag bits to turn on xc#number * specify local mode bits to turn off xs#number * specify local mode bits to turn on pl#number specify page length in lines pw#number specify page width in characters py#number specify page height in pixels px#number specify page width in pixels ff=string specify string that causes printer to form feed fo output form feed when device is opened mc#number specify maximum number of copies of a job allowed mx#number specify maximum file size in blocks allowed sc specify that multiple copies should be prevented sf specify that form feeds should be prevented sh suppress the printing of headers
* see explanation below Table 11.4. Configuration Settings for /etc/printcap.
Flag Bits
Flag bits are used to specify various communication settings for the printers. Table 11.5 shows the meanings and octal values of the more important bits. The flag bits that are to be turned on are specified using the fs identifier (see Table 11.4). Those flag bits to be turned off are specified using the fc identifier.
The values for these fs and fc are obtained by adding the octal values from Table 11.5 together.
For example: Clear all delay bits and echo/full duplex 0040000 + 0010000 + 0020000 + 0002000 + 0000400 + 0001000 + 0000010 = 0073410 printtab entry = fc#0073410
Set even and odd parity, enable automatic flow control. 0100 + 0200 + 0001 = 0301 printtab entry = fs#0301
David Jones (11/30/94) Page 191. Remember these numbers are in octal (base 8). If you don't know how to do addition in base 8 obtain a calculator which supports octal. Most good scientific calculators should.
Octal Value Description 0040000 form feed delay, 2 seconds 0010000 carriage return delay, 0.08 second 0020000 carriage return delay, 0.16 second 0002000 tab delay 0000400 newline delay 0001000 newline delay, 0.1 second 0000200 even parity 0000100 odd parity 0000040 pass all characters from filter to printer immediately 0000020 translate linefeed into carriage return&linefeed 0000010 echo, full duplex 0000002 pass characters from printer to filter immediately 0000001 automatic flow control
Table 11.5. Flag Bits for a Serial Printer.
Local Bits
Local mode bits are used to configure the serial driver and use the same format as flag bits only with xc and xs instead. Most of these settings are intended for terminals. Those relevant for printers are listed in Table 11.6.
Octal Value Description 000040 prevent serial driver from playing with codes destined for printer 040000 minimize flow control interference from line noise 000001 tell the printer to backspace when it receives an erase character
Table 11.6. Local Mode Bits for a Serial Printer.
The BSD Print Commands
The lpq Command
lpq displays the list of jobs that are currently waiting to printed on a specified printer.
lpq [-Pprinter][-l][+[interval]][job#...][username]
With no parameters lpq will display a list of all print jobs on the default printer.
David Jones (11/30/94) Page 192. Options Purpose -P printer display the queue of the specified printer -l display using long format +[interval] display the queue periodically until it empties, interval specifies how many seconds it should sleep job# display only those jobs with matching job numbers username display only jobs belonging to the specified user
Table 11.7. lpq Command Options.
The lpr Command
lpr is the command used to send information to be printed to the spooling area.
lpr [-Pprinter][-#copies] filenames
Options Purpose -Pprinter specify printer to send information to, by default this is lp -#copies produce the number of copies indicated
Table 11.8. lpr Command Options.
The lprm Command
lprm [-Pprinter][-][ job#...][username...]
lprm is used to remove jobs from the printer queue. It removes jobs matching printer, job# and username. The printer name defaults to lp and the job number defaults to the current job. Username defaults to the user invoking it.
Only the root user can remove someone else’s print job.
The lpc Command
lpc is used to disable and enable the lpd daemon, disable and enable spooling and various other administrative duties on printers.
lpc [ command [ parameter.. ]]
If no commands are given lpc will enter interactive mode and present the lpc> prompt at which commands can be given.
David Jones (11/30/94) Page 193. Table 11.9 lists some of the commands that can be given to lpc. There are a number of other commands, refer to your manual page. These commands can be given at both the command line and from the lpc> prompt.
Command Purpose ? [command] help [command] provide short description of command abort [all | printer ] terminate the daemon and then disable printing for the specified printers. enable [all | printer ] start spooling for the specified printers start [all | printer ] start printing for the listed printers stop [all | printer ] stop a spooling daemon and disable printing
Table 11.9. lpc Command Options.
Exercises.
11.2. Complete the software process for connecting your printer to your Unix box.
Conclusions
The process of adding a printer to a UNIX machine involves two processes, hardware and software. The hardware steps involved in adding a printer are very similar to those involved in adding a terminal.
The UNIX print software is much more complex than that of a single-user operating system and is based on the concept of print spooling. The print services of BSD and SysV are completely different.
Review Questions
11.1. Describe the print software of your machine.
11.2. What is the purpose and relationship between the lp and lpr commands?
11.3. What is the purpose and relationship between lpd and lpsched?
11.4. What is the purpose of the /etc/printcap file?
David Jones (11/30/94) Page 194. Chapter 12
UNIX NETWORK BASICS
Network: Anything reticulated or decussated at equal distances, with interstices between the intersections. A Dictionary of the English Language (1755)
Objectives
The aim of this chapter is to • introduce you to the basic concepts of networking • provide an explanation of what TCP/IP is and how it works • define the terms hostname, network addresses, DNS, routers, gateways, port numbers, domain name resolution • introduce various TCP/IP protocols including SMTP, FTP, telnet • introduce you to some of the network daemons • explain commands like ping netstat • introduce you to various network configuration files including /etc/hosts /etc/services /etc/inetd.conf
Introduction
In this day and age it is rare for a UNIX machine to be totally unconnected to other machines. In the commercial environment one of the major uses of UNIX is as a network file server. Networks are only going to become more pervasive as time progresses. The System Administrator is typically involved very closely with the network, either through setting up machines to talk to the network or even having to look after the network. This and the next chapter will introduce you to the intricacies of UNIX networking.
What is a Network?
A network is simply a collection of machines connected in some way that allows them to communicate with each other and share information. To do this the machines have to be connected in some way that allows communication, and have an agreed upon a language to talk when they do communicate.
Characteristics of a network include • individual hosts The network is a collection of individual machines sometimes referred to as hosts. Each host must have some unique identifier that allows other hosts to talk to it.
David Jones (11/30/94) Page 195 • communications connection To communicate the hosts must be connected in some way that allows communication. Typical physical connections include ethernet, token ring, serial line, modems and various other formats. • network protocol In order to communicate the parties must speak the same language. Languages on computer networks are referred to as network protocols. A network protocol is simply a set of rules and formats that govern how information is sent and in what format it is sent. Some of the different network protocols used today include TCP/IP (Internet and UNIX favourite), IPX (Novell), Appletalk (guess who?), DECnet and various others. • network services To be of use to users the network will provide various services including file, print and device sharing, electronic mail etc.
There are at least two categories in which networks are placed • LAN (local area network) All the hosts in the network are directly linked to each other. • WAN (wide area network) Much larger than a LAN and all machines are not directly connected. WANs will generally have a lower throughput than a LAN.
Separate LANs can be connected together to form a WAN as show in Diagram 12.1.
LAN A LAN B
a1 a2 b1 b2
a3 a4 b3 b4
a5 a6 b5 b6
LAN D
c1 c2 d1 d2
c3 c4
c5 c6
LAN C
Diagram 12.1. LANs and WANs.
David Jones (11/30/94) Page 196 Some Definitions
Like any field of computing, networking has its own terminology. This section provides definitions for some of the terms you'll come across. • packets and datagrams, Many networking protocols transmit information as packets. Information being sent across the network is divided into small (hundreds of bytes usually) chunks of data, called packets, that are put back together by the network protocol at the receiving machine. Under TCP/IP packets are referred to as datagrams. • routing, The art of making sure that when machine a1 (from Diagram 12.1) sends a packet to machine c6 it actually gets there and if at all possible in the most efficient manner. In some cases the most efficient route must be chosen from multiple different routes. • bridge, A device that is used to join more than one network (e.g. machine c2 from Diagram 12.1). All the machines on LAN C can transmit packets directly to all the machines on LAN C. But if they send a packet to LAN D it must be passed across the bridge machine. Networks joined by bridges typically use different networking hardware. A bridge could be a computer or a specially designed device. • router, Responsible for performing the routing for a network, a router is typically a device that connects multiple networks together. (The terms router, gateway and bridge are sometimes used interchangeably). • gateway, and A device that connects two totally different types of network. For example you might have a gateway machine sitting between an IPX network and a TCP/IP network. The gateway allows the two different networks to talk to each other by converting the protocols they are using. • ethernet. The most common form of network hardware used in LANs (especially with UNIX boxes).
TCP/IP and the Internet
Most versions of the UNIX operating system comes with in-built support for networking. The default network protocol that UNIX systems are typically designed to talk is TCP/IP. TCP/IP is more correctly called the Internet Protocol Suite.
TCP/IP is the protocol used by the Internet, a network of networks spread throughout the world connecting two million machines with over twenty million users. It is not necessary to be connected to the Internet to use TCP/IP. (However being able to connect to the Internet is one of the advantages of using TCP/IP.)
David Jones (11/30/94) Page 197 The Protocols of TCP/IP
TCP and IP are two of the many protocols that make up the suite of Internet protocols. Table 12.1 lists some of the others. Some of these protocols will be discussed in detail in the next chapter. Diagram 12.2 displays the four different layers of TCP/IP and where some of the protocols fit within those layers.
TCP/IP is an example of a layered communications suite. The advantage of this layered approach is that the protocols at higher levels can safely assume that the lower level protocols will carry out their responsibilities. For example TCP does not need to know anything about the hardware being used because it is hidden by the layers below it.
4 Application Layer FTP TELNET SMTP RIP DNS NFS RFC-959 RFC-854 RFC-821 RFC-1058 RFC-1035 RFC-1094
EGP RFC-904
3 Host to Host Transport Layer
TCP UDP RFC-793 RFC-768
2 Internet Layer
IP ICMP RFC-791, RFC-792
1 Network Access Layer
Hardware, ethernet, serial line, token
Diagram 12.2. Four Layers of TCP/IP.
TCP/IP uses four layers • network access layer Responsible for the transformation of IP datagrams into a format that can be placed onto the hardware. • internet layer The Internet Protocol is the heart of this layer. Functions IP performs include defining the IP datagram and addressing scheme, and routing datagrams between remote hosts. It provides a well-defined protocol that from the higher layers (transport, application) will appear the same regardless of the hardware. This means that the transport protocols do not have to know about the different types of hardware since the Internet layer hides the hardware dependencies. • transport layer The transport layer is responsible for dividing (and putting back together) the user data into correct size datagrams for network transmission, and providing any additional transport controls such as error detection and reliable delivery.
David Jones (11/30/94) Page 198 • application layer Protocols in the application layer add additional functionality by building on the transport layer.
In most instances you will come across TCP/IP running on an ethernet based local area network. However TCP/IP is not limited to ethernet It can run over various different types of communications hardware including modems, serial lines, fibre optic cable, microwave and satellite links.
Serial line internet protocol (SLIP) and Point to Point Protocol (PPP) are two protocols that allow TCP/IP to be run over serial lines and modems.
Transport Protocols
The protocols TCP and UDP are the major transport protocols of the Internet protocol suite. The majority of the higher level application protocols use one of TCP or UDP. TCP and UDP differ in much the same way as the telephone and postal services.
A TCP connection is like a telephone connection. Before any information is sent the two ends connect and make a channel (sometimes referred to as a stream) and all information sent between the two hosts is sent down that stream. TCP performs checks that make sure that all the information sent down the channel actually arrives at the destination in the same order as it was sent.
UDP is like the postal service in that all information is sent in individual messages (called packets or datagrams). These datagrams do not always take the same route when going from one point to another.
UDP is an unreliable, connectionless protocol. TCP is a reliable, connection oriented transport protocol.
UDP is used to send small infrequent messages while TCP is used in situations where large amounts of information must be transferred safely.
Application Protocols
At the top layer of the TCP/IP protocol stack are the application protocols. These protocols specify higher level application specific protocols. Examples include smtp (the simple mail transfer protocol) that specifies how mail messages are to be exchanged.
David Jones (11/30/94) Page 199 Protocol Full Name Purpose SMTP Simple Mail Transfer defines the format of mail messages Protocol and the commands used to send and receive mail messages Telnet telnet defines a protocol used to login remotely to other machines on the network TFTP FTP (Trivial) File Transfer protocol for transferring files between Protocol two machines TCP Transmission Control a connection oriented, reliable, full- Protocol duplex byte stream service UDP User Datagram Protocol connectionless, unreliable datagram service IP Internet Protocol ensures that individual datagrams get to their destination ICMP Internet Control Message used to send the error/control Protocol messages for the TCP/IP software ARP/RARP (Reverse) Address performs the translation from Internet Resolution Protocol addresses to ethernet addresses (specific to hardware)
Table 12.1. Some of the Internet Protocols. RFCs
The standards used on the Internet are specified in documents called Request for Comments (RFCs). Someone proposing a new standard will write and submit an RFC. The RFC will be distributed to the Internet community who will comment on it and may suggest changes. The standard proposed by the RFC will be adopted as a standard if the community is happy.
Diagram 12.2 includes RFC numbers that correspond to the different protocols. If you have access to the Internet you can anonymously ftp RFCs from a number of well-known sites (including archie.au).
RFCs can and often are very technical and hard to understand unless you are familiar with the area (the RFC for ftp is about 80 pages long).
Names and Addresses
Every machine on a network has to have a unique identifier. This unique identifier is used by the network protocol to deliver information to the host and used by the network users so they know which machine they are using.
David Jones (11/30/94) Page 200 TCP/IP provides two types of unique identifier for every host • host address This is the unique identifier that the protocols use to deliver packets to the correct host. For example the host jasper at CQU has the address 138.77.1.1. All network communication uses the host address. • host name Host names are meant to provide a unique identifier that is easier for the users of the network to remember. For example the address 138.77.1.1 refers to the machine with the host name jasper within the domain cqu.edu.au. Names are easier for humans to remember than numbers are.
Addresses
TCP/IP addresses are currently 32 bit numbers that are usually represented as four 8 bit (octets) numbers separated by full stops e.g. 132.22.42.1. Using 8 bits the maximum range that can be represented is 0-255 (256 numbers).
Since IP is the protocol designed to route information to a particular host this address is sometimes referred to as the IP address.
Every machine on a network must have a unique IP addresses. When setting up a LAN this can be reasonably simple and IP addresses can be assigned by the Systems Administrator.
However the Internet is classed as one network so every machine on the Internet must have a unique IP address. IP addresses for a LAN are assigned by a central authority to ensure that no two machines have the same address.
This central authority generally does not (currently) hand out individual IP addresses. A site will usually apply for a network address (for example, CQU’s network address is 138.77). This network address provides you with a set of IP addresses that can be used for the entire LAN. There are currently three types of network address being handed out. The type of network addresses assigned depends on the size of a site.
There are three network address classes that are assigned to sites, classes A, B and C (there are two others but they aren't normally used). Each class has a maximum number of machines that can be connected. Table 12.2 lists the types of network addresses and their characteristics.
The current problem with Internet addressing is that there isn't a network class for sites that have more than 254 hosts but less than 64,000 hosts (mainly medium size companies).
David Jones (11/30/94) Page 201 Address Class Value first byte Number of Hosts A1 to 12616 million B128 to 19164,000 C 192 to 223 254
Table 12.2. IP Address Classes.
The Central Queensland University has been assigned the Class B address 138.77. Any machine that has an IP address starting with 138.77 is part of the University network.
Some network addresses are reserved for specific purposes • 127.0.0.1 This is the loopback address. Data sent to the address 127.0.0.1 is sent straight to the machine sending the information (it refers to the current host). If you are not connected to a network this is probably the only IP address you can use. • network.255 and network.0 (where network is the network address) These are used for broadcast addresses. Sending information to 138.77.37.255 causes every machine on the network 138.77.37 to receive the information.
Names
IP addresses are fine for machines but the users also need a mechanism by which they can identify machines and trying to remember 138.77.1.1 is not easy. For this reason machines using TCP/IP also have host names.
A host name under TCP/IP follows the format hostname.site.domain.country. • hostname A name by which the machine is known. This name must be unique to the site to which it belongs. • site A name given to the site (company, University, government department etc) on which the machine resides. • domain Each site belongs to a specific domain. A domain is used to group sites of similar purpose together. Table 12.3 provides an example of some domain names. • country Specifies the actual country in which the machine resides. Table 12.4 provides an example of some country names.
David Jones (11/30/94) Page 202 For example the CQU machine jasper has the fully qualified name jasper.cqu.edu.au, where jasper is the hostname, cqu is the site name, the domain is edu and the country is au.
Domain Purpose edu Educational institution (University etc) com Commercial company gov Government department
Table 12.3. Example Domain Names.
Country Code Country nothing United States au Australia uk United Kingdom in India ca Canada fr France
Table 12.4. Example Country Codes. jasper.cqu.edu.au is a fully qualified name and uniquely identifies the machine jasper on the CQU campus to the entire Internet. There cannot be another machine called jasper at CQU. However there could be another machine called jasper at James Cook University in Townsville (its name would be jasper.jcu.edu.au). A fully qualified name must be unique.
It is not always necessary to specify a fully qualified name. If a user on aldur (see Diagram 12.3) enters the command telnet jasper it will be assumed that because it isn't fully qualified the user means the machine jasper within the current domain (cqu.edu.au). Name Resolution
IP always uses the IP address and not the name when it is sending information. Users on the other hand can choose which scheme they wish to use. A hostname is always resolved to its corresponding IP address before the command is executed. For example the commands telnet jasper.cqu.edu.au and telnet 138.77.1.1 have exactly the same effect (to connect to the machine with the IP address 138.77.1.1). How is the resolution performed?
David Jones (11/30/94) Page 203 The TCP/IP software performs name resolution. It accepts the name jasper.cqu.edu.au and then resolves that into the machine’s IP address. There are two methods that can be used to perform this translation from hostname to IP address • the /etc/hosts file, and • the Domain Name Service.
The /etc/hosts File
A UNIX machine maintains a text file /etc/hosts that has one line per host. Each line is of the format IP_address hostname . For example the hosts file of the machine aldur looks like this 127.0.0.1 localhost loopback 138.77.36.29 aldur aldur.cqu.edu.au 138.77.1.1 jasper jasper.cqu.edu.au 138.77.37.28 pol pol.cqu.edu.au
When a user on aldur enters the command telnet jasper.cqu.edu.au the software first looks in the hosts file for an entry for jasper. If it finds an entry it obtains jasper's IP address and then can execute the command.
If it can't find an entry for a machine there are two options • failure Report back to the user unknown host or some similar error. • make use of DNS Domain Name Service is the other method the Internet uses to perform name to address translation
The DNS
The fact that there are over two million machines connected to the Internet implies the possibility of some very big /etc/hosts files. In fact there is no machine on the Internet that has a hosts file that contains entries for all the machines on the Internet. Not only would the files be huge there is no easy way to keep them up to date.
For example: How does an administrator in Rockhampton, Australia know when a small company in Bolivia goes bankrupt and all its machines are sold and are no longer on the net?.
DNS is a hierarchical hostname management system that allows each site (or part of a site) to maintain its own IP address information without having to inform some central site.
David Jones (11/30/94) Page 204 Instead each network maintains its own authoritative name server that can perform the translation from hostname to IP address for that network. If a local machine changes its IP address then only the local name server is updated. Other sites will know how to contact the networks name server by using the DNS.
For example: At CQU the Mathematics & Computing Department might maintain a name server for its network (knuth). Up the hierarchy CQU may contain a name server for the entire University (jasper). jasper would now that the name server for the Maths & Computing network is knuth. jasper would not know anything about the Maths & Computing network.
Moving further up there might be a name server for all the Queensland Universities that would know the name servers for each individual Queensland University.
If someone on a machine at Queensland University types the command telnet aldur.cqu.edu.au the user's machine must obtain the IP address for aldur.cqu.edu.au before it can carry out the command. It will do this using the following steps • look in its /etc/hosts file, If users on this particular machine connect to aldur.cqu.edu.au quite often it might have been put in the hosts file by the administrator. But typically it would not be there. • ask its name server, If it isn't in the hosts file the name server for the Queensland University site would be asked. It would probably not be there. However this name server would contain a pointer to the name server next up the hierarchy. • ask the name server for Queensland Universities, This name server will generally contain the address for the name server of CQU to which the request will be passed • ask CQU's central name server (jasper), and jasper will ask the Maths & Computing name server knuth. • knuth will perform the hostname to IP address translation. knuth will contain the necessary information which it will pass back up the chain.
The requesting machine will keep a cache that contains the name/address translation for aldur. This is so the next request for aldur does not have to follow the above steps. This in cache copy will be lost when the power is lost.
David Jones (11/30/94) Page 205 A machine specifies its name server in the file /etc/resolv.conf. The resolv.conf file for aldur might contain nameserver 138.77.36.1 # Maths & Computing nameserver nameserver 138.77.1.1 # CQU name server
Notice that the machines in the resolv.conf file are specified by their IP address and not their hostnames?
Exercises.
12.1. You are logged into the machine pol.cqu.edu.au and you enter the command telnet sunsite.unc.edu but you get a message host not found. What might have gone wrong (there are a number of reasons)?
Routing
Routing is the act of deciding how each individual datagram sent finds its way through the multiple different paths to its destination. What follows in this section is a simplified explanation of that process.
TCP/IP is based on a model in which smaller individual networks are connected together to form larger networks. Diagram 12.3. is a small section of the network at the Central Queensland University.
The CQU campus is divided into a number of separate networks (LANs) all using ethernet. Each separate network has its own network number. For example the Maths & Computing Department network is 138.77.36. All machines in the Department have IP addresses that start with 138.77.36.
Each individual network is limited to 254 machines because two of the numbers 0 and 255 are reserved for broadcast addresses.
Any machine on an ethernet LAN can send information straight to another machine on that LAN. When a machine places a packet on an ethernet network every host on that LAN can see that information. However they will ignore it unless it is specifically addressed for them (or they have been programmed to look at everything).
David Jones (11/30/94) Page 206 Aldur 138.77.36.29
Maths & Computing Staff Network (138.77.36.*) Knuth 138.77.36.1
138.77.37.1
IT Building Student Campus Fibre Backbone Network (138.77.37.*)
138.77.1.1 Pol Jasper 138.77.37.28
ITD Network (138.77.1.*)
Fibre to AARNET and the Internet
Diagram 12.3. Part of the CQU Campus Network.
For example when the machine aldur (138.77.36.29) from diagram 12.3 wants to send some information to the machine 138.77.36.1 it places the packets on the ethernet. All the machines on the LAN will see the information but only 138.77.36.1 will accept the information.
When a packet is placed onto ethernet every machine on the network is able to view that packet. In practice only the machine to which it is addressed will ask for the packet. However it is possible to set up machines to accept all packets placed onto the net, not just the packets destined for it. In this way criminals or idiots can intercept information they were never meant to have. This is one method of obtaining passwords and other confidential information. The solution is implement some form of encryption so that even if they obtain the information they can’t use it.
Figure 12.1. Security Problem with Ethernet.
For two machines that reside on the same physical network both machines will know the address of the other and the exchanging of information is fairly simple. But the Internet is a collection of hundreds and hundreds of connected companies and institutions, many of which are at least as large as the CQU net. Each of these networks is in turn made up of sub-nets which in turn have hundreds of machines connected.
David Jones (11/30/94) Page 207 How does information find its way from one machine to another? It is obvious that every machine connected to the Internet CANNOT know about the address or the route to every other machine.
The gateway machine is basically the connection from one network to another. It is convention that the gateway machine (it might be a computer or a purpose built device) be the first machine on the network. For example the gateway machine for the Maths & Computing LAN is 138.77.36.1.
If a user on aldur wants to send some information to the machine pol it would place the information onto the ethernet with an address 138.77.37.28 (after already going through the name/address translation scheme outlined above). All the machines on the LAN would ignore the packet (because of the address) except the gateway machine. It would see that the information is meant for another network.
The gateway machine would pick that packet up and place it onto the outside net (in this case the campus fibre backbone). All the machines connected here would ignore it except the gateway machine for the 138.77.37 network that would recognise the address as one of its own. The gateway machine 138.77.37.1 would then place the packet onto the ethernet for the 138.77.37 network and the machine with the corresponding address (pol) would recognise it and snaffle it.
The process works the same way if you are sending information across the world.
Exercises.
12.2. What would happen if a user on aldur wanted to send information to the machine sunsite.unc.edu?
Hardware Routing
When a host recognises a packet as its own (as is mentioned in the above section) it isn't actually using the IP address for recognition. When using ethernet as the hardware medium the ethernet address will be used.
Every ethernet card has built into it a 48 bit address (called an Ethernet address or a Media Access Control address). The high 24 bits of the address are used to assign a unique number to manufacturers of ethernet addresses and the low 24 bits are assigned to individual ethernet cards made by the manufacturer.
Every packet of information sent on ethernet contains a source and destination MAC address. If the card in your machine recognises the destination MAC as its own it passes the packet to the protocol in the next layer (refer to Diagram 12.2).
David Jones (11/30/94) Page 208 ARP, the address resolution protocol, maintains tables of translations between IP address and Ethernet address. These tables are dynamic and are added to as different machines are contacted. UNIX supplies a command arp that can be used to examine these tables. arp -a will list the current entries. Refer to your manual pages for more information.
For example:
bash$ arp -a Address HW type HW address Flags 138.77.37.1 10Mbps Ethernet 00:00:0C:03:79:2F C bash$ telnet bertha . . bash$ arp -a Address HW type HW address Flags 138.77.37.1 10Mbps Ethernet 00:00:0C:03:79:2F C 138.77.37.37 10Mbps Ethernet 00:40:F6:60:4D:A4 C
Exercises.
12.3. If you are connected to a LAN use the arp command to examine the tables of your machine.
12.4. Use a command to connect to a machine not listed in the tables and reexamine the arp tables to see if it has been added.
Network Services
We've looked at how different hosts are identified and taken a simple look at how information is routed from one host to another. In this next section we look at how the various network services are provided. When you telnet to another machine, how does it work? When you send an e-mail message to a user at another host, how is it delivered?
The provision of network services like ftp, telnet, e-mail and others relies on three different components • network ports, Network ports are the logical connections through which the information flows into and out of a machine. A single machine can have thousands of network ports all sending and receiving information at the same time. • network daemons, Network daemons are the programs that sit listening at pre-defined ports waiting for connections from other hosts. These daemons are in charge of fulfilling the request. • network client programs, and Users access network services using client programs. For example to connect to a remote machine using the telnet protocol a user will execute a telnet client program.
David Jones (11/30/94) Page 209 • network protocols. Network protocols specify how the daemons are to communicate. For a machine to send mail to another they must talk the same protocol (under TCP/IP this is SMTP, simple mail transfer protocol) so they can exchange mail messages.
Network client programs and network daemons are completely different programs and will generally use different ports.
For example: What happens when a user types the following command? telnet jasper First they are running the telnet client program. This program will get the IP address of the machine to connect to, connect to a port on the local machine, and then contact the telnet daemon on the remote machine. The inetd daemon will be waiting on a specified port (23) for a connection request. When it gets one it will start up the telnet daemon.
jasper
Port 1467 telnet client program
TCP connection
Port 23 aldur Incoming telnet port defined in /etc/services
Diagram 12.4. Example telnet Connection.
The following are commands taken from the actual machines to demonstrate the above diagram. The netstat command is discussed later in this chapter. However you can see the port numbers that are being used (jasper.telnet means that the telnet port is being used) aldur$ telnet jasper ....login procedure onto jasper jasper$ netstat | grep aldur Active Internet connections these two header lines have been added for understanding Proto Recv-Q Send-Q Local Address Foreign Address (state) tcp 0 0 jasper.telnet aldur.cqu.EDU.AU.1467 ESTABLIS back on aldur aldur$ netstat Active Internet connections Proto Recv-Q Send-Q Local Address Foreign Address (state) tcp 0 0 aldur.cqu.EDU.AU.1467 jasper.cqu.EDU.A.telne ESTABLIS
David Jones (11/30/94) Page 210 Ports
Any packet sent on a TCP/IP network is identified by five individual bits of information • source and destination IP address, • source and destination port number, and At any one time a machine is likely to have multiple programs sending and receiving information to and from the network (multiple users all using telnet or ftp). How can multiple programs be using the network at once? How does the machine know which packet is for which program? The answer is the port number (explained below). • transport protocol being used. Usually either TCP or UDP.
Every program that talks to the network is assigned a port number. It is through this port that all information flows from the computer to the outside world and vice versa (some programs might have two ports, one for outgoing and one for incoming). An individual computer will have approximately 64,000 ports to use (the port address is a 16 bit number). Some of these ports are used for predefined purposes. The ports 0-256 are set aside for well known Internet services (e.g. telnet, FTP, SMTP). Ports in the range from 256-1024 are used for UNIX specific purposes and anything above can be used by user programs.
Different protocols use predefined ports on which they listen for incoming calls. For example if you telnet to another machine your machine first contacts the process listening on port 23 of the remote machine to negotiate the connection. Table 12.5. lists some of the other well known ports.
Port Number Purpose 20 ftp-data 21 ftp 23 telnet 25 smtp (mail) 119 nntp (network news)
Table 12.5. Reserved Ports.
Which port is used by which protocol is stored in the file /etc/services. Each line in the services file is of the format service-name port/protocol aliases, where service- name is the official name, port is the port number that it listens on, protocol is the transport protocol it uses and aliases is a list of alternate names.
David Jones (11/30/94) Page 211 For example: The following are extracts from an example /etc/services file echo 7/tcp echo 7/udp discard 9/tcp sink null discard 9/udp sink null systat 11/tcp users daytime 13/tcp daytime 13/udp ftp-data 20/tcp ftp 21/tcp telnet 23/tcp smtp 25/tcp mail nntp 119/tcp usenet # Network News Transfer ntp 123/tcp # Network Time Protocol
You should see the similarities between the numbers listed in this listing and those in Table 12.5.
Network Daemons
The various network services provided are usually supplied using a variety of network daemons. For a particular service to be available (e.g. telnet ftp mail etc) the correct daemon must be listening to the correct port. Table 12.6 contains a list of some of the network daemons (there are many others).
Daemon Purpose Name inetd Internet services daemon used to manage most of the TCP/IP daemons. Configured using the /etc/inetd.conf file rwhod periodically sends information to every other host on the network about who is logged in, how long they've been logged in etc. (used by the rwho and ruptime commands) routed maintains dynamic routing information named used by the DNS to contact other machines (see previous section) timed used to ensure that the system clocks on all machines are synchronised nfsd handles requests for the network file system biod used by NFS clients to do read ahead and write behind smtpd accepts incoming mail messages telnetd accepts incoming telnet requests
Table 12.6. Various Internet Daemons.
David Jones (11/30/94) Page 212 For a particular protocol to work there must be a daemon running that is listening to the specified port to accept the information. There are two ways in which a daemon might be started • executed as a normal program (usually in the startup files) These should show up in a ps list of all the current running processes and are always waiting for some information to arrive at the specified port. One example is the smtpd daemon (it might be called smail on some systems). • by the inetd daemon Most network daemons are not executed until information arrives for them. The inetd daemon listens at a number of ports and when information arrives, it executes the daemon for that port that is specified in its configuration file /etc/inetd.conf.
To disable a particular protocol all you do is ensure that its daemon is not running. There are two basic methods for disabling a daemon • remove its entry in the startup file • remove its entry in the inetd.conf file
The /etc/inetd.conf File
The /etc/inetd.conf file specifies the daemons that the inetd daemon should execute when a request for a specific protocol has arrived. The inetd.conf consists of one line per network service using the following format
service-name socket-type protocol flags user server_program args
service-name corresponds to the name of the network service and is the same as the one listed in /etc/services. user is the username to run the program as. The following is from the inetd.conf file of one machine.
ftp stream tcp nowait root /usr/sbin/in.ftpd in.ftpd telnet stream tcp nowait root /usr/sbin/in.telnetd in.telnetd nntp stream tcp nowait root /usr/public/bin/nntpd nntpd
Whenever the machine receives a request on a port the inetd daemon will examine the inetd.conf file to determine the program that should be executed to respond to the request. For example a request on port number 21 (taken from the /etc/services file) will be identified as a telnet request and the program to run for telnet is /usr/sbin/in.telnetd (taken from the /etc/inetd.conf file).
David Jones (11/30/94) Page 213 Exercises.
12.5. The smtp protocol is used to handle electronic mail messages. Examine the entry for smtp in the /etc/services file. smtp must have a daemon program running to respond to requests. Find out where it is started on your machine.
Network Protocols
Each network service generally uses its own network protocol that specifies how the information is transferred. For example the ftp network protocol defines the commands that can be used to move files from machine to machine. The smtp network protocol specifies the format of mail messages and how they are moved from one machine to another.
The format of these Internet protocols is laid down in documents referred to as request for comments (RFCs). Each protocol will have its own RFC that describes it. RFCs are available off the network from a number of anonymous ftp sites including archie.au and ftp.lcs.mit.edu.
Some of these protocols smtp, ftp and nntp, are text based. They make use of simple text based commands to perform their duty. Table 12.7. contains a list of the commands that smtp understands.
Command Purpose HELO hostname startup and give your hostname MAIL FROM:sender-address start transaction from sender RCPT TO:recipient-address name recipient for message VRFY address verify deliverability of address EXPN address expand mailing list address DATA start text of mail message RSET reset state, drop transaction NOOP do nothing DEBUG [level] set debugging level, default 1 HELP produce a help message QUIT close SMTP connection
Table 12.7. SMTP Commands.
When transferring a mail message the two daemons will connect and carry out a conversation using the above commands. Since these are straight text commands it is possible for a user to pretend to be a daemon.
David Jones (11/30/94) Page 214 For example: The following is the start of an example conversation between a smtp daemon and a normal user. The text in this font are explanations. bash$ telnet aldur 25 connect to the smtp port (see /etc/services) Trying 138.77.36.29 ... Connected to aldur.cqu.edu.au. Escape character is '^]'. 220 aldur.cqu.edu.au Amix Smail3.1.28.1 #2 ready at Sun, 28 Aug 94 12:04 EST helo aldur tell the machine who I am (the name of another machine not a user) 250 aldur.cqu.edu.au Hello aldur mail from: [email protected] this is who the mail is coming from 250
It doesn't take a genius to see the application some immature or morally corrupt people might use this "feature" for.
The above example makes use of a "useful" feature of the telnet command. By default the format for telnet is telnet hostname. When executed like this it connects to port 23 of the remote host since that is where the telnet daemon is listening. However it is possible to tell telnet to connect to another port using the format telnet hostname port-number
Exercises.
12.6. Try doing the same task for ftp as was done above for smtp. What is the port number for the ftp daemon?
Some Useful Commands
This final section lists some of the commands that will be useful to you as both a Systems Administrator and a user.
netstat
The netstat command provides various types of information about the status of your network including • a list of all active connections By default it provides a list of all active connections into and out of your machine.
David Jones (11/30/94) Page 215 For example: bash$ netstat Active Internet connections Proto Recv-Q Send-Q Local Address Foreign Address (state) tcp 0 0 aldur.cqu.EDU.AU.login jasper.cqu.EDU.A.1018 ESTABLISHED tcp 0 0 aldur.cqu.EDU.AU.1796 *.* LISTEN tcp 0 0 aldur.cqu.EDU.AU.ftp aldur.cqu.EDU.AU.1794 CLOSE_WAIT tcp 0 0 aldur.cqu.EDU.AU.1794 aldur.cqu.EDU.AU.ftp FIN_WAIT_2 tcp 0 0 aldur.cqu.EDU.AU.1025 *.* LISTEN tcp 0 0 aldur.cqu.EDU.AU.2767 *.* LISTEN tcp 0 0 aldur.cqu.EDU.AU.liste *.* LISTEN tcp 0 0 aldur.cqu.EDU.AU.print *.* LISTEN udp 0 0 localhost.domain *.* udp 0 0 aldur.cqu.EDU.AU.domai *.*
• net statistics Statistics about the amount of information being sent on the network, the number of collisions etc. For example: display statistics every 5 seconds bash$ netstat 5 input (aen0) output input (Total) output packets errs packets errs colls packets errs packets errs colls 0 273435 275709 0 273435 274 273435 275983 0 273435 0 1 1 0 1 0 1 1 0 1 0 1 1 0 1 0 1 1 0 1 0 1 1 0 1 0 1 1 0 1 • routing information Display the route tables that are used to distribute information (netstat -r).
ping
The ping command is a simple utility that provides a simple test of whether or not you can talk to another machine. The format of the ping command is simply ping host and the output generally takes one of two formats • verbose In verbose mode ping continually sends a packet of information from your machine to the destination and reports back the IP address it is sending, how long the information took, and various other stats. • not verbose In this format ping reports back a simple message indicating whether or not the destination machine is reachable.
There are two cases where ping might fail • the machine is not reachable There is no network connection between your machine and the destination for it to use (e.g. someone has put a back hoe through the fibre linking the two machines) • the machine is not up The destination machine is not turned on or not responding to the network for some reason.
David Jones (11/30/94) Page 216 Exercises.
12.7. Use the netstat command to find out the status of your network (if you are connected to one).
12.8. Use the ping command to test a connection to a destination machine. Try to get both output formats discussed above (if you are not on a network try pinging the local host).
Conclusions
Learning how to manage a network is a subject in itself. In this chapter you have hopefully been introduced to the basics of UNIX and TCP/IP networking, including • some basic concepts and definitions like gateway, router, network protocol, bridge etc. • a look at the protocols that make up the Internet protocol suite • discussion of IP addresses, host names and the resolution from one to the other • how IP routes information • an introduction to the commands arp ping netstat • the concept of ports and network daemons
Review Questions
12.1. Explain the terms gateway, router, port, protocol, daemon and bridge.
12.2. People are unable to telnet to the machine hades. The machine is connected to the network, turned on and every other network service is working. What could be wrong?
12.3. What is the purpose of TCP and UDP?
12.4. Explain how the hostname aldur might be resolved to its IP address.
12.5. Explain the relationship between the files /etc/services and /etc/inetd.conf
12.6. What are the four layers used by TCP/IP?
David Jones (11/30/94) Page 217 Chapter 13
MORE NETWORKING
The only means of strengthening one’s intellect is to make up one’s mind about nothing - to let the mind be a thoroughfare for all thoughts. -- John Keats.
Objectives
This chapter aims to extend your knowledge of networking in the UNIX environment. It will examine the following topics • the Berkeley r commands including rcp, rsh, rlogin, • the concepts of host and account equivalence and the security implications of using the Berkeley r commands, • setting up and using the network file system (NFS) to share directories and files transparently between machines, • the process of adding a host to an existing network, and • a variety of commands and tips for debugging a network.
Introduction
The major aims of implementing a network are • to allow communication between users and hosts, and • to allow the sharing of computing resources such as printers and disk drives.
This chapter will introduce you to some of the commands and software protocols that enable a UNIX machine to fulfil these aims. In addition it will examine how you add a machine to an existing network and some of the tools that can be used to diagnose problems and observe the state of a network.
This chapter and the last chapter combine to provide an introduction to the topic of networking under the UNIX operating system. The entire area is much more complex and deserves a subject in its own right.
David Jones (11/30/94) Page 218 The Berkeley r Commands
A user with accounts on two or more machines would like to be able to move quickly between the two different accounts and maybe share files between the two accounts. The sort of operations a user might like to perform include • logging in without a password to an account as long as they are coming from their other account, • being able to execute a command on the other machine without having to log into that machine, and • being able to copy files from an account on one machine to the other.
Table 13.1 lists some of the Berkeley r commands that are supplied with most versions of the UNIX operating system.
Command Purpose rsh remote shell, used to execute commands on a remote machine without logging in rlogin remote login, logs into a remote machine, if set up properly no password will be asked for rcp remote copy, copy a file from a remote machine to the local machine
Table 13.1. Some of the Berkeley r Commands.
On some systems rsh is a restricted shell. It acts like a normal shell except it only allows users to execute a limited set of commands. It is not very secure!
By default executing these commands will require the password of the remote account to be entered. If set up correctly the password would not be necessary.
For example: perform an ls on the remote machine aldur bash$ rsh aldur ls Password: Mail core masters.tar.z rcos.uu telnet.doc Masters dead.letter mbox research
However if the two level equivalence system that these commands use is satisfied then no password will be requested. The two levels of equivalence used are • host level equivalence One machine is considered equivalent to another.
David Jones (11/30/94) Page 219 • account level equivalence Accounts on different machines are considered equivalent if they have the same username (and host level equivalence is enforced) or if a particular configuration file is set up.
Host level equivalence is checked first, if this check fails then a password will be required. If the check for host equivalence succeeds then account level equivalence is checked. If account level equivalence fails then a password is requested.
NOTE: This is not always the case. Some systems do not require host level equivalence before checking account level equivalence.
Host Level Equivalence
Given two machines, jasper and aldur, aldur will consider jasper an equivalent host if jasper is included in aldur’s /etc/hosts.equiv file. This host level equivalence means that users with accounts on both jasper and aldur with the same usernames may use the r commands to connect from jasper to aldur without being asked for passwords.
Host and account equivalence are one way. In the above jasper is an equivalent host to aldur but not the other way around.
hosts.equiv file is a text file containing lines of the following format host_name [user_name]
For example: A machine pol might have the following /etc/hosts.equiv file aldur jasper david It specifies that all users on the machine aldur, and the user david from jasper are allowed to use the r commands to connect to pol without specifying passwords.
Account Equivalence
Host equivalence will only work when the usernames on the two different systems are the same. Account level equivalence allows you to specify that accounts, with different usernames are equivalent. Cases where you may wish to use this include • users having different usernames on different machines, or For example, a user might have the username jonesd on the machine jasper and the username david on the machine aldur. To login from jasper to aldur the user would type rlogin -l david
David Jones (11/30/94) Page 220 aldur. However without account level equivalence, users would always have to enter the password. • a user may wish a different user on a remote machine to have access to his or her account. The users may belong to the same project and have to share files.
Account level equivalence is achieved by the file $HOME/.rhosts (a file .rhosts in the home directory of the individual user). The .rhosts file is a text file containing lines of the following format hostname [username_list]
If a username and host is listed in the .rhosts file it means that the specified account is equivalent to this one and may use the r commands without a password.
Using Host and Account Equivalence
Allowing the ability to switch between machines without entering passwords is a security hole that has been used before for nefarious purposes. Account and host equivalence were one of the security holes used by the Internet worm which brought down most of the American machines connected to the Internet.
If host and account level equivalence are allowed the following guidelines should be followed • make the access as fine grain as possible, and • never allow network workstations to be equivalent hosts to a file server.
Workstations (including PCs) are inherently vulnerable. Having insecure machines as equivalent hosts for a network file server is asking for trouble. Having the network file server as an equivalent host for the workstations is common place.
Exercises
13-1. Set a local machine up so that you have host and account equivalence between accounts on two machines.
David Jones (11/30/94) Page 221 Exercises
13-2. Having set up equivalence use the rsh command to perform the following tasks. a)obtain a directory listing of the /etc directory b) create a file passwd in the home directory of the local machine that contains the /etc/passwd of the remote machine (use rsh not rcp) c) create a file listing on the remote machine that contains a directory listing of the home directory on the local machine d)create a file directory.tar on the local machine that is a tar file that contains an archive of your entire home directory on the remote machine.
d) is a method that is sometimes used to implement backups across a network.
Network File System
A major aim (if not the major aim) of all operating systems is to make tasks as simple as possible for the user. The Berkeley r commands provide a bare bones approach to allowing the sharing of information between machines. The increasing importance and level of networking means that there are more users on networks and they all want a much simpler, if not transparent, mechanism for sharing files between machines.
The Network File System (NFS) was developed by Sun Microsystems, a manufacturer of UNIX machines. The specification for NFS was placed in the public domain and is now implemented on every version of UNIX currently available and implementations also exist on other platforms including MS-DOS and OS/2.
NFS runs on top of TCP/IP and is basically a new file system much like the ext2 and msdos file systems discussed in Chapter 7. It provides the ability to connect directories from remote machines into the local machine’s normal file system. This means that users may manipulate the remote files and directories using the same commands they use on local files and directories.
In Diagram 13.1 NFS provides the jonesd and balsys directories that are located on another machine somewhere on the network. NFS allows the user to perform commands like ls /home/jonesd to obtain a directory listing of the jonesd directory.
David Jones (11/30/94) Page 222 An Example Unix File System.
/
usr bin etc home
adm jonesd balsys Partition 1 Local Machine
Using
Partition 2 Local Machine NFS Partition 5 Remote Machine mounted over a network
Diagram 13.1. UNIX File system Using NFS.
Before using NFS the • NFS daemons need to be running, and For NFS to work there are a number of daemons that must be running so that NFS requests can be satisfied. • the appropriate configuration files must specify the directories to import or export.
In the following discussion a client machine is a machine that imports (connects to) NFS partitions from other machines. The machine that exports the NFS partitions is referred to as a server machine. One machine can import and export NFS partitions at the same time.
NFS Daemons
For NFS to work there must be a number of daemons running. The daemons required by clients (machines importing directories via NFS) differ to those required by servers (machines exporting directories via NFS). It is possible for a machine to be both server and client.
The daemons and their purpose are summarised in Table 13.2 for servers and in Table 13.3 for clients.
The NFS daemons are started in the system startup scripts. The exact format of the startup scripts (as discussed in Chapter 9) will differ from system to system. Included below is an example from a server machine’s startup scripts.
David Jones (11/30/94) Page 223 Daemon Purpose nfsd handles NFS requests and executes them on the server, a server will typically run multiple nfsd daemons mountd handles the mount requests portmap performs the translation from RPCs to ports biod the block I/O daemon, typically multiple copies are running rpc.statd the network status daemon, notifies lockd if network goes down rpc.lockd handles file locking and lock recovery
Table 13.2. NFS Server Daemons.
Daemon Purpose portmap performs the translation from remote procedure calls to ports biod the block I/O daemon, typically multiple copies are running rpc.statd the network status daemon, notifies lockd if network goes down rpc.lockd handles file locking and lock recovery
Table 13.3. NFS Client Daemons.
For example: if [ -f /etc/exports ] # does /etc/exports exist? then exportfs -a nfsd 8 echo -n ’ nfsd’ rpc.mountd -n fi
Exporting NFS File systems
/etc/exports is the configuration file used to specify which file systems are available for export via NFS when the system commences. On many systems (like the one the above script was taken from) the NFS server system will not be initialised unless the /etc/exports file exists.
The /etc/exports file is a text file containing lines of the following format directory export_options
directory is the full path name of the directory that is to be exported. The choice of directory must follow these rules • any part of a file system may be exported, NFS allows you to export a whole file system or a part of a file system. • a sub-directory of an exported file system can itself be exported only if it is on a different physical device,
David Jones (11/30/94) Page 224 • a parent directory of an exported file system can only be exported it it is on a different physical device, and • the exported directory must be local. You cannot export via NFS a directory you have imported via NFS.
export_options is a list of options taken from the list of options that NFS recognises. Table 13.4 lists some of these options.
NFS Option Purpose rw=host:host specify which hosts can read/write to the exported directory ro specify that the directory can only be mounted read only access=host:host only allow specified hosts to have access anon=uid map anonymous or unknown users to the specified uid secure clients must use secure RPC to access the file system root=host grant superuser access to the specified hosts
Table 13.4. Export Options for NFS.
For example: An example /etc/exports file might be /home -rw=bertha:pol /usr/local -ro,access=bertha:pol:jasper:aldur This file specifies that the /home directory is exported for the hosts bertha and pol and they can read and write to that directory. /usr/local is exported read only to the hosts specified.
Exercises.
13.3. The machine aldur has the following directory hierarchy
/
/usr
/usr/local /usr/users
/usr/local/temp
and the following /etc/fstab file /dev/hda1 / ext2 defaults /dev/hda2 /usr ext2 defaults /dev/hdb1/usr/local ext2defaults Indicate whether the following /etc/exports files contain valid or invalid entries. If they are invalid, why? a) /usr /usr/users b) /usr /usr/local
David Jones (11/30/94) Page 225 Importing NFS File Systems
There are two methods that can be used to import file systems from other hosts • modify the /etc/fstab or /etc/vfstab file, or As mentioned in Chapter 7 these files control which file systems are automatically mounted when the system is booted. • use the mount command. The mount command can be used to independently mount file systems without rebooting the system.
Chapter 7 contains coverage of the format and purpose of /etc/fstab or /etc/vfstab files. When mounting NFS directories • the file system type is nfs, and • the device to mount is a combination of the hostname and directory name of the server system.
If it was required to mount the directory /home/slackware from the machine pol the entry for the fstab file might look like this pol:/home/slackware (or on some systems /home/slackware@pol).
For example: An example /etc/fstab file might be /dev/hda1 / ext2 /dev/hda2 /usr/local ext2 /dev/hdb1 /dosc msdos pol:/home/slackware /usr/local/slackware nfs
For example: The command to mount the directory /home/slackware from the host pol is mount -t nfs pol:/home/slackware /mnt
NOTE: The actual format of these commands and files will differ slightly between different implementations.
Exercises
13-4. Determine the locations of the files for the various NFS daemons listed in Tables 13.2 and 13.3.
13-5. Determine where these daemons are started (if they are).
13-6. Determine whether your machine is set up as an NFS client or server (or not at all).
David Jones (11/30/94) Page 226 Exercises
13-7. On your local site set up two machines as NFS server and client and using NFS share a particular directory between the two. Experiment with the options listed in Table 13.4.
Other Useful Commands
Part of Systems Administration is discovering and using useful tools and commands. This section will mention some of the more useful network related commands. Table 13.5 summarises the commands covered.
Command Purpose traceroute to discover the route used by information flowing over the network between two hosts nslookup used to query nameservers to obtain IP addresses and other information about hosts
Table 13.5. Useful Network Commands.
traceroute
For some reason or another, users on one machine cannot connect to another machine or if they can any information transfer between the two machines is either slow or plagued by errors. What do you do?
Remember it is not only the machines at the two ends you have to check. If the two machines are on different networks the information will flow through a number of gateways and routers. It might be one of the gateway machines that is causing the problem.
The traceroute command provides a way of discovering the path taken by information as it goes from one machine to another and the problems that may occur. On the Internet that path may not always be the same.
For example: The following are the results of a number of executions of traceroute from the machine aldur (138.77.36.29). Refer back to Diagram 12.3 knuth is on the same network so no gateway was used
bash$ traceroute knuth traceroute to knuth.cqu.edu.au (138.77.36.20), 30 hops max, 40 byte packets 1 knuth.cqu.EDU.AU (138.77.36.20) 2 ms 2 ms 2 ms
David Jones (11/30/94) Page 227 a host one network away bash$ traceroute jasper traceroute to jasper.cqu.edu.au (138.77.1.1), 30 hops max, 40 byte packets 1 centaurus.cqu.EDU.AU (138.77.36.1) 1 ms 1 ms 1 ms 2 jasper.cqu.EDU.AU (138.77.1.1) 2 ms 1 ms 1 ms a machine still on the CQU site but a little further away bash$ traceroute jade traceroute to jade.cqu.edu.au (138.77.7.2), 30 hops max, 40 byte packets 1 centaurus.cqu.EDU.AU (138.77.36.1) 1 ms 1 ms 1 ms 2 hercules.cqu.EDU.AU (138.77.5.3) 4 ms 2 ms 12 ms 3 jade.cqu.EDU.AU (138.77.7.2) 3 ms 13 ms 3 ms A host still in Australia (but a long way from CQU) bash$ traceroute archie.au traceroute to archie.au (139.130.23.2), 30 hops max, 40 byte packets 1 centaurus.cqu.EDU.AU (138.77.36.1) 1 ms 1 ms 1 ms 2 tucana.cqu.EDU.AU (138.77.5.27) 2 ms 2 ms 2 ms 3 138.77.32.10 (138.77.32.10) 5 ms 5 ms 5 ms 4 qld.gw.au (139.130.60.1) 21 ms 13 ms 51 ms 5 national.gw.au (139.130.48.1) 35 ms 36 ms 40 ms 6 plaza.aarnet.edu.au (139.130.23.2) 38 ms 35 ms 68 ms
A host in the Eastern United States bash$ traceroute sunsite.unc.edu traceroute to knuth.cqu.edu.au (139.130.23.2), 30 hops max, 40 byte packets 1 centaurus.cqu.EDU.AU (138.77.36.1) 1 ms 1 ms 1 ms 2 tucana.cqu.EDU.AU (138.77.5.27) 2 ms 2 ms 3 ms 3 138.77.32.10 (138.77.32.10) 5 ms 5 ms 5 ms 4 qld.gw.au (139.130.60.1) 13 ms 20 ms 13 ms 5 national.gw.au (139.130.48.1) 51 ms 36 ms 36 ms 6 usa.gw.au (139.130.29.5) 37 ms 36 ms 38 ms NOTICE THE TIME TO CROSS 7 usa-au.gw.au (203.62.255.1) 233 ms 252 ms 264 ms THE PACIFIC 8 * * t3-0.enss144.t3.nsf.net (192.203.230.253) 224 ms 9 140.222.8.4 (140.222.8.4) 226 ms 236 ms 258 ms 10 t3-3.cnss25.Chicago.t3.ans.net (140.222.25.4) 272 ms 293 ms 266 ms 11 t3-0.cnss40.Cleveland.t3.ans.net (140.222.40.1) 328 ms 270 ms 300 ms 12 t3-1.cnss48.Hartford.t3.ans.net (140.222.48.2) 325 ms 355 ms 289 ms 13 t3-2.cnss32.New-York.t3.ans.net (140.222.32.3) 284 ms 319 ms 347 ms 14 t3-1.cnss56.Washington-DC.t3.ans.net (140.222.56.2) 352 ms 299 ms 305 ms 15 t3-1.cnss72.Greensboro.t3.ans.net (140.222.72.2) 319 ms 344 ms 310 ms 16 mf-0.cnss75.Greensboro.t3.ans.net (140.222.72.195) 343 ms 320 ms * 17 cnss76.Greensboro.t3.ans.net (192.103.68.6) 338 ms 319 ms 355 ms 18 192.103.68.50 (192.103.68.50) 338 ms 330 ms 330 ms 19 rtp5-gw.ncren.net (128.109.135.254) 357 ms 361 ms * 20 * rtp2-gw.ncren.net (128.109.70.253) 359 ms 334 ms 21 128.109.13.2 (128.109.13.2) 374 ms 411 ms 451 ms 22 * calypso-2.oit.unc.edu (198.86.40.81) 418 ms 415 ms
Exercises.
13-8. For those of you connected to the Internet use the traceroute command to obtain the route between your machine and a machine in another country. Perform the same command twice. Each time send the output to a file. Compare the results.
nslookup
There are a number of times when the DNS may not work. In these situations the command nslookup can be used to interactively ask questions of Domain Nameservers. When started the command automatically connects to the server listed in your machine’s /etc/resolv.conf file and displays a prompt. At this prompt you can issue various commands recognised by nslookup (some of the commands are explained in Table 13.6).
David Jones (11/30/94) Page 228 Command Purpose ls [-ah] [domain] list information available for the current domain, -a lists host aliases, -t lists CPU and operating system information (output can be redirected to a file using >) set keyword [ = value ] change a variety of state information that affects how lookups are done host display information about host server host change the server to query to host root change the default server to the root server for the entire Internet, the machine, ns.nic.ddn.mil
Table 13.6. Commands recognised by nslookup.
For example: bash$ nslookup Default Server: jasper.cqu.EDU.AU Address: 138.77.1.1
> ls cqu.edu.au all the machines this server knows about appear
Exercises.
13-9. Using nslookup answer the following a) What aliases are defined for your site? b) How many machines does the nameserver recognise? c) What is the real name of the machine www.cc.uq.oz.au? The nameserver for the domain cc.uq.oz.au is the host cuscus (or at least it was at the time of writing).
Adding a Host
We have examined some of the concepts involved in networking, some of the commands that can be used and some of the services available. The next section lists the steps to take to add a new host to an existing network. The process of setting up a new network is beyond the scope of this text.
The process can be divided into the following steps • install the network hardware and reconfigure the kernel, • obtain the necessary information such as IP address, host name etc, • modify the various configuration files to reflect this information, • modify the startup and configuration files to correctly commence the network services required, • notify the outside world that the machine is connected, and
David Jones (11/30/94) Page 229 • test the network connection.
Pre-Requisite Information
Before connecting the machine to the network you will require the following information • IP address As mentioned in the last chapter every host must have a unique address. This will have to be assigned by the person in charge of the network. • hostname and domain name Every machine must have a unique (within the domain at least) host name. Depending on your site either you will decide the hostname or you will have one assigned to your machine. • IP address and hostname of your gateway When setting up your machine you will have to inform it which machine to use as a gateway host. • IP address and hostname of your nameserver You will have to have a nameserver to take part in the DNS. • Network address Used as part of the configuration of the hardware. • Broadcast address
Configuration Files
Most of the information from the previous section will have to be entered into network configuration files for the machine to be properly configured. Table 13.7 summarises the information and the configuration file in which it belongs.
There are a number of commands that may also make use of the above information and should be included in the startup scripts. Table 13.8 summarises these commands.
ifconfig is used to set up the software side of network interfaces so that they properly represent the systems values.
For example: An example ifconfig command is /sbin/ifconfig eth0 138.77.37.28 broadcast 138.77.37.255 \ netmask 255.255.255.0
David Jones (11/30/94) Page 230 Information Configuration File IP address and hostname /etc/hosts hostname used by the hostname command in the startup files Domain name /etc/resolv.conf IP address and hostname of /etc/gateways gateway IP address and hostname of /etc/resolv.conf nameserver network address used in startup files and in /etc/networks broadcast address used in startup files
Table 13.7. Configuration Files for Networking.
Command Purpose ifconfig performs configuration so software will recognise each network card hostname initialises the hostname of your machine
Table 13.8. Startup Commands for Network Configuration.
Network Hardware and the Kernel
Before any new hardware will be recognised by the UNIX operating system the operating system must have a device driver for that device. Network hardware is no different. Before you can connect to a network the kernel for your machine must contain a device driver for the type of network hardware you are going to use.
Installing a new device driver will differ between versions of the UNIX operating system and is covered in a later chapter.
Startup Files and Network Services
The system startup files will have to be modified to • set up and configure the networking hardware • set up and configure the networking software • start the various required network services.
David Jones (11/30/94) Page 231 Notify the Outside World
Once the previous steps have been followed you should be able to talk to other machines on the network. However they will probably not be able to talk to you as your hostname will not yet have been added to the local nameserver.
Now you should notify those in charge of the local nameserver (possibly yourself) that your machine is up and talking to the network. They will then add your machine’s host and IP address to the local nameserver that will allow other machines to connect to your machine.
Testing the Connection
As a last step you should carry out tests to see that everything works. A suggested order for tests is • start by pinging a remote host • telnet to a remote host • from a remote host ping your new host • ftp some files to and from your host • attempt to send electronic mail to and from your host
When testing using a remote host do not limit yourself to a host that is on the same network as the machine you have just added. Test with machines that are on different networks.
Conclusion
In this chapter you have been introduced to • the Berkeley r commands • Sun’s Network File system • the commands traceroute and nslookup, and • the process for adding a host to an existing network
Over the last two chapters you have received a gentle introduction to UNIX networking. There is a great deal more to learn and experience in this field.
David Jones (11/30/94) Page 232 Revision Questions
13.1. Table 13.e1 lists the UNIX machines and their purposes at your new workplace.
Host Purpose gandalf system administrator's machine bilbo used by software development team frodo used by administration
Table 13.e1.
Table 13.e2 lists the more important users on each system, their accounts and requirements.
User Position Host Username for equivalent account yourself System gandalf, bilbo, your username Administrator frodo gandalf root bilbo root frodo root Arnold Pane Managing Director frodo arnold bilbo panea Andrew software engineer bilbo andrew Hacker Jim Tidey operator gandalf tideyj bilbo backup frodo backup
Table 13.e2.
(a) List the full paths, host and contents of all the files that need to be created to allow each user to use the remote commands (rlogin, rsh, rcp etc) between all of their accounts.
(b) Why might this not be a good idea?
David Jones (11/30/94) Page 233 Chapter 14
SECURITY
A chain is only as strong as its weakest link. -- Proverb.
If a hacker obtains a login on a machine, there is a good chance he can become root sooner or later. There are many buggy programs that run at high privileged levels that offer opportunities for a cracker. If he gets a login on your computer, you are in trouble. -- Bill Cheswick.
Objectives
This chapter aims to introduce you to the basics of security under the UNIX operating system. You will be introduced to • the types of security threats facing a computer system and the methods and tools available under UNIX to combat these problems, • a list of checks that can be used to maintain and check the security of a system, • a number of public domain packages that can be used to enhance the security of a UNIX system, • the problems involved with search paths, set-uid/set-gid scripts, and the permissions of important system files and directories, and • a number of accounting and log files and the commands that can be used to observe the state of a UNIX machine.
Introduction
A machine running the UNIX operating system can be made into a very secure system if the right amount of effort is expended. However a very secure system is usually too inconvenient for normal users to use. In implementing a security scheme the Systems Administrator must weigh the following costs • the importance of the machine, its availability and the data stored on it, • the amount of effort required to make and keep the system secure, and • how the security features will effect the users of the system.
A system can be made as secure as is necessary but in doing so you might lose all ability to make use of the machine. A machine in a room with no door and no outside connection is very secure, no-one can use it. The Systems Administrator must balance the needs for convenience against the need for security.
David Jones (11/30/94) Page 234 To implement security on a system you should first identify the possible threats to the system. There are three major types of threats to a computer system • physical threats The building burns down, an earthquake hits or an intruder breaks into your office or machine room. • unauthorised access to the system and its data, and • denial of service. A vandal performs some action that makes the system unavailable to other users. For example, consuming all the disk space and running CPU intensive programs.
Having identified the threats to the machine a security plan must be implemented. A security plan should have two components • methods for preventing or decreasing the possibility of threats, We will examine methods used under UNIX to prevent threats in this chapter. • penalties and policy, There must be a policy stating what is allowed on the system and what is not. There must also exist set punishments for breaking the policy. • a recovery plan. There is never a 100% guarantee with any security plan. Plans must exist that enable the system to recover from a security breach as quickly as possible.
Physical Threats
Physical threats include • unauthorised access to system consoles and other devices, and • acts of nature (i.e. floods and earthquakes).
Not all attacks on computer systems rely on intimate knowledge of computer hardware and software. The quickest way of denying service is to steal or destroy the physical hardware. Mechanisms should be in place to prevent access to the physical hardware of a system.
One part of modern computer systems that is often overlooked in a security plan is the cabling. The simplest way to bring a site's computer network to the ground is to take a shovel and dig up a few fibre optic cables. This does not always happen on purpose. CQU’s network has been taken down a number of times by people (accidentally) digging up fibre.
Acts of Nature
While every effort can be taken to minimise damage from acts of nature there is always the possibility that an event will occur that can destroy a system. This is one possibility that must be served by the site’s recovery plan.
David Jones (11/30/94) Page 235 The old maxim don’t put all your eggs in one basket is very applicable. Copies of backup tapes should be kept at another site. A number of sites in earthquake prone California send copies of backup tapes to other States to make sure that tapes are out of the earthquake zone.
Logical Threats
Logical threats typically take the form of • human beings making mistakes, Security threats are not always intentional. A student learning how to program might by mistake write a program that forks off thousands of child processes that bring the machine to a grinding halt (I know, I have done it). • human idiots with too much time on their hands, or People who break into computer systems generally are not very smart. They simply have a great deal of patience and time. They rely on the fallibility of users and a variety of information sources to provide them with security holes through which to gain access to computer systems. • programs written by human idiots with too much time on their hands. Table 14.1 lists some of the terminology used to describe these programs.
Computer systems today are complex congregations of interacting programs. The complexity of these programs and their interactions means that security holes crop up every now and then. It is these holes that bad guys use to break into systems.
It is important that a Systems Administrator keeps up to date with possible security holes in the systems being managed. Possible source of information about security holes are • system vendors, Some (not all) vendors will attempt to keep their customers aware of security holes that have been discovered in their systems. Do not depend on them doing so. • trade magazines and other publications, There are various books and magazines that discuss Systems Administration and security. However the nature of the published word means that notification may not always arrive in time. • the Internet, and There are various forums on the Internet that discuss security issues. The net is generally more up to date than the previous two sources. • various Computer Emergency Response Teams (CERTs).
David Jones (11/30/94) Page 236 Name Method bacteria programs that reproduce rapidly and eventually overwhelm the system viruses programs that hide themselves inside other programs and cause bad side-effects when the enclosing program is executed worms programs that use networks to move from system to system perhaps leaving behind viruses and bacteria Trojan horses programs that appear to provide the functionality of another program but really do something else back doors undocumented methods of access within programs
Table 14.1. Nasty Program Categories.
Computer Emergency Response Teams
The main aims of a Computer Emergency Response Team (CERT) are • work with the Internet community to facilitate its response to computer security events, • to take proactive steps to raise the community’s awareness of computer security issues, and • to conduct research targeted at improving the security of existing systems.
The first CERT was formed by the Defence Advanced Research Projects Agency (DARPA) in response to the Internet worm incident in late 1988. Some of the services provided by CERT include • 24 hour technical assistance for responding to computer security incidents, • product vulnerability assistance, • technical documents and seminars, • a number of mailing lists, and • an anonymous FTP server that provides access to a variety of security related documents and tools.
One of the major services provided by CERT are CERT advisories. A CERT advisory provides information on how to patch a known computer security problem or a warning about ongoing attacks.
Much of the information in this section is taken from the FAQ (frequently asked questions) list available from CERT’s anonymous FTP site.
David Jones (11/30/94) Page 237 Information Source Description comp.security.announce newsgroup to which CERT advisories are posted cert.org anonymous FTP site for CERT [email protected] mailing list for CERT advisories [email protected] mailing list about security tools
Table 14.2. Various Information sources provided by CERT.
Passwords
Passwords are the first line of defence in the security of a computer system. They are also usually the single biggest security hole. The main reason is that users do dumb things with passwords including • write their password on a bit of paper and then leave it laying around, • type their passwords in very slowly while someone is watching over their shoulder, and • choose really dumb passwords like password or their first name.
These methods are the easiest way of breaking into accounts on most computer sites, especially Universities.
There have been a number of experiments that attempt to discover how many users actually choose dumb passwords. All of these experiments have found an alarmingly high percentage of users choose stupid passwords. One experiment found that approximately 10 to 20% of passwords could be guessed using a password list containing variations on login names, user’s first and last names and a list of 1800 common first names
Every year the program Crack (more on this program later in the chapter) is run using the password file of the machine used by students of the Systems Administration subject offered by Central Queensland University. Every year between 10 to 20% of the passwords are discovered by Crack.
There are a number of schemes a Systems Administrator can use to help make passwords more secure including • shadow passwords, Once they have a system’s encrypted passwords bad guys can crack these passwords using a variety of methods. Mentioned in the chapter on adding users, shadow passwords remove the encrypted password from the /etc/passwd file (a file readable by every user) and place them into a file readable only by the root user. This prevents the bad guys from (easily) getting a copy of your encrypted passwords.
David Jones (11/30/94) Page 238 • proactive password programs, Passwords are set by using the passwd command. A proactive password program replaces the normal passwd command with one that enforces certain rules. For example, ensuring that all passwords are greater than 5 characters in length and not accepting insecure passwords like usernames, the word password, 123456789 etc. If a user’s new password breaks these rules the new passwd program will reject the password. • password generators, Some sites do not allow users to choose their own passwords but instead they use password generators that provide the user with a list of passwords, consisting of random strings of characters, and asking them to choose one. The passwords that are generated have to be easy to remember or else users start writing them down. • password aging, The longer a password is used the greater the chance that it will be broken. Password aging is usually built into most shadow password suites. The system forces passwords to be changed after a set time period. In addition the system may remember past passwords thereby preventing a user simply cycling through a list. Care must be taken that the time period after which passwords must be changed is not too frequent. If it is users start forgetting passwords and resort to writing them down. • password cracking, and Most crackers will not break passwords by decrypting an encrypted password. Instead they encrypt a list of words (often taken from a dictionary) and compare the encrypted word with the encrypted password. If a match is found they know the password. There is a public domain program called Crack (see Table 14.7) that performs this task automatically. Even though it can consume a great deal of CPU time it can be useful to run Crack on a system’s passwords regularly to identify the users who have insecure passwords. • user education. Users do not want other people breaking into their accounts. If the users of a system are educated in the dangers of using bad passwords most will choose good passwords. One effective education program might be breaking their passwords with Crack and then telling them what their password is (if you can do it, the bad guys can).
Threats to /etc/passwd
The /etc/passwd file is the cornerstone of the password security system. The Systems Administrator should perform a number of checks on the contents of the /etc/passwd file. These checks are performed to make sure someone has not compromised security and left a gaping hole. The checks include looking for • accounts without passwords, In most systems today this might mean checking the /etc/shadow file.
David Jones (11/30/94) Page 239 • accounts without usernames, You cannot login to an account without a username using the normal login procedure. However you can become that user by using the command su ””. • accounts other than root with a user id of 0, An account with a uid of 0 will have the same access permissions as the root user since the operating system thinks that anyone with uid 0 is root. • accounts other than root and other system accounts with a group id of 0, Generally only the root user and one or two system accounts will belong to group 0. Any other account being in that group will obtain permissions it should not. • new additions or deletions from the /etc/passwd file that weren’t performed by members of the Systems Administration team and, The only modifications made to the /etc/passwd file should be made by the Systems Administration team. Any changes not made by that team implies someone has broken the security of your system. One method of checking this is keeping an up to date copy of the passwd file somewhere else and regularly comparing it with the /etc/passwd file. • the file permissions on the file itself The passwd file is usually owned by root. Only the owner of the file should have write permission on the passwd file. If these permissions have changed someone has broken your security.
Exercise.
14-1. Write a shell procedure passwdck that performs all of the checks listed above.
Passwords are only the first line of defence for a system. It must be assumed that someone will be able to break this line of security. The following section examines what steps can be undertaken to improve the internal security of the system.
Search Paths
When you enter a command the shell will search through all the directories listed in the PATH variable for an executable file with a filename that matches the command name. It is almost standard for users to include the current directory (signified by .) in their search path.
If the current directory is included in the search path it should be the last one.
David Jones (11/30/94) Page 240 If the current directory is the first directory in the path then whenever the user executes a command the shell will look first in the current directory. This is a security hole. One practice of bad guys is to place programs with names that match standard commands (like passwd and su) everywhere in the directory hierarchy.
They do this to take advantage of situations like the following • the current directory is the first directory in the search path of the user, • the user is in the directory /usr, • a bad guy has placed a program called passwd in that directory, and • the user wants to change their password so they enter passwd.
The shell will find the passwd program written and placed in the /usr directory by the bad guy. Bingo, the vandal has a password.
The current directory SHOULD NOT be in the search path for the root user.
Some Systems Administrators are so worried about this situation that they will always enter the full path of every command executed as root.
For example: Instead of typing bash$ su They will enter bash$ /bin/su
File Permissions
If a bad person has actually managed to crack someone’s password and break into their account the next step they will want to take is obtain an account with more access (root if possible). The major hurdle they must overcome is UNIX file permissions.
A system’s file permissions should be set up in such a way that will prevent users from accessing areas that they should not. The Systems Administrator is responsible for first setting up the file permissions correctly and then maintaining them.
The issues involved with file permissions include • maintaining correct protection settings on all files and directories, In setting a system up each file and directory must have the correct permissions. This is especially true of important system files including device files and system configuration files (/etc/passwd, the startup scripts etc). • tracking changes, and Once set up regular checks on the file permissions should be performed to ensure that no-one has been tampering with them.
David Jones (11/30/94) Page 241 • tracking and limiting disk usage. If the naughty person is a simple vandal interested only in bringing the system down he might try something like the following #!/bin/sh while [ 0 ] do mkdir .temp #start with a dot so it is normally hidden cd .temp cp /bin/* . done
One way to prevent this type of thing from happening is to use disk quotas (see the next Chapter 15) to limit the amount of disk space any user can use.
Permission Settings
The Systems Administrator must ensure • that file system permissions are set correctly to start with, and • ensure that they do not change from these correct settings.
It is important that a system have the correct permission settings on all files especially important system files and the device files. If the system files, such as /etc/passwd, have incorrect permissions bad guys can do a number of nasty things to a system.
The device files are equally important. Having world read and write on device files for disk drives is a security hole. Since the disk can be read from the device file a bad person does not have to go through the file system (the file system enforces the file permissions system). With a little knowledge about the file system structure they can write a program to read (or delete) any file on the disk.
Permission settings are not always set correctly by the manufacturer. One version of a well-known workstation was distributed with sound capabilities, including a microphone. The problem was that the permissions, as set in the factory, for the microphone device file included world read. This meant that if the microphone was turned on anyone on the system could listen to what was being said in the vicinity of the microphone.
Once the permissions are set correctly they must be continually checked to ensure that they are not changed. This can be accomplished by a simple shell script that • performs an ls -l on all important files and sends the output to a summary file, • the summary files are kept for comparison, • the new summary file is compared against the previous summary file using the diff command, and • any differences between the two files means there has been a change.
David Jones (11/30/94) Page 242 A script of this description will report changes in • file ownership, • file permissions, • file modification dates, and • file owner and group owner.
Exercises.
14-2. Write a shell script fs_check that performs the roles outlined above.
Setuid Programs and Shell Scripts
Create a text file called i_am.cc that contains the following C program
#include
void main() { int real_uid, effective_uid; int real_gid, effective_gid;
real_uid = getuid(); effective_uid = geteuid(); real_gid = getgid(); effective_gid = getegid();
printf( "The real uid is %d\n", real_uid ); printf("The effective uid is %d\n", effective_uid ); printf("The real gid is %d\n", real_gid ); printf("The effective gid is %d\n", effective_gid ); }
Compile the program by using the following command cc i_am.cc -o i_am This will produce an executable program called i_am. Run the program.
Every time you execute a program the operating system creates a process. Associated with every process is a great deal of information including • the real user and group identifiers (both numbers), and • the effective user and group identifiers.
The real user and group identifiers are used to store the user and group ids of the user who executed the command. The effective user and group id are used to determine what access the process has, what files and directories it can read, write and delete.
Normally the effective ids will be the same as the real ids. However the set user identifier (set-uid) and the set group identifier permissions (set-gid) can be used to change the effective identifiers.
David Jones (11/30/94) Page 243 The process will be executing a program. The code for that program will be contained in a file somewhere. When the set-uid and set-gid bits are set the effective ids are changed to the uid and gid of the file’s owners (instead of those of the user running the program).
The passwd command is a prime example of the use of set uid permissions. passwd allows any user to modify either the /etc/passwd file or the shadow password file. How is this possible if only the root user has write permission on these files? passwd is a set-uid program.
Exercises.
14.3. Examine the permissions of the passwd program. What is different?
14.4. Examine the manual page for chmod to discover how a file is made setuid and setgid.
14.5. Change the ownership of the executable file i_am so it is owned by the root user and is setuid root. Run the program and see how this changes the output.
14.6. Make the i_am program setgid root.
Never ever write a setuid shell script. In fact, some operating systems will not allow setuid shell scripts. The reason is that there are well-known ways of using setuid shell scripts to obtain extra privileges for nasty purposes.
Once bad people get root access on a machine they want to keep it. One method of doing this is to write a program or script that is setuid root and secret it away in some part of the file system. The next time they login all they have to do is run the program and they have root access again (even if the root password has been changed).
You should check regularly for the addition of any setuid and setgid programs anywhere in the file system.
Exercises.
14-7. Incorporate into the script fs_check the ability to keep a track of current setuid and setgid programs.
David Jones (11/30/94) Page 244 Log Files and User Activity
Having all the above precautions in place is only the first step in maintaining security. In addition the Systems Administrator should perform regular checks on • what currently logged on users are doing, • what users have done or have tried to do.
Just to make certain nothing untoward is going on.
What are the Users up to?
A Systems Administrator should have some idea about what is considered normal behaviour for the system they maintain. A Systems Administrator should know the characteristics of their system including what is the normal load, the normal disk usage, when do certain users normally logon and what the users normally do.
This knowledge can help identify when something unusual is going on. There are a variety of system commands and systems that allow the Systems Administrator to keep an eye on the system. These commands are summarised in Table 14.3.
Command Purpose Availability ps List information about current processes all systems top Similar to ps but provides additional BSD systems plus features some others w and who display information about what users are all systems logged on whodo Displays information that combines w SysV and ps with some extras
Table 14.3. System Observation Commands.
BSD and SysV each have different options for the ps command. Tables 14.4 and 14.5 list some of the more useful ps options for observing the current state of the system.
Switch Purpose -a display processes owned by other users -x include process with no controlling terminal -S display accumulated CPU time for the process -g display all processes -c display the real command name
Table 14.4. BSD ps Switches.
David Jones (11/30/94) Page 245 Switch Purpose -e display every process now running -f generate a full listing
Table 14.5. SysV ps Switches.
For example: The whodo command produces output like the following bash$ whodo Thu Sep 22 10:44:53 1994 UNIX_System_V
pts/1 david 10:41 pts/1 4926 0:00 bash pts/1 4944 0:00 elm
pts/2 phil 9:36 pts/2 4806 0:00 bash pts/2 4946 0:00 whodo
What have the Users done?
Most UNIX systems have a number of accounting functions (discussed in detail in Chapter 15) that maintain logs of what has been done to and with the system. The type of information normally kept includes • a record of every process executed including who executed it, when it started, when it finished and what percentage of system resources it used, • when users logged in and when they logged out, • various warning and error messages including when somebody has used su, any unsuccessful logins and various information about network connections, and • printer usage.
Table 14.6 summarises some of the commands that can be used to view this information.
David Jones (11/30/94) Page 246 Command Purpose Availability sa displays, summarises process accounting BSD information lastcomm displays information about the last commands BSD that have been executed ac displays information about login, logout BSD last displays last login times for users all pac various information about printer use BSD quot amount of disk space used by individual users all syslog a logging function that records certain events, BSD, some SysV errors and warnings du amount of disk space used all df amount of disk space free all
Table 14.6. Commands for examining Accounting Information.
These commands and systems are discussed in more detail in the next chapter.
Network Security
Connecting a computer to a network is probably the worst thing you can do from a security point of view. Connection to a network adds a new pile of security holes that have to be plugged and adds a whole new population of bad guys who will try to break your system.
The following is a brief introduction to network security. It is a topic too large to cover properly in an introductory Systems Administration course.
In this networking age it will be a rare computer site that is not networked in some way. The following are some of the schemes that are used to make networking more secure • Internet firewalls, It is reasonably straight forward to connect an existing TCP/IP based LAN to the Internet. However there is one problem doing it simply. People outside your LAN can see and access each individual machine. This means that all the machines on the LAN must be secure. An Internet firewall surrounds the LAN and allows people outside the LAN to see only one machine. This one machine is set up as the drawbridge between the LAN and the outside world. All network traffic passing through this draw bridge is closely monitored and any traffic that is not allowed is thrown into the moat. The advantage of this system is that there is one central point used to control access to the local LAN. A firewall should be a compulsory part of an Internet connection.
David Jones (11/30/94) Page 247 • packet filtering, Most commercially available routers and gateways provide some form of packet filtering. This means that the Systems Administrator can specify the type of network packets that are allowed from the outside network onto the local network. Packets can be accepted or rejected on the basis of the network they come from, the particular protocol they use or a number of other criteria. A packet filtering gateway is typically part of an Internet firewall. • encryption, and Most information sent across a network is sent in plain text (including passwords). It is possible to listen in on most of this information. One scheme to combat this is to encrypt the information passing across the network. One system that implements this is the Kerberos system (see Table 14.7) • special network daemons. The public domain program TCPWrappers provides a special daemon that provides some additional security, access control and logging facilities on network connections. TCPWrappers works by being inserted between the inetd daemon and the various network daemons that are executed by inetd.
Old System. ftpd
Incoming inetd talkd network request inetd starts the right daemon nntpd
Diagram 14.1. How network daemons are started without TCPWrappers.
Instead of running the appropriate network daemon inetd will instead run the TCPWrapper daemon. This daemon performs a variety of security checks and some logging functions before running the network daemon that inetd would have run in the original system (Diagram 14.2).
TCPWrappers ftpd inetd starts tcpd Incoming inetd tcpd talkd network request tcpd does some security checks and starts the nntpd right daemon
Diagram 14.2. How network daemons are started with TCPWrappers.
David Jones (11/30/94) Page 248 Public Domain Security Tools
There are a number of public domain packages that can assist the Systems Administrator in maintaining the security of a machine. Table 14.7 lists some of the more popular, the purpose of the packages and the Internet addresses they can be obtained from.
Name Purpose Location ftp://archie.au/security/cert/cops COPS checks the status of a ftp://cert.sei.cmu.edu/pub/cops UNIX system, checks permissions, for Trojan horses ftp://brolga.cc.uq.oz.au/pub/athena/kerberos Kerberos network encryption ftp://aeneas.mit.edu/pub/ATHENA/kerberos system ftp://archie.au/security/cert/tools/crack Crack cracks insecure ftp://cert.sei.cmu.edu/pub/tools/crack passwords ftp://archie.au/security/tools/tripwire Tripwire checks for security of ftp://cert.sei.cmu.edu/pub/tools/tripwire and changes to the file system
Table 14.7. Public Domain Security Packages. Conclusions
A UNIX machine can be made secure if care and effort is expended. How secure your system is to become must be balanced with other factors including ease of use.
Passwords are the first line of defence and often the weakest. Users must be encouraged to use secure, hard to guess passwords. The internal security supplied by file permissions is the second line of defence. To maintain the security of a machine the Systems Administrator should regularly • have in place a good and up to date recovery plan, • discourage poor choice of passwords and regularly check the security of the passwords, • check the file permissions of system configuration files, • maintain a close watch on the current state of the machine, • use disk quotas and other resource limit schemes to prevent denial of service, • check accounting and logging information for any suspicious events, and • if possible install and use a variety of security packages.
David Jones (11/30/94) Page 249 Review Questions
14.1. List the issues that are important for system security.
14.2. List ways in which password security can be enhanced.
14.3. List possible problems with the internal security of a system.
14.4. How would you go about breaking into a system?
David Jones (11/30/94) Page 250 Chapter 15
THE KERNEL
It is perfectly true that government is best which governs least. It is equally true that that government is best which provides most. -- Walter Lippmann.
Objectives
This chapter aims to introduce you to the kernel of a UNIX operating system. It will use the Linux kernel and its associated procedures as an example. By the end of the chapter you should • be able to explain what a UNIX kernel is and what its responsibilities are, • be able to list the components of a kernel, • know what a patch file is, • be familiar with the commands diff and patch, • know the Systems Administration’s responsibilities with respect to the kernel, • be aware of what is involved with configuring the kernel, • talk about the importance of keeping backup working kernels, and • be able to compile, install and use a new kernel.
Introduction
Anyone who states that the UNIX operating systems is harder to use than Windows is wrong. What they really mean to say is that the interface of Windows is easier to use than the UNIX interface (the shell).
The UNIX operating system (sometimes referred to as the kernel) is actually a conglomeration of functions, procedures and data structures that perform a variety of tasks. The only way in which someone may use the operating system is via a number of function calls called system calls.
The specific responsibilities of the operating system kernel are to • provide a simple, efficient interface and management of the hardware of the computer, and • provide a number of additional services for the use of other programs.
A Simple Interface
Writing programs that directly access computer hardware is difficult and hard to do correctly. In addition the ability to directly access the hardware of a computer means that it is very easy to bring a machine to its knees (just
David Jones (11/30/94) Page 251 ask anyone who has learnt how to program in C on an MS-DOS box). An operating system kernel provides an easy to use interface to the hardware of the computer as well as protecting the bare hardware from interference.
The interface provided by the operating system is the set of system calls it understands. System calls are basically function calls that are used by user programs to access the services provided by the operating system.
The system calls that are understood by a UNIX operating system will be listed in the second section of the manual page (remember man 2 kill). All a user program knows about the operating system is the set of system calls it will understand.
Additional Services
The kernel also provides a number of “value added” services that are used by the systems processes. A UNIX kernel provides a number of services including • managing the CPU, • executing programs, • scheduling input/output tasks, • performing accounting and quotas, • providing a logical file system, • providing device drivers that control the variety of physical devices and how information is transferred between them and the computer, • allocate and de-allocate memory between the various programs, • provide a variety of security and protection abilities, and • many more.
Most UNIX operating systems will come with the code of the kernel. The Linux operating system typically places the kernel code in the directory /usr/src/linux.
Exercises.
15.1. Examine the kernel code for your system. List some of the different functionality included in the kernel.
15.2. Count the number of separate source files and directories that make up the kernel source code. HINT: use a combination of the find and wc commands.
15.3. Discover the list of system calls supported by your system. (Hint: examine the manual page for the man command and in particular look for mention of a shell variable MANPATH.)
David Jones (11/30/94) Page 252 Kernel Responsibilities of the Systems Administrator
The Systems Administrator has a number of responsibilities related to the kernel including • ensuring the system has a valid kernel to boot with, • configuring the kernel to represent the system’s make up, • modifying the kernel code, and • compiling the kernel.
Valid Kernels
When a UNIX system boots, the bootstrap program will examine certain set places for a kernel. On most systems it will attempt to use a particular file from the root directory. The Systems Administrator must ensure that there is always a valid, working kernel in this location. If the kernel is incorrectly configured the system will not boot.
Reserve Kernels
When the Systems Administrator starts recompiling and installing new kernels there is always the possibility that a mistakes will be made. For example, in specifying the interrupt address for a network card you might enter the wrong value.
These types of mistakes may cause the kernel to freeze during initialisation. If the only bootable version of the kernel freezes during initialisation it can be very difficult to get the operating system functioning.
There should always be at least one backup method of starting the system including a backup kernel. The reserve kernel is used in cases where the primary boot method has failed for a variety of reasons.
In most cases the reserve kernel will be kept on • a floppy disk, • a CD-ROM, or • a magnetic tape.
Exercises
15.4. Prepare a reserve kernel for your system. The process will be system specific. Under Linux examine the contents of the /usr/src/linux directory for instructions.
15.5. Use the reserve kernel to reboot your system.
David Jones (11/30/94) Page 253 Configuring the Kernel
The kernel source tree will contain the code for a wide variety of purposes. In some cases it may not be necessary to include all of this source into the final kernel. For example, the kernel might contain code for a sound card driver. If you do not have a sound card then there is no point in including that device driver in the kernel.
The kernel does not contain code alone. It also includes a number of data structures used by the operating system.
For example: Under Linux information about every process is stored in an array containing elements of the following structure. struct task_struct { /* these are hardcoded - don't touch */ volatile long state; /* -1 unrunnable, 0 runnable, >0 stopped */ long counter; long priority; unsigned long signal; struct sigaction sigaction[32]; . /* file system info */ int link_count; int tty; /* -1 if no tty, so it must be signed */ unsigned short umask; struct inode * pwd; struct inode * root; struct inode * executable; struct vm_area_struct * mmap; . } The size of this array is controlled by the following macro. By changing this value a large system can handle more processes. /* * This is the maximum nr of tasks - change it if you need to */ #define NR_TASKS 256
Configuring the kernel is the act of deciding • what sections of the kernel source tree should be compiled and included into the kernel, When the system boots the entire kernel is loaded into RAM and it stays there while the machine running. The less code included in the kernel, the smaller it will be and the more RAM there will be for user programs. • the size of some of the data structures. Larger machines will require larger data structures than smaller systems.
David Jones (11/30/94) Page 254 The method used to configure the kernel will differ from system to system. Currently under the Linux operating system it is performed using the command make config inside the root directory of the kernel source tree.
This command runs a shell script that asks a number of questions about what should or should not be included into the kernel.
An Example make config
The following section lists some of the questions that Linux will ask during a make config. It is included here to provide an idea of what may be included in the kernel of a UNIX operating system. In addition to the questions there are some annotations explaining the questions. (Some of the questions have been left out due to space considerations.)
kernel math emulation? used on systems without a math co-processor (386) normal harddisk support? Normal means one of ide, rll, mfm, esdi, etc. SCSI is not normal.
Xt hard disk support? For MFM, RLL, ESDI Type drives on an 8 bit controller. Not needed if you have a 16 bit card, or only IDE or SCSI. Networking support? Needed if you are going to connect to a LAN or use serial line TCP/IP protocols like SLIP or PPP. As a suggestion it should be all the time. Limit Memory to below 16M? Answer yes if your physical memory is below 16Mb OR your motherboard design prevents you from caching any RAM above 16Mb (systems with 64Kb cache). If your machine has 128Kb or 256Kb cache and more than 16Mb of RAM answer no. SysV IPC? YES Includes interprocess communication primitives that originated in SysV based UNIX. Includes shared memory and semaphores. use -m486 flag? Performs optimisations for 486 based processes. Answer yes if you have any type of 486 processor (including Pentiums).
Network Questions.
TCP/IP ? Answer yes to this all the time. Even if you aren’t on a network you can use the loopback device to practice with networking software. SCSI support? Answer yes if you have a SCSI interface of any sort (including SCSI-2 and SCSI-3). If you do have a SCSI interface you will be faced with a range of SCSI options. Network device support? Yes if you plan to use Ethernet cards to connect to a LAN. Dummy net driver?
David Jones (11/30/94) Page 255 Not really necessary but can be useful for network programming. SLIP? Do you plan to use SLIP (serial line internet protocol) to connect to a network? CSLIP? CSLIP is compressed SLIP, same answer as you gave to the SLIP question. PPP? PPP (point-to-point protocol) is another serial line internet protocol. Load balancing? Very experimental and should be left out.
The next set of questions will ask whether or not to include drivers for specific network cards. Basically if you have the card you include the driver.
After the network cards it will ask a number of questions related to CD- ROM drives. The following is the list of CD-ROM drives Linux supports.
Sony? Mitsumi? Matushita/Panasonic/(Some Creative Labs too)
After CD-ROMs come the file systems questions. If you want your kernel to support a different file system (MS-DOS for example) you must compile into the kernel the code for that particular file system. File systems supported by Linux include:
minix? extended? second extended? The major file system used by Linux. xiafs? msdos? /proc?
nfs? NFS is the network file system. iso9660? The standard CD-ROM file system. os/2 hpfs? system V and coherent?
parallel printer support? Yes if you want to use a parallel printer.
The next questions are related to drivers for various “non-standard” mice. You include them if you have one.
logitech bus mouse? ps/2 mouse?
David Jones (11/30/94) Page 256 microsoft busmouse (This refers to the bus mouse not the normal serial mouse) Atixl busmouse?
selection? Allows you to cut and paste on the normal text screens. I’ve never used it and so don’t include it. It appears to cause problems with bus mice. QIC-02? Qic 117? Both specify a type of tape-drive. If you don’t have one there is no need to include it. Sound card support? Yes if you have a sound card. You will have to answer a number of questions about various sound card options if you do. kernel profiling? Only really used when you are developing kernel code. When turned on will generally slow the system down.
Exercises.
15.6. Perform the necessary operations to configure the kernel on your system.
Modifying the Kernel
Reasons why a kernel may have to be modified include • the addition of new devices or systems, The addition of a brand new device will require the addition of new device drivers. • removing errors from the operating system, or • making personal modifications to the operating system.
The kernel source code contains a large number of source files spread throughout a rather complex directory hierarchy (sometimes referred to as the kernel source tree). It is not uncommon for there to be slight mistakes hidden away within this large array of files.
If a vendor discovers these errors there are two ways in which the problem can be fixed • send out a whole new kernel source tree, or • send out a small set of patches.
Sending out a whole new kernel source tree would be expensive because of the size of the media needed to distribute it. In most cases a small set of patches will be distributed.
Patching the kernel means applying patches to the kernel source tree to produce a slightly modified version of the kernel source.
David Jones (11/30/94) Page 257 What are Patch Files?
A patch file is a text file that is produced by the output of the diff command. The diff command is used to compare the differences between two (or more) files. The output of diff is the changes between the first file and the second one (see Diagram 15.1).
User has two files. file1 file2 original changed file file
diff file1 file2 > file3
file3 1c1 < original --- > changed
Diagram 15.1. Using diff to produce a patch file.
To produce a patch file the output of the diff command is redirected to a file. The patch command can then be used to change a file that looks like the first file into the second file (see Diagram 15.2).
first.file original file
patch first.file < file3
first.file.orig first.file original changed file file
Diagram 15.2. Applying a patch file. Applying patches using the patch command will typically result in two files being created. In Diagram 15.2 the two resulting files are • first.file which contains the modified file, and • first.file.orig that contains a backup copy of the original file. Most versions of patch will keep a backup of the original file. Some may use a different extension instead of .orig.
David Jones (11/30/94) Page 258 Exercises.
15.7. Create a file called abc.doc that contains a number of lines of text. Create another file called def.doc that contains a modified version of what is in abc.doc. Produce and use a patch file to transform abc.doc so that it is exactly the same as def.doc.
Compiling the Kernel
Once the kernel is configured it must then be compiled into one ‘executable’ file or image. On most UNIX systems this will entail using some form of the make command.
The make command is used to compile and link a large number of source files according to some user specified rules. These rules are usually laid down in a file called a Makefile (it usually uses that as a filename).
The make command will take a number of parameters that will effect the way it works. In the previous section you saw how the command make config (using the Makefile in /usr/src/linux) runs a configuration script. The Linux kernel Makefile will also accept a number of different command line parameters.
These parameters include • dep, This checks the dependencies of the source files. Amongst other things it checks to see that all the files required actually exist. • clean, This cleans out any existing targets. A target is usually the end product of a make. For example in this case one of the files deleted (cleaned) will be any existing version of the kernel in the kernel source tree. This typically will not be the kernel you booted the system with. The Makefile will usually compile a version of the kernel and place it in the root directory of the kernel source tree. The new kernel must be moved to the appropriate location if the machine is to be booted with it. • zlilo, and Will place the final kernel image in a specific location on the hard-drive. The next time the system is booted it will use the new kernel. (This only works if you have lilo installed properly). On most systems the kernel will be placed into the root directory of the root file system. • zImage. Will place the final kernel image in a file called zImage in the /usr/src/linux directory. The idea is that this file is copied onto a floppy disk and that disk will be used to boot the system.
David Jones (11/30/94) Page 259 To install a kernel image onto a floppy you have to execute the following commands • cp zImage /dev/fd0 OR dd if=zImage of=/dev/fd0 Both these commands will place the kernel image onto the floppy disk. • rdev -R /dev/fd0 1 This copies into the kernel information about which hard-drive partition is the root partition. The kernel needs to know how to find the root partition when it starts up (it needs to find all the startup scripts, the init program etc.).
Typically you will not be able to use a boot floppy from your system on someone else’s system.
The main reason being is that their root partition is not always guaranteed to be the same as yours. So when the kernel goes looking for the root partition and the files it is supposed to contain it will end up being disappointed.
Exercises.
15.8. Compile a new kernel for your system. (This will typically take quite sometime).
15.9. Carry out the necessary tasks to reboot your system using the new kernel.
Conclusions
The kernel is the basic, essential component of the UNIX operating system. Without it none of the services and functionality required to run user programs will be available. The Systems Administrator is responsible for ensuring that the kernel of a system is correctly configured and will work.
User programs access the kernel's functions by using system calls.
The type of operations a Systems Administrator might perform on a kernel include • patching the kernel, • configuring the kernel, • making or compiling the kernel, and • booting with the kernel.
David Jones (11/30/94) Page 260 Review Questions
15.1. What is a kernel?
15.2. What are some of the kernel related responsibilities that a Systems Administrator must fulfil?
15.3. What is the purpose of the patch and diff commands?
David Jones (11/30/94) Page 261 Chapter 16
CRON, ACCOUNTING, AND QUOTAS
I’m not going to limit myself just because people won’t accept the fact that I can do something else. -- Dolly Parton.
Objectives
This chapter aims to introduce you to • the cron daemon, • the location and format of the crontab files, • the crontab command, • mechanisms for limiting which users can use cron. • the type of information stored by the UNIX operating systems accounting systems, • the commands last, lastcomm and other commands that access the accounting information, • the syslog system that provides a central distribution point for system status and information messages, • the BSD disk quota system, and • other methods for limiting the amount of resources users may consume.
Introduction
The facilities introduced in this chapter allow the Systems Administrator to • schedule actions to be carried out automatically at set and regular times, The cron system. • track and record the actions carried out by the system’s users, and Accounting systems and syslog. • limit the amount of resources that are consumed by these users. Quotas and other schemes.
cron
A number of the responsibilities of a System Administrator are automated tasks that must be carried out at the regular times every day, week or hour. Examples include, early every morning freeing up disk space by deleting entries in the /tmp directory or running the various security scripts described in the previous chapter.
David Jones (11/30/94) Page 262 Most of these responsibilities require no human interaction other than to start the command. Rather than have the Administrator start these jobs manually, UNIX provides a mechanism that will automatically carry out certain tasks at set times.
crond is a system daemon that will execute other programs at the times specified in the appropriate configuration files. The daemon is normally started in one of the system’s start up files (/etc/rc*).
The configuration files that control the operation of cron are called crontab files (cron tables). These files contain information about the time, date and command to execute. Different versions of UNIX store these crontab files in different locations and use a slightly different format.
On most systems the crontab files can only be modified using the crontab command. Even though the crontab files are text files they should not be modified using an editor.
Table 16.1 summarises the various components of the cron system.
Component Purpose crond the daemon that reads the configuration files and runs the commands crontab the command used to create the configuration files cron.log a file used to store the output and results of commands executed by crond /usr/spool/cron/crontabs the directory in which the configuration files are kept cron.allow cron.deny files used to control which users can create their own crontab entries (only available on some systems)
Table 16.1. cron Files, Commands and Daemons.
crontab Files
The crontab files are stored under the system’s spool directory (usually either /usr/spool or /var/spool) in the directory cron/crontab. SysV and some BSD systems will have individual crontab files for every user, while some BSD systems combine crontab entries for all users into one crontab file.
The crontab files are text files with one entry per line. Each entry contains either six (SysV) or seven (some BSD systems) fields separated by white space. Table 16.2 summarises the fields of a cron configuration file.
David Jones (11/30/94) Page 263 The additional field on some BSD sysetms indicates the username of the user the entry belongs to. This field isn’t needed under SysV systems as each user has their own crontab file.
Field Purpose minute minute of hour, 00 to 59 hour hour of the day, 00 to 23 (military time) day day of the month, 1 to 31 month month of the year, 1 to 12 weekday day of the week, either 1 to 7, or 0 to 6, username name of user executing command command command to execute
Table 16.2. Fields of a cron Entry.
Weekdays usually start with Sunday as either 0 or 1. However a few systems start the week on Monday. The manual pages on cron or crontab should list the format used a particular system.
Comments can be used and are indicated using the # symbol. Anything that appears after a # symbol until the end of that line is considered a comment and is ignored by crond.
The first five fields use any one of the following formats. • an asterix that matches all possible values, • a single integer that matches that exact value, • a list of integers separated by commas (no spaces) used to match any one of the values • two integers separated by a dash (a range) used to match any value within the range.
Some systems allow lists to contain ranges (e.g. 1,2,5-10,12), however most do not.
For example: every hour (when minutes=0) display Cuckoo Cuckoo on the system console 0 * * * * echo Cuckoo Cuckoo > /dev/console
at half past the hour, between 9 and 5, for every day of January which is a Sunday, Wednesday or Saturday, append the date to the file date.file 30 9-17 * 1 0,3,6 echo `date` >> /date.file
David Jones (11/30/94) Page 264 The commands are run by the crond daemon. This means that they are not associated with any particular terminal and so there is no place for them to get standard input or put standard output and standard error. This means if the output of these commands are to be used it must be redirected. Specific log files (like date.file from the example above) or electronic mail are common destinations.
Some systems provide a file cron.log that is used to store the output of commands executed by crond.
Exercises.
16-1. Using the manual pages discover the following a)What format does your system’s crontab files use? b)Does your system have separate crontab files for every user? c) What day starts the week on your system?
16-2. Using the format of your system write crontab entries for the following. a)remove all the contents of the directory /tmp at 5:00am every morning b)execute a shell script /root/weekly.job every Wednesday c) run the program /root/summary at 3, 6 and 9 pm for the first five days of a month
Allow and Deny
Some systems provide the ability for all users to make use of the cron system. On some systems the Systems Administrator may wish to restrict access to cron. Access can be restricted on most systems using using the files cron.allow and cron.deny.
These files contain a list of usernames, one username to a line. These files specify who can and who can't use the cron system using the following rules • if a username appears in cron.allow that user can use cron, • if a username appears in cron.deny that user cannot use cron, • if neither file exists only root can use cron, and • if cron.allow does not exist and cron.deny does but is empty then everyone can use cron.
The location of these files can differ from system to system.
David Jones (11/30/94) Page 265 The crontab Command
crontab files should not be modified using an editor. Instead the crontab command is provided. The actual format of the command may change between systems but the following provides an introduction.
1. crontab [file] or 2. crontab [-e | -r | -l ] [username]
Version 1 is used to replace an existing crontab file with the contents of standard input or the specified file.
Version 2 makes use of one of the following command line options -e allows the user to edit the crontab file using an editor (the command will perform some additional actions to make it safe to do so) -r remove the user's crontab file -l display the user's crontab file onto standard output
By default all actions are carried out on the user’s own crontab file. Only the root user can specify another username and modify that user’s crontab file.
For example: create a normal file that contains my crontab file crontab -l > my.crontab edit the normal file using an editor and make the necessary modifications vi my.crontab replace the existing crontab file with the new one crontab my.crontab
Exercises.
16-3. What would the contents of the cron.allow/cron.deny files be so that a)only root may make use of cron b)only root and one other user may make use of cron
16-4. Add entries to root’s crontab to run the shell script passwdck (created as an answer to one of the exercises from Chapter 13) every second day at 7am. Output of the script should be mailed to the root user.
David Jones (11/30/94) Page 266 Quotas and Process Resource Limits
Quotas and process resource limits allow the Systems Administrator to place limits on the amount of resources a user can consume. The main use of these systems is to ensure that individual users do not consume the majority of a resource and as a result prevent other users from making use of the system. For example, it is unlikely for it to be okay for an individual user to consume 95% of disk space. Limits can also be put in place to prevent run away process from bringing your system down.
The choice of whether or not to implement quotas and process resource limits is up to the Systems Administrator. The decision may depend on • the maturity of your users, and • the importance of your system being available.
Quotas is another area in which BSD and SysV based Unices differ. The standard SysV quota system is very limited and does not offer as much functionality as that offered by the BSD based quota system. Many recent releases of SysV UNIX offer a BSD like disk quota system.
Process Resource Limits
Both the C shell and the Korn shell support commands that implement an ineffective form of process resource limits. The type of process resources these systems can limit are specified in Table 16.3. The C shell uses the limit command and the Korn shell the ulimit command to view the limits.
Both systems use the concept of • soft limits, and Default limits applied when a process is created that can be increased by the user up until the hard limit is reached. • hard limits. System wide hard limits that cannot be bypassed. The hard limits are usually specified in the kernel and sometimes cannot be modified by the Systems Administrator.
C Shell Korn Shell Resource cputime time total accumulated CPU time filesize file largest possible file size datasize data size of a process’ data segment stacksize stack size of a process’ stack coredumpsize coredump size of a core file memoryuse memory amount of memory
Table 16.3. Possible Process Resource Limits.
David Jones (11/30/94) Page 267 Process limits suffer from a number of problems • the hard limits are hard wired into the kernel, • users may change their own soft limits, and • sometimes the limits aren’t actually enforced.
The BSD Disk Quota System
The BSD disk quota system allows the Systems Administrator to limit • the number of disk blocks a user can consume, and • the number of i-nodes a user owns (every file needs one i-node).
Under the BSD system disk quotas are handled on a per user, per file system basis. This means disk quotas can be set individually for each user on each file system.
For example: The user jonesd might have one set of quotas for the /home file system and another set of quotas for the /usr/local file system. The user munstert can have a different quota under /usr/local and no quota under /home.
The system allows the specification of two limits (similar to process resource limits) • a hard limit, and This is the absolute limit. The user will not be allowed to exceed this limit. The file system will simply refuse to carry out any request that increases the size. • a soft limit. The soft limit serves as a warning. If the user passes the soft limit they will receive a warning message. After a set number of warnings the soft limit will begin to act like a hard limit.
How it Works
There are various commands associated with the BSD disk quota system that are summarised in Table 16.4.
Command Purpose quota display a user's disk usage and quota quotacheck ensure that quota records match disk usage quotaon turn quotas on quotaoff turn quotas off repquota report on implemented quotas edquota modify a user’s quota
Table 16.4. Summary of Quota Commands.
David Jones (11/30/94) Page 268 For quotas to work the following must be carried out • the file system and the kernel must support quotas, • the quotaon command must be used to turn quotas on (usually during one of the systems start up scripts), • the file systems that are to have quotas working must have their entries in the /etc/fstab or /etc/vfstab files modified to include quotas in the options, and • entries must be added to the quota control files to set the quotas for individual users.
The quota information for each user is stored in a quota control file usually called quotas. Some systems use quota.user and quota.group to set quotas on individual users and groups.
Each partition for which quotas have been enabled will have its own quota control file. The file is kept in the root directory of the partition and contains the following information • the current disk usage for each user, and • the current hard and soft limits for each user.
quotas is a binary file and is not in a human readable form. Individual users can view their quotas by using the quota command. The quotas for all users can be viewed by using the repquota command. Typically only the root user can view another user’s quotas.
For example: A system has the following /etc/fstab file /dev/hdb1 swap swap defaults /dev/hda2 / ext2 defaults /dev/hdb2 /home ext2 defaults,usrquota /dev/hda1 /home/ftp/pub ext2 defaults none /proc proc defaults Quotas have been turned on for the /home parition (notice) the additional option usrquota. This means there should be a quotas (or on this system quota.user) file in the root directory of that partition. bash$ ls -l /home/quota.user -rw------1 root students 164368 Sep 21 13:37 /home/quota.user bash$ quota jonesd Disk quotas for user jonesd (uid 975): File system blocks quota limit grace files quota limit grace /dev/hdb2 7 9 16 6* 5 10 none This user has a very small quota and is actually over the soft limit on the number of files allowed (notice the 6*) Quotas are set for individual users by root using the edquota command. This command enters the default editor (defined by the shell variable EDITOR) and allows the root user to modify the the assigned quotas. When the editor is exited the new modified quotas will be added to the quotas file.
David Jones (11/30/94) Page 269 For example: The following provides an example of what the user sees when they use edquota bash$ edquota jonesd Quotas for user jonesd: /dev/hdb2: blocks in use: 7, limits (soft = 9, hard = 15) inodes in use: 6, limits (soft = 5, hard = 10) The root user can now modify these numbers to increase or decrease the user’s quota.
The quotacheck command is used to ensure that the record of current disk usage stored in the quotas file is correct. When quotacheck is executed it searches throughout the file system and summarises how many disk blocks and i-nodes each individual user owns and updates the statistics stored in the quotas file.
Users adding or deleting files at the same time quotacheck is being executed can result in inconsistent results. For this reason the quotacheck command should only be executed when there are no users on the system. As a result it is normally executed during system startup before users can login.
Installing Quotas
The process of installing quotas is listed below.
1. Reconfigure and recompile the kernel The file system code within the kernel must support quotas. If it doesn't the kernel will have to be reconfigured and recompiled so that it does. This will be possible only on some systems, others simply don’t support the BSD quota system. 2. Modify the /etc/rc script Quotas can be turned on or off at any time. It is usual to turn quotas on at boot time using one of the /etc/rc* files. On some systems the quotacheck command will be execute before hand (this command can take a long time to execute)
For example: echo -n 'checking quotas:' > /dev/console /etc/quotacheck -a -p > /dev/console 2>&1 /etc/quotaon -a
3. Modify the /etc/fstab or /etc/vfstab file. Each partition for which quotas are to be installed must be mounted with the appropriate option. Normally this should be done when the partition is first mounted. This is controlled by the fstab or vfstab files.
David Jones (11/30/94) Page 270 For example: The following example is of an /etc/vfstab file from a SysVR4 machine. /dev/sd0a / 4.2 rw 1 1 to /dev/sd0a / 4.2 rq 1 1 The change from w to q is a change from write to write with quotas.
4. Create the quota control file. Create an empty file called quotas on the root directory of all the file systems which are to have quotas applied. touch quotas do this as root chmod u=rw quotas
5. Set each user's quota. Use the edquota command to set the quotas for each user.
Exercises.
16-5. If your system supports them turn quotas on for the file system that contains your users’ home directories.
16-6. Create a new user and set some very strict quotas that allow a very low number of files and blocks. Experiment with the user account to see what happens when the soft and hard limits are exceeded.
BSD Accounting
Most UNIX systems automatically keep log files that summarise what is being done with various parts of the system. The majority of the information is appended onto a variety of accounting files. Most systems supply a number of commands that turn accounting on or off, display the accounting information in a (sometimes barely) human readable format, and summarise and shorten the accounting files.
Both the BSD and SysV accounting systems store similar information but use different commands to summarise and display the information. SysV accounting relies on a greater number of shell scripts and C programs SysV accounting will not be examined in this chapter..
The system administrator has two responsibilities with regards to accounting • deciding how to use the information, and Keeping the logging information doesn’t make sense unless you use it. Uses may include documentation to management stating the need for more equipment, charging users for use of the system and as a method to track what the users are actually doing.
David Jones (11/30/94) Page 271 • pruning the files. The accounting/log files grow continually and should be pruned occasionally by the administrator otherwise they can grow to consume a lot of disk space.
Accounting on some systems automatically ceases if the file system becomes more than 95% full.
The BSD accounting information is recorded by the by the kernel appending information to a specific accounting file. A number of specific commands are provided to examine the contents of those files. Table 16.5 summarises what information is kept, in what files and what commands can be used to examine the information.
Information Commands Files processes accton, sa, specified by accton command line, usually lastcomm /usr/adm/acct * current users w, who /etc/utmp connect time ac, last /usr/adm/wtmp disk usage quot, du no file printer usage pac defined by af entry in printers printcap entry *the file needs to exists beforehand Table 16.5. Summary of BSD Accounting Information.
The files that are written to by the accounting system will continue to grow and take up disk space. They should regularly be pruned. The administrator must also decide whether or not to archive these files. The simplest method to deal with these accounting files is to have entries in the crontab so that the system will automatically take care of the problem.
For example: Summarise the accounting file, this produces a much smaller file. # summarise /usr/adm/acct file into /usr/adm/savacct 0 1 * * 1-6 /etc/sa -s
Truncate the wtmp file to nothing 0 2 1 * * cp /dev/null /usr/adm/wtmp
The administrator must decide whether or not to save the old version of the accounting files for future use before truncating them.
David Jones (11/30/94) Page 272 Process Accounting
Process accounting is started by the accton command. It stores the following information about every process executed by the operating system. • executable file name, • CPU time used in user and system time, • how long the process took, • when the process started, • the process’ user and group IDs, • memory usage, • the number of characters read and written, • number of disk I/O blocks read and written, • the terminal that started the process, and • various flags about the process and its exit status.
Process accounting is usually started during the system startup scripts with a line something like if [ -f /var/adm/acct ] then accton /var/adm/acct echo -n ’ accounting’ > /dev/console fi
The information is appended onto the file specified in the accton command and is usually /var/adm/acct (on some systems /usr/adm/acct). This file becomes very lengthy reasonably quickly. The sa command can be used to produce a file /var/adm/savacct that is formed by merging the records in the acct file.
This information can be examined using the • lastcomm command, and lastcomm can be used to examine that data by individual command or user. • the sa command. On systems that support it the sa command is used to produce usage reports and summarise the accounting files. Reports generated by the sa command produce a wide range of information depending on the command line switches used.
David Jones (11/30/94) Page 273 Connect Time
The times when users log in and out are stored in the binary files • wtmp, and This file is used to record the log in and log out time of every user session on the computer. The information is accessed by the last and ac commands. • utmp. The utmp file is used to record the users that are currently logged onto the system. The information is accessed by a variety of commands including w and who.
syslog
Both BSD and SVR4 (the latest release of SysV) support the syslog system. This system is used by a variety of programs and sub-systems to direct status information and error messages to one central location, the syslogd daemon. Prior to the existence of syslogd these messages would be sent to a variety of files and locations all over the file hierarchy.
syslogd provides a central distribution point for status information and error messages and provides the Systems Administrator with the ability to direct these messages to be • appended to specific files, • forwarded to a different machine, or • sent to the consoles of all or some of the logged in users.
Where the messages are sent is controlled by the syslog configuration file /etc/syslog.conf. syslog.conf is a text file with the following format facility.level[;facility.level...] destination
facility describes where the information is coming from. Some of the possible sources are listed in Table 16.6. level describes the severity of the information (Table 16.7) and destination specifies where the message is to be sent.
Value Source kern the kernel mail the mail system lpr the printing system daemon the various system daemons auth the login authentication system
Table 16.6. Facilities that may use syslog.
David Jones (11/30/94) Page 274 Level Description emerg a system panic alert an error that requires immediate action crit a critical error err errors warn warnings notice non-critical messages info informative messages debug debugging information
Table 16.7. Levels of syslog messages.
For example: The following is an example /etc/syslog.conf taken from a Linux box. # # NOTE: YOU HAVE TO USE TABS HERE - NOT SPACES. #
kern.* /home/megan/kern.messages *.=info;*.=notice /usr/adm/messages *.=debug /usr/adm/debug *.warn /usr/adm/syslog
Exercises.
16.7. Examine the syslog.conf file on your system (if it supports it) and the various log files the syslogd sends the messages.
16.8. Modify the syslog.conf file so that the most serious messages are sent to the system console.
Conclusions
The crond daemon is used in conjunction with crontab files to schedule the automatic execution of commands at specific, regular times.
The UNIX operating system provides a number of systems that allow the Systems Adminstrator to • observe what is and has been done with the system, and • limit the percentage of resources any particular user may use.
David Jones (11/30/94) Page 275 This chapter has introduced you to • the cron system, • the BSD accounting system, • the BSD disk quota system, • the various ineffective process limit schemes, and • the syslog system.
Review Questions
16.1. List and discuss the components of the cron system.
16.2. Describe the effects of the following crontab entry * * 1 1,6,12 0-3 echo this day is ‘date‘>> date.log
16.3. Describe how the BSD disk quota system works.
16.4. What is the purpose of the syslog system?
David Jones (11/30/94) Page 276 Chapter 17
PROFESSIONAL ISSUES
Professional men, they have no cares; Whatever happens, they get theirs. -- Ogden Nash
Objectives
This chapter introduces you to some resources that a professional Systems Administrator might find useful including, • various professional organisations for Systems Administrators, • some of the books and magazines that can be useful for finding more information about Systems Administration, and • some of the sources of information on the Internet that may be useful for Systems Administrators.
Introduction
Systems Administration has only started to achieve some of the trappings of a profession over the last few years. Previously there were very few books and no professional organisations dedicated to Systems Administration. There are now numerous books and magazines discussing Systems Administration and related fields and there are a number of professional organisations throughout the world.
Professional Organisations
Belonging to a professional organisation can offer a number of benefits • recognition of your abilities, • a way to communicate with other people performing a similar job, and This enables the group to share experiences and solutions. An individual can benefit from the experience of others. • a variety of side benefits. Most professional organisations distribute newsletters that carry useful information. Many also organise annual conferences that enable professionals to gather in one place and discuss the profession (and to have some fun at the same time).
David Jones (11/30/94) Page 277. Professional organisations a Systems Administrator might find interesting include • Systems Administrators Guild of Australia (SAGE-AU), • Australian UNIX Users Group (AUUG), • Australian Computer Society (ACS), • Systems Administrators Guild (SAGE), • and, Usenix.
The SAGE Groups.
The early 90s saw the development throughout the world of a number of national SAGE groups in the United States, Australia and the United Kingdom. These groups were set up to be professional organisations for Systems Administrators.
The Australian SAGE was started in 1993 and currently has a membership of over 200 Systems Administrators. It holds an annual conference and distributes a bi-monthly newsletter (a copy of which is included in the Resource Materials).
Reading
Resource Materials Chapter 17.1
The Australian Computer Society.
The ACS is the main computing professional society in Australia servicing people from all computing disciplines. The flavour of the ACS is much more business oriented than SAGE-AU. However it does provide a number of services that can make belonging worthwhile including • various magazines, • professional development courses, and • a low cost Internet link.
The ACS is also moving towards some form of certification of computing professionals and some jobs now require ACS membership.
UNIX User Groups.
There are various UNIX user groups spread throughout the world. AUUG is the Australian UNIX Users Group and provides information of all types on both UNIX and Open Systems. Usenix was one of the first UNIX user groups anywhere and is based in the United States.
The first SAGE group grew out of the Usenix Association.
David Jones (11/30/94) Page 278. Reading
Resource Materials Chapter 17.2
Purpose
Provide you with some introductory information about the various professional organisations so that you may make a decision about whether or not to join.
Useful Books and Magazines
When a new computing person asks a technical question a common response will be RTFM. RTFM stands for read the fine (and other words starting with f) manual and implies that the person should go away and look up the documentation for the answer.
It seems like only yesterday that for a Systems Administrator RTFM meant reading the on-line man pages, some badly written manual from the vendor or maybe if lucky a newsgroup or two. Trying to find a book that explained how to use cron or how to set up NFS was a task destined to failure.
However the last couple of years has seen an explosion in the number of books and magazines available that cover Systems Administration and related fields. One of the few advantages of being an Academic is that publishers send evaluation copies of any books they have in an area in which you are teaching. The following is an annotated list of the books I have sitting on my book shelves.
This is by no means an in-depth review of every UNIX book there is. Instead it is a list of personal opinions about the books I have on hand.
There are a number of sections under which publications have been listed. Under most sections the books have been ranked in an order that represents my preference.
General Systems Administration
The main problem with many general Systems Administration books is that they are written by educators (like myself) or programmers with limited “real-life” experience in Systems Administration. Only a Systems Administrator with a number of years experience can write a really good general Systems Administration text.
David Jones (11/30/94) Page 279. 1. Evi Nemeth, Garth Snyder, Scott Seebass, UNIX System Administration Handbook, Prentice-Hall, 1989, 593 pp, ISBN 0-13-933441-6
Almost universally considered the Systems Administration bible. Covers almost every topic needed with a number of comments from the authors who are experienced Systems Administrators.
The only slight problem with this book (and one that is getting worse all the time) is that it is starting to age. For example it talks about a 474Mb hard disk that fits a 19-inch rack. This problem is about to be solved with rumours of a new edition on the horizon. Definitely a book to look out for.
2. Aeleen Frisch, Essential System Administration, O’Reilly & Associates, 1993, 440pp, ISBN 0-937175-80-3
This book is almost an equal to the Nemeth book. It offers coverage of BSD, SYSV and XENIX with additional coverage of AIX (IBM’s version of UNIX). Not as complete as Nemeth but more up to date.
Now come the also rans. While one or two might be okay they are not up to the quality of the first two.
3. Dave Fiedler, Bruce Hunter, Ben Smith et al, UNIX System V Release 4 Administration 2nd edition, 1991, 436pp, ISBN 0-672-22810-6
Specific to the SVR4 version of UNIX this book has less detail in many areas than the previous two books. Students who have used this book find it easier to understand than Nemeth but once they understand find it doesn’t take them as far.
4. Rebecca Thomas, Rik Farrow, UNIX Administration Guide for System V, 1989, 636pp, ISBN 0-13-942889-5
Written by two of UNIX World’s (now Open Computing) columnists this book covers the pre-SVR4 version of UNIX. No real point in going anywhere near this one now. Its coverage of topics is limited.
5. Levi Reiss, Joseph Radin, UNIX System Administration Guide, Osborne McGraw Hill, 1993, 640pp, ISBN 0-07-881951-2
I don’t like this book at all. It offers a gentle introduction to the art of Systems Administration that is tainted by its brevity and the apparent fact that the authors haven’t done very much real UNIX Systems Administration.
One of the really annoying habits of the book is that it is littered with “meticulously crafted scripts” and “valuable source code....for adding and removing users....and performing daily system administration tasks”.
David Jones (11/30/94) Page 280. No mention is made of public domain tools that perform the same tasks better and some of the scripts are horrible. For example the “adduser” script • doesn’t automate tasks like getting a user id or group id, • it doesn’t check if the new uid or usernames already exist, • it adds the new /etc/passwd entry by appending the information onto the end of the file, and • it uses dot files (.cshrc, .profile) that are hard wired into the script itself instead of using something like /etc/skel.
Security
1. David Curry, UNIX System Security: A Guide for Users and System Administrators, Addison-Wesley, 1992, 279pp, ISBN 0-201-56327-4
Provides a nice introduction to UNIX security and has the added advantage of having an adorable cartoon daemon character on the front cover. The most common first impression of the book is “Oh isn’t that cute!”
2. Rik Farrow, UNIX System Security, Addison-Wesley, 1991, 278pp, ISBN 0-201-57030-0
Haven’t had the time to look through this book so its second placing here is not indicative of its quality.
There is another UNIX security book that I don’t have in my library called Practical UNIX Security by Simson Garfinkel and Gene Spafford (ISBN 0- 937175-72-2). It is a book from O’Reilly and Associates and so should be of a high standard and reviews of the book have indicated this.
Shell Programming
1. Stephen Kochan, Patrick Wood, UNIX Shell Programming Revised Edition, Hayden Books, 490pp, ISBN 0-672-48448-X
This is my favourite and the one I recommend. An in-depth coverage of Bourne shell programming with mention of the Korn shell.
Lowell Jay Arthur, UNIX Shell Programming, John Wiley & Sons, 272pp, ISBN 0-471-51821-2
Takes a different approach and covers some ground that Kochan didn’t including lex and a chapter on the UNIX System Administrator. Again I haven’t had a close look at this book but on the occasions that I’ve used it as a reference it has sometimes come up short.
David Jones (11/30/94) Page 281. Using UNIX
The following are in no particular order as I have never done a sit down comparison of them all. The first two are more in the traditional style of UNIX books, a little terse but cover most things in detail. The last three are easier to read and understand but don’t cover the area in as much detail (this is of course a generalisation).
Kaare Christian, The UNIX Operating System 2nd edition, John Wiley & Sons, 455pp, ISBN 0-471-84781-X
S. R. Bourne, The UNIX System, Addison-Wesley, 351pp, ISBN 0-201- 13791-7
A golden oldie. This is the one that the Department of Computer Science at the University of Queensland set for their UNIX text and so it was my first introduction to the world of UNIX. It was by no means a gentle introduction.
Stephen Kochan, Patrick Wood, Exploring the UNIX System, 3rd Edition, SAMS, 466pp, ISBN 0-672-48517-8
Two authors I like and the feedback I’ve had from students who use this book indicate that it is a good book.
Mitchell Waite, Don Martin & Stephen Prata, UNIX System V Primer 2nd edition, SAMS, 563pp, ISBN 0-672-30194-6
David Solomon, Tanya Rodrigue et al, Using UNIX, QUE, 693pp, ISBN 0- 88022-519-X
George Leach, UNIX Self-Teaching Guide, John Wiley & Sons, 249pp, ISBN 0-471-57924-6
UNIX Programming
1. W. Richard Stevens, Advanced Programming in the UNIX Environment, Addison-Wesley, 1992, 744pp, ISBN 0-201-56317-7
The bible, everything and anything you needed to know about UNIX programming is in this book. If you plan to spend any time writing UNIX programs this is the book you need.
Marc Rochkind, Advanced UNIX Programming, Prentice-Hall, 1985, 265pp, ISBN 0-13-011800-1
An older, much thinner book that would have been recommended before the arrival of Stevens.
David Jones (11/30/94) Page 282. Andrew Oram, Steve Talbott, Managing Projects with make, O’Reilly & Associates, 1991, 149pp, ISBN 0-937175-90-0
A very useful little book that explains the make utility.
David Curry, Using C on the UNIX System, O’Reilly & Associates, 1991, 231pp, ISBN 0-937175-23-4
Another useful book in the O’Reilly Nutshell range but Stevens offers more in-depth coverage.
John Valley, UNIX Programmer’s Reference, QUE, 1991, 826pp, ISBN 0- 88022-536-X
Operating Systems Internals
Maurice Bach, The Design of the UNIX Operating System, Prentice-Hall, 1986, 471pp, ISBN 0-13-201757-1
Most consider this book the bible of the internal structure of UNIX. It covers much of the functionality of the kernel with good explanations and easy to follow pseudo code. Even though its age starts to show in places it is still a good book. It is based on the System V Release 2 version of UNIX
Berny Goodheart, James Cox, The Magic Garden Explained: The Internals of UNIX System V Release 4, Prentice Hall, 1994, ISBN 0-13-098138-9
A book that is quickly replacing the Bach book. It is essential if you are working with SVR4 versions of UNIX and want to know how the internals work.
The third book in the world of the internals of the UNIX operating system covers another version of UNIX (BSD 4.3). I don’t have a copy but here are the details.
Samuel Leffler, Marshall McKusick, Michael Karels, John Quarterman, The Design and Implementation of the 4.3BSD UNIX Operating System, Addison-Wesley.
Network Programming
Bill Rieken, Lyle Weiman, Adventures in UNIX Network Applications Programming, John Wiley & Sons, 1992, 446pp, ISBN 0-471-52858-7
A good book that provides some easy to understand explanation of the concepts involved.
David Jones (11/30/94) Page 283. Richard Stevens (author of Advanced Programming in the UNIX Environment) also has a book in the network programming field. My copy is on loan at the moment so I can’t supply the details. However like his programming text this book is considered by many to be the book of UNIX network programming.
Magazines
The magazines are not ranked in any sort of order and do not include those magazines supplied by the various professional organisations like AUUG, SAGE-AU (the SAGE-AU newsletter in particular is a wonderful publication with a particularly talented editor :) ) and Usenix.
Open Computing
A bit like the Byte magazine for UNIX and open computing. Doesn’t provide much in-depth technical knowledge, more a collection of helpful hints, product reviews and news.
Sys Admin
A technical journal for UNIX Systems Administrators. Includes articles that explain in detail technical areas of Systems Administration. The only one of its type to my knowledge.
Information Sources on the Network
The nature of the Internet means that it provides a variety of information sources that are useful to Systems Administrators. This section serves to list some of them. It assumes that you are familiar with basic Internet concepts and will not explain them.
A connection to the Internet can be justified solely by the advantage it provides to a Systems Administrator. As a source of timely, cheap and expert opinion and advice nothing can beat “the net”.
Newsgroups
There are numerous newsgroups set up to serve Systems Administrators in general and Systems Administrators of particular machines. Table 17.1 lists some of the more obvious.
David Jones (11/30/94) Page 284. Newsgroup Purpose comp.org.usenix Discussions related to the Usenix association aus.org.sage SAGE-AU discussion and announcements comp.UNIX.admin discussion about generic UNIX administration comp.sources.unix postings of source code to various UNIX utilities and programs comp.security.unix discussion about security and UNIX comp.os.linux.* a range of newsgroups about the Linux operating system comp.os.* various newsgroups discussing most operating systems
Table 17.1. Useful Newsgroups.
Vendor Information
Most vendors and software publishers now have a presence on the net. Microsoft, Novel and Sun are all on the net. They can be reached at Internet addresses using the format company.com.
For example: Microsoft has ftp and WWW servers (as does Sun and Novell) at ftp.microsoft.com www.microsoft.com
The network servers of these various companies are often the quickest and easiest method to discover information about • the companies’ products and plans, and Most companies will place press releases and product information onto their network servers. • any patches and bug fixes. Any slight updates or fixes to a companies products are usually available from their network servers.
Conclusion.
Systems Administration is a fulfilling, active and growing profession that is changing in tune with the technology. It is important to keep up to date and in touch with what is going on in the industry if you are not to become an anachronism.
Todays Systems Administrator can now choose from a wide array of books, magazines, professional organisations and the Internet to provide the necessary information to achieve this.
David Jones (11/30/94) Page 285. Solutions to Exercises Appendix A
Appendix A
SOLUTIONS TO EXERCISES
It is a commonplace of modern technology that problems have solutions before there is knowledge of how they are to be solved. -- John Kenneth Galbraith
Chapter 2
2.1. a) echo * all the files b) echo *[!0-9] all the files that don't end in a number, memo2.sv c) echo m[a-df-z]* any file starting with an m followed by any lowercase letter other than e. d) echo [A-Z]* all files starting with an upper case letter e) echo jan* all files starting with jan f) echo *.* all files with a name that contains a . UNIX is not MS-DOS there is no need for a file to have a . in its name. g) echo ????? any file with a five character name h) echo *89 any file ending with 89 i) echo jan?? feb?? mar?? jan85 jan86 jan87 jan88 feb86 mar88 j) echo [fjm] [ae] [bnr] * feb86 jan12.89 jan19.89 jan26.89 jan5.89 jan85 jan86 jan87 jan88 mar88
2.2. a) ls wc -1 Count the number of lines output by the ls command. This has the affect of counting how many files are in the current directory. When output from ls is sent to the screen it is usually put into columns. Whenever the output from ls is sent to a pipe it is sent one filename per line. Try the following commands ls and ls cat Do you see the difference.
David Jones (11/30/94) Page 286 Solutions to Exercises Appendix A
b) rm ??? remove all files with three character filenames from the current directory c) who wc -1 Count the number of lines output by the who command. In effect this counts the number of users who are currently logged onto the computer. d) mv progs/* /usr/steve/backup Move all the files from the progs subdirectory into the directory /usr/steve/backup e) ls *.c wc -1 See a). This command counts the number of files with .c extensions in the current directory. f) rm *.o Remove all files with a .o extension from the current directory g) who sort Sort the output of the who command in alphabetical order. h) pwd Display the current working directory i) cp memo1 .. Copy the file memo1 into the parent directory j) plotdata 2>errors & Run the program plotdata in the background and redirect standard error so that it is written to the file errors rather than to the screen.
2.3. a) no, to be able to obtain a directory listing a user needs to have read and execute access on a directory. b) yes, the user astaff has the required privileges because it is a member of the group owner. c) yes, astudent can obtain a directory listing. However if the other access permissions on the jonesd directory were changed so that others had no access at all astudent would not be able to get a directory listing of the docs directory. To access a subdirectory a user must have at least execute access on all the parent directories. d) yes, astudent can create a file in the docs directory. To be able to create a file in a directory you need write access.
2.4. a) 605 b) 777 c) 073
2.5. a) --x--x--x b) r-xr-x--- c) rwxr-x---
David Jones (11/30/94) Page 287 Solutions to Exercises Appendix A
2.6. If you can't see the value for shell by just typing set try one of the following commands: echo $SHELL echo $shell
2.7. Yes the shell variable should change if you run a different type of shell.
2.8. The output of the command echo Multiply is signified by the * symbol will be Multiply is signified by the file_list symbol where file_list will be all the filenames in the current directory.
2.9. The output of the following two lines string=hello there how are you echo $string will be hello The shell variable is assigned only the first word. The shell interprets the space after hello as separating out a command and ignores the rest of the arguments. Using quote characters as follows would have solved the problem string="hello there how are you"
2.10. I've use the touch command to create the file but there are a number of methods that could've been used. touch accepts a file name and creates a file of that name if one already doesn't exist. If one does exist it updates the time the file was last modified. a) touch stars\* b) touch "hello my friend" touch hello\ my\ friend c) touch \"goodbye\" d) rm stars\* "hello my friend" \"goodbye\"
2.11. a) echo "** hello **" output: ** hello ** The " " cause the shell to treat the * character as a normal character. b) echo this is a star * output: this is a star file_list file_list will be the list of all files in the current directory. The shell interprets the * as a wildcard character and replaces it with the appropriate list of filenames.
David Jones (11/30/94) Page 288 Solutions to Exercises Appendix A
c) echo ain\\\\'t you my friend output: > The shell will be waiting for some more input. The shell interprets the \\\\ to mean that it should treat two \ characters as normal characters and display them. This leaves the command line with one '. The shell now waits for another '. At the > prompt enter a ' character. d) echo "the output of the ls command is `ls`" output: the output of the ls command is ls_output ls_output will be the output of the ls command. e) echo 'the output of the pwd command is `pwd` output: the output of the pwd command is `pwd` The ' ' quotes around the command means that anything inside them is treated as a normal character and just displayed on the screen.
Chapter 3
3.1. Fairly self explanatory. Just follow the instructions.
3.2. #!/bin/sh echo The current directory is `pwd` and it contains `ls`
3.3. a) e) and f) are valid variable names The others are not because a shell variable name must start with either a letter or an underscore, and can only contain letters, numbers or an underscore (not #.)
3.4. a) ls $home b) mv ${home}temp ${home}temp2
3.5. #!/bin/sh echo $0 accepted $# command line arguments echo they were $*
$0 should be used here as it allows the shell procedure to be renamed and it will still work as specified.
3.6. #!/bin/sh if [ $# -eq 0 ] then echo No parameters were passed. exit 1 fi echo $0 accepted $# command line arguments echo they were $*
David Jones (11/30/94) Page 289 Solutions to Exercises Appendix A
3.7. #!/bin/sh if [ $# -eq 0 ] then echo usage: $0 filename echo $0 will test to see if filename exists exit 1 fi if [ -f $1 ] then echo The file $1 exists. else echo The file $1 does not exist. fi
3.8. #!/bin/sh if [ $# -eq 0 ] then echo usage: $0 filename echo $0 will test to see if filename exists exit 1 fi for filename in $* do if [ -f $filename ] then echo The file $filename exists. else echo The file $filename does not exist. fi done
David Jones (11/30/94) Page 290 Solutions to Exercises Appendix A
Chapter 4
4.1. #!/bin/sh
if [ $# -eq 0 ] then echo USAGE: $0 list_of_filenames echo $0 will return what type of files they are exit 1 fi
file_type() { if [ -d $1 ] then type=directory elif [ -h $1 ] then type="symbolic link" elif [ -c $1 ] then type="character device file" elif [ -b $1 ] then type="block device file" elif [ -p $1 ] then type="named pipe" elif [ -f $1 ] then type="normal file" else type="something very unusual" fi }
for filename in $* do file_type $filename echo $filename is a $type done
David Jones (11/30/94) Page 291 Solutions to Exercises Appendix A
4.2. #!/bin/bash ok() { echo -n '(y,n) ==> ' read response # put the response into uppercase # it makes the comparison easier response=`echo $response | tr [a-Z] [A-Z]` case $response in N | NO) ;; Y | YES) ;; *) response=ERROR;; esac }
ok echo $response
4.3. #!/usr/local/bin/bash
while read num1 num2 do echo $num1 + $num2 = `expr $num1 + $num2` >> totals done < amounts
4.4. #!/usr/local/bin/bash
trap "echo $0 received signal 15" 15
while [ 0 -eq 0 ] do # do nothing useful echo > /dev/null done
4.5. a) any text starting with abc, followed by any character, followed by def b) first letter a or A, second letter b or B, third letter c or C followed by dD c) matches the end of a line d) any text starting with zero or more characters that aren't s or S, followed by zero or more occurrences of one of a e i o u followed by zero or more occurrences of any lower case letter e) the text hello^there$friend (remember ^ and $ only have special meaning if they are at the start or the end of an RE) f) any text starting with either a b c and ending with a *
David Jones (11/30/94) Page 292 Solutions to Exercises Appendix A
g) any occurrence of the text [hello there. where the . can be repeated zero or more times, and the text must be at the end of the line
4.6. a) grep ^j /etc/passwd b) grep ^j.*bash$ /etc/passwd c) grep .*:.*:.*:[0-9][:0-9][^0-9] /etc/passwd The .*: are used to ignore the first few fields until we get to the group field of /etc/passwd. The [0-9][:0-9][^0-9] takes into account the following situations (for the group number) • the first character is a number and the second is a : (if the group is less than 10) • the first character is a number and the second is a number (if the group is less than 100) • it also checks to see that the third character is not a number (this rules out group numbers 100 or greater) There is still one instance when this will fail and that is when the group number is less than 10 and the first character in the GCOS field (the next field after the group field) is a number.
4.7. a) 1,$s/UNIX/UNIX\[tm\]/g b) /BEGIN/,/END/s/Write/Writeln/g c) 1,$s/^>//
4.8. a) delete all lines from the line after the current line until the end of the file b) for all lines in the file replace all occurrences of OSF with Open Software Foundation c) from the first line in the file to the next line that contains end find any number of lower case letters followed by a space and then by any number of numeric characters and swap them around. i.e. abc 43222 becomes 43222 abc
4.9. awk 'BEGIN { FS=":" } { total=$3+$4 printf "%s, %s %3d/n", $1, $2, total }' results.dat
David Jones (11/30/94) Page 293 Solutions to Exercises Appendix A
4.10. #!/usr/local/bin/bash
if [ $# -eq 0 ] then echo Usage: $0 group_id echo $0 will return a list of users belonging to the group id exit 1 fi
for group in $* do awk 'BEGIN { FS=":" count=0 } /.*:.*:.*:'$group':/{ print $1 count++ } END { if ( count !=0 ) printf "Group '$group' has %d users\n", count else printf "Group '$group' has no users\n" }' /etc/passwd done
Chapter 5
5-1. Factors that might affect policy etc. might include • the work carried out at the site has a high security factor A machine used by one of the Armed Forces to contain war plans will have a higher security level than one used by undergraduates to play games. • the relationship between the Sys Admin (or site management) and the users If the users are seen as unimportant by the Sys Admin or site management they are likely to have less resources and less personal touches (e.g. usernames like ba045 instead of david, more restrictions on access etc). For example, undergraduates at a University compared to a programming project team at a company that makes it many from programmers writing code. • the size of the site, the number of users, the amount of administrative support for the Sys Admin and a whole range of other issues.
5-2. Other areas that might require policy include • methods for outside access to the on-site machine Do you allow anyone in the outside world to gain access via modems and the network? Or do you implement firewalls, one-time passwords and dialback modems? • restrictions or guidelines for what the Sys Admin may do The Sys Admin can only access user files if certain conditions are satisifed. • definitions of what define acceptable use of the machine by users
David Jones (11/30/94) Page 294 Solutions to Exercises Appendix A
Chapter 6
6-1 The location and format in which to specify an alias will be system dependent. On a Linux machine using smail (smail is the mail system) the file to modify will be /usr/lib/aliases. The format of the entry will look something like temp:david
This defines an alias temp. All mail sent to temp will actually be forwarded to the account with the username david.
6-2. Remember that .login and .logout are files used by the C shell only. If your login shell is bash or another of the Bourne shell variants then these files will not be executed.
Add the following to your .profile (if you are using a Bourne shell derivative) or to your .login (if you using a C shell derivative). echo Logged in `date` > $HOME/account.usage Add the following to your .logout echo Logged out `date` > $HOME/account.usage
6.3 & 6.4. These questions are best answered for your system by • looking at the manual pages for passwd, • and looking for an /etc/shadow file
6-5. #!/usr/local/bin/bash # # FILE: create_mail # PURPOSE: Create a new mail file for a user with #correct permissions. Login name is first #parameter # AUTHOR: David Jones # HISTORY: 18/9/94 Created #
MAIL_DIR=/usr/spool/mail
# check to see parameter passed in if [ "$1" = "" ] then echo 'ERROR: expecting login name as first parameter' exit 1 fi
David Jones (11/30/94) Page 295 Solutions to Exercises Appendix A
# check to see if file already exists if [ -r $MAIL_DIR/$1 ] then echo 'ERROR: mail file already exists' exit 2 fi
# create the file touch $MAIL_DIR/$1 chown $1 $MAIL_DIR/$1 chgrp mail $MAIL_DIR/$1 chmod 660 $MAIL_DIR/$1
6-6. There are a number of possible solutions there are two solutions listed below. Which solution do you think will take the most time? Use the time command to find out.
#!/usr/local/bin/bash # # FILE: get_uid.sh # PURPOSE: Using commands other than awk obtain #the next free UID number from the #/etc/passwd file # AUTHOR: David Jones # HISTORY: 18/9/94 Created #
getuid() { NEXT_UID=`sort -t: -n +2 -3 /etc/passwd | cut -d: -f3 | tail -1` NEXT_UID=`expr $NEXT_UID + 1` } getuid echo $NEXT_UID
There is a problem with the above solution in how it picks the UID. The question asked for the next free UID but in most cases what you really want here is the next free UID between a certain upper and lower value. For example, on my machine aldur there is a predefined user nobody with a UID of 60000 so the above solution will return a number greater than that leaving about 59000 free UIDs below it. Also as mentioned in the chapter it is often a good idea to leave UIDs less than 100 to the system users. However on a number of systems there are not 100 system users to start with.
The following solution using awk includes a specification of a lower and upper bound for the UID.
David Jones (11/30/94) Page 296 Solutions to Exercises Appendix A
#!/usr/local/bin/bash # # FILE: get_uid.awk # PURPOSE: Using awk obtain the next free UID #number from the /etc/passwd file # AUTHOR: David Jones # HISTORY: 18/9/94 Created #
# next uid must be > LOWER and < GREATER LOWER_UID=100 UPPER_UID=60000
getuid() { awk 'BEGIN { FS=":" UID=0 } { if (($3>UID) && ($3>'$LOWER_UID') && ($3<'$UPPER_UID')) UID=$3 } END { printf "%d\n", UID }' < /etc/passwd > /tmp/get_uid.$$
NEXT_UID=`cat /tmp/get_uid.$$` NEXT_UID=`expr $NEXT_UID + 1` rm /tmp/get_uid.$$ } getuid echo this is it $NEXT_UID
6-7. #!/usr/local/bin/bash # # FILE: make_username # PURPOSE: Accept a firstname and a lastname as #parameters # Convert them into a username #consisting of the #first seven letters of the surname #and the first initial # AUTHOR: David Jones # HISTORY: 18/9/94 Created # # check that we got everything if [ $# -ne 2 ] then echo -n ’ERROR: expecting both firstname and ‘ echo ’lastname as parameters’ exit 1
David Jones (11/30/94) Page 297 Solutions to Exercises Appendix A
fi
INITIAL=`echo $1 | cut -c1` REST=`echo $2 | cut -c1-7`
username=$REST$INITIAL
Chapter 7
7-1 & 7-2. These two are up to you to explore on your system.
7-3. All terminals should be character devices.
7-4. In most cases you should find the device file for a terminal owned by the user that is logged into the terminal and they will have read/write access (for fairly obvious reasons). The group owner will typically be the group tty and the group will have write access.
Terminals not being used will have different permissions. The user becomes owner of the terminal during the login process
7-5. ls -l /dev > device.listing or on some systems that have multiple sub-directories under /dev try ls -lR /dev > device.listing
7-6. Deleting the device file associated with a current session will not affect the current session. However once logged out you will not be able to log back in on that same terminal. The operating system automatically runs a couple of processes (explained in a later chapter) which display the login: prompt. These process need to be able to access the device file for the terminal if they are to work.
Don’t forget to use mknod to recreate the device file.
7-7. The manual pages for system calls are stored in section two of the manual. By typing man kill it will display the first manual page for something called kill. This will generally be the user command kill. User commands are stored in section one of the manual and are thus found first.
man 2 kill Tells the system to display the manual page for kill that appears in section two of the manual.
Some systems display the user with a choice of all matching man pages.
David Jones (11/30/94) Page 298 Solutions to Exercises Appendix A
For example: bash$ man kill
1 kill(1) kill - terminate a process by default 2 kill(2) kill - send a signal to a process or a group of processes
View which man page? (q to quit) [1] 7-8. This question entails (shock! horror!) some mathematics but it is fairly simple maths at that.
direct blocks = 10 * 512 single indirect = 256 * 512 double indirect = 256 * 256 * 512 triple indirect = 256 * 256 * 256 * 512
The actual total works out to be something in excess of 8 Gigabytes. This is a very large figure, larger than normal. Mainly because on a real system a 512 byte block is not going to be able to contain 256 block pointers. (Under System V a 1K block contains 256 block addresses).
If System V uses 1K blocks that can contain 256 block addresses and the i-node uses the above structure (10 direct blocks and a single, double, triple indirect block) what is the biggest size file System V can handle?
7-9. With a hard link you should see that the i-node numbers are exactly the same but the i-node numbers will differ with the soft link.
7-10. The permissions for a file are stored on the i-node. Since a hard link uses the same i-node as the file it points to if you change the protection on either you change the protection for both.
7-11. Since a soft link uses a different i-node changing the permissions on the link or the original file won’t affect the other.
7-12. bash$ cd / bash$ ls -i 5422 Mail 159 home 163 proc 8795 UNIX 33 bin 3 lost+found 164 sbin 168 usr 34 dev 161 mnt 166 stand 287 var 48 etc 162 opt 11928 tmp bash$ ls -id / 2 /
David Jones (11/30/94) Page 299 Solutions to Exercises Appendix A
7-13. Use the mount command without parameters to discover existing mounted file sysetms and examine the file /etc/fstab or /etc/vfstab to discover those that are mounted at startup. On some systems some file systems may be mounted automatically using individual mount commands in the systems startup scripts (talked about in a later chapter).
7-14. If you enter the command man mount and look at the files associated with this command (a section towards the end of the man page) you may find mention made of the fstab, vfstab, or other files your system uses.
7-15. Under Linux you might do the following • insert a floppy disk • enter the command mkfs -t ext2 /dev/fd0 (or /dev/fd1) depending on which drive you placed the floppy into • enter the command mount -t ext2 /dev/fd0 /mnt
7-16. It may take a number of attempts to successfully ruin the floppy’s file system.
7-17. You should now know that UNIX buffers information in memory before writing it to disk. Most UNIX operating systems also have a background process running that wakes up every 30 seconds or so and flushes the buffers (writing them all to disk). This process is usually called update, fsflush or bsdflush (does it appear on your system?)
The command sync forces the buffers to be flushed straight away. One of the first things that happens when you use umount is that it flushes the buffers.
Chapter 8
8-1. bash$ cat > $HOME/tmp/temp.dat this is temp.dat
tar is one command where the switches don’t need the -.
David Jones (11/30/94) Page 300 Solutions to Exercises Appendix A
8-2. bash$ tar xvf backup.tar $HOME/tmp/temp.dat
The above may or may not work depending on whether or not the version of tar you are using automatically makes absolute path names into relative paths (by removing the first /). If it does it will not find any file in the archive starting with a / in the tape archive (which is what the extract command has told it to do). In that case you will have to repeat the command but put in the full relative path for the file you wish to extract.
You should be careful where the file you are attempting to extract will be placed. If you are extracting a using relative path the file will be placed relative to your current directory. If not it will be placed at the absolute location specified by the path.
8-3. dd if=/dev/fd0 of=$HOME/floppy.file You might change the floppy device file if you are using another device.
8-4. dd if=temp.tar.old | gzip - > temp.tar.gz
Chapter 9
9-1 & 9-2. For you to answer.
9-3. One definition of a daemon is a process that has no controlling terminal (not a strictly correct definition). By default, most ps commands do not display all processes. To display all processes you generally have to use some command line arguments. These arguments will differ from system to system but -ax or -ae are good ones to start with. If these don’t work examine the manual page for ps.
One of the columns displayed by ps will be the controlling terminal. For a process without a controlling terminal some systems will display ?. Working on this premise the following command may work ps -ae | grep \?
David Jones (11/30/94) Page 301 Solutions to Exercises Appendix A
Chapter 10
10-1. A task for you to perform.
10-2. Those of you using the bash shell may have problems changing the keys for erase, though changing intr appears to work as expected (under bash). The following is an example of it done using the Bourne shell.
$ stty -a line = NTTYDISC; speed 9600 baud erase = DEL; kill = ^u; min = 6; time = 1; intr = ^c; quit = ^|; eof = ^d; eol = ^`; start = ^q; stop = ^s; parenb -parodd cs7 -cstopb -hupcl cread -clocal -loblk -ignbrk brkint ignpar -parmrk inpck istrip -inlcr -igncr icrnl -iuclc ixon -ixany -ixoff isig icanon -xcase echo echoe echok -echonl -noflsh opost -olcuc onlcr -ocrnl -onocr -onlret -ofill -ofdel $ stty erase \^i change the erase character to control-i $ stty -a line = NTTYDISC; speed 9600 baud erase = ^i; kill = ^u; min = 6; time = 1; intr = ^c; quit = ^|; eof = ^d; eol = ^`; start = ^q; stop = ^s; parenb -parodd cs7 -cstopb -hupcl cread -clocal -loblk -ignbrk brkint ignpar -parmrk inpck istrip -inlcr -igncr icrnl -iuclc ixon -ixany -ixoff isig icanon -xcase echo echoe echok -echonl -noflsh opost -olcuc onlcr -ocrnl -onocr -onlret -ofill -ofdel $ d^?^?^? try to use the backspace key three times d: not found $ stty erase ^? change erase back to the backspace key
10-3. This should cause no real problems and you should see how it changes the output of the commands.
10-4. Something for you to complete.
10-5. Much the same result can be achieved by changing your current terminal to something completely wrong. If your terminal is currently set up as a vt100 change it to a tvi912b and see how this affects the output of various commands.
10-6. The following is what happens on my system. The reason being is that vi has looked up the system’s terminal configuration database looking for myterm. It hasn’t found an entry so it doesn’t know about any extended capabilities (like how to clear the screen, position a character at a specific location on the screen etc). It then defaults back to what it calls open mode which means no extended features (one line at a time much like ed). bash$ TERM=myterm bash$ vi /etc/passwd
David Jones (11/30/94) Page 302 Solutions to Exercises Appendix A
myterm: Unknown terminal type I don't know what kind of terminal you are on - all I have is 'myterm'. [Using open mode]
10-7. The following solution is for a system using terminfo. The solution for a system using termcap is to bring the /etc/termcap file into vi and then search for your terminal name (e.g. /vt100).
bash$ echo $TERM vt100 bash$ ls /usr/lib/terminfo/v/vt100 /usr/lib/terminfo/v/vt100 bash$ infocmp -C vt100 # Reconstructed via infocmp from file: /usr/share/lib/terminfo/v/vt100 vt100|vt100-am|dec vt100 (w/advanced video):\ :am:mi:ms:xn:xo:bs:pt:\ :co#80:li#24:\ :DO=\E[%dB:LE=\E[%dD:RI=\E[%dC:UP=\E[%dA:ae=^O:as=^N:\ :cd=50\E[J:ce=3\E[K:cl=50\E[H\E[J:cm=5\E[%i%d;%dH:\ :cs=\E[%i%d;%dr:ct=\E[3g:ho=\E[H:k0=\EOy:k1=\EOP:\ :k2=\EOQ:k3=\EOR:k4=\EOS:k5=\EOt:k6=\EOu:k7=\EOv:\ :k8=\EOl:k9=\EOw:kb=\b:kd=\EOB:ke=\E[?1l\E>:kl=\EOD:\ :kr=\EOC:ks=\E[?1h\E=:ku=\EOA:nd=2\E[C:\ :r2=\E>\E[?3l\E[?4l\E[?5l\E[?7h\E[?8h:rc=\E8:sc=\E7:\ :se=2\E[m:so=2\E[1;7m:sr=5\EM:st=\EH:ue=2\E[m:\ :up=2\E[A:us=2\E[4m:
Chapter 11
11-1 & 11-2. Both for you to answer on your system.
Chapter 12
12-1. If you receive a host not found message there are number of possible problems • No such machine of that name exists (user error, maybe even mis-typed or misspelt). • Your machine is set up only to use /etc/hosts to perform name resolution and the machine you asked for doesn’t have an entry in the /etc/hosts. • You have incorrectly specifed your name server (its IP address might be 138.77.1.1 but you’ve entered 138.77.1.11 in the resolv.conf file). • Your name server is down. • Your name server is incorrectly set up. • Any and all problems associated with name servers are being suffered by any of the name servers that must be contacted to finally get the IP address.
David Jones (11/30/94) Page 303 Solutions to Exercises Appendix A
12-2. Let’s say the user type telnet sunsite.unc.edu • first aldur would have to obtain the IP address of sunsite.unc.edu as outlined in the study guide • aldur would then place the datagrams containing the information onto its local ethernet • the gateway machine on that network would pick the datagram up and place it onto the next network (in this case CQUs) fibre optic backbone • the gateway machine between the University and the outside world would pick the datagram up and place it onto the fibre optic cable running from Rockhampton to Brisbane • another gateway in Brisbane would pass it onto another fibre cable to get to Melbourne • a machine in Melbourne would send it across the Pacific, and • so on until it reaches sunsite.unc.edu
12-3. Use the command outlined in the chapter.
12-4. bash$ arp -a pol.cqu.EDU.AU (138.77.37.28) at aa:0:4:0:b:1c jasper.cqu.EDU.AU (138.77.1.1) at aa:0:4:0:b:1c bash$ ping bertha bertha.ucq.edu.au is alive bash$ arp -a pol.cqu.EDU.AU (138.77.37.28) at aa:0:4:0:b:1c bertha.cqu.EDU.AU (138.77.37.37) at aa:0:4:0:b:1c jasper.cqu.EDU.AU (138.77.1.1) at aa:0:4:0:b:1c
12-5. The standard daemons for smtp are usually smtpd or maybe smail (amongst others). They are typically started up in the systems startup files.
12-6. telnet jasper 21 Trying 138.77.1.1... Connected to jasper.cqu.edu.au. Escape character is ‘^]’. 220 jasper.cqu.edu.au FTP server....ready. help ....list of valid commands user anonymous log on as an anonymous user 331 Guest login ok, send your complete e-mail ... pass david@pol send my e-mail address as a password
David Jones (11/30/94) Page 304 Solutions to Exercises Appendix A
12-7. Any number of uses of the netstat command will answer this. Try -s or refer to your systems manual pages.
12-8. Another for you to discover, try looking at the manual pages.
Chapter 13
13-1. Follow the instructions contained in the study guide.
13-2. The order of hostname and -l username can differ from system to system. On the machine aldur running SysV -l username comes first. a) rsh jasper ls /etc b) rsh jasper cat /etc/passwd > $HOME/passwd c) ls | rsh jasper ’cat - listing’ d) rsh jasper tar cvf - > directory.tar
13-3. a) This /etc/exports is invalid because the directory /usr/users is a sub-directory of /usr and is on the same device. This breaks the second rule outlined in the study guide. b) Valid. (in very old versions of this question the file contained the directory /usr/local/temp which would make it invalid, but this directory has been removed from the question)
13-4. Try the command find / -name PUT_NAME_HERE -print
13-5. Most of these daemons will be started by the systems startup files.
13-6. Simplest method is to examine the output of an appropriate ps command and see if any of the daemons mentioned in the study guide are running. You can also look in the /etc/fstab, /etc/vfstab and /etc/exports files.
13-7. Follow the steps outlined in the study guide.
13-8. Performing a traceroute that passes through a large number of gateways increases the possibility of two different runs being different. The Internet has the ability to use different routes to get to the same destination.
David Jones (11/30/94) Page 305 Solutions to Exercises Appendix A
13-9. a) bash$ nslookup Default Server: jasper.cqu.edu.au Address: 138.77.1.1
> ls -a cqu.edu.au [jasper.cqu.edu.au] Host or domain name Alias mailroom janus.cqu.edu.au library onyx.cqu.edu.au civax topaz.cqu.edu.au info janus.cqu.edu.au cq-pan bertha.cqu.edu.au www janus.cqu.edu.au newsroom janus.cqu.edu.au bindmaster jasper.cqu.edu.au
b) > ls -h cqu.edu.au > hosts [jasper.ucq.edu.au] ########## Received 545 records. looking at the file hosts revealed 136 hosts
c) > server cc.uq.oz.au Default Server: cc.uq.oz.au Address: 130.102.128.5 > www.cc.uq.oz.au Server: cc.uq.oz.au Address: 130.102.128.5 Name: dingo.cc.uq.oz.au Address: 130.102.2.14 Aliases: www.cc.uq.oz.au
David Jones (11/30/94) Page 306 Solutions to Exercises Appendix A
Chapter 14
14-1.
#!/usr/local/bin/bash # # FILE: passwdck # PURPOSE: Perform the following checks on the /etc/passwd file # * that all accounts have passwords # * that all accounts have usernames # * only the account root has uid of 0 # * only the proper accounts have gid of 0 # * only additions performed by Sys Admin have been # made. Keeps a copy of /etc/passwd in a protected # file $ROOTDIR/.old.passwd # * file permissions on /etc/passwd haven’t been # altered # AUTHOR: David Jones # HISTORY: 2/9/94 Created #
PASSWORD_FILE='/etc/passwd' # file containing encrypted passwords PERMISSIONS="-r--r--r--" # permissions for /etc/passwd file ROOTDIR=/root # directory in which .old.password is stored
# Check that all accounts have passwords awk -F- '{ FS=":" } { if ( $2=="" ) printf "passwdck: ERROR user %s has no password\n", $1 }' < $PASSWORD_FILE
# display user ids for accounts without usernames # display usernames of accounts other than system accounts with uids of 0 awk '{ FS=":" } { if ( $1=="" ) printf "passwdck: ERROR userid %s has no username\n", $3
if ( $3=="0" && $1!="root" && $1!="setup" && $1!="powerdown" && $1!="sysadm" && $1!="checkfsys" && $1!="makefsys" && $1!="mountfsys" && $1!="umountfsys" ) printf "passwdck: ERROR user %s has uid of 0\n", $1 }'< /etc/passwd
# check for changes in the password file diff /etc/passwd $ROOTDIR/.old.password > /dev/null if [ $? -ne 0 ] then echo passwdck: ERROR $PASSWORD_FILE has been illegally changed. RETURN=1 fi
# check on the permissions for the passwd file if [ ! "`ls -l /etc/passwd | cut -d' ' -f1`" = $PERMISSIONS ] then echo passwdck: ERROR illegal permissions on /etc/passwd fi
David Jones (11/30/94) Page 307 Solutions to Exercises Appendix A
14-2. The following is a very coarse grained answer to the question.
#!/usr/local/bin/bash # # FILE: fs_check # PURPOSE: Perform the following checks on the security of the # filesystem # AUTHOR: David Jones # HISTORY: 2/9/94 Created #
ROOTDIR=/root # directory in which summary files are kept
# directories or files to be checked could be placed into a specific file DIRECTORIES='/dev/hda* /etc/passwd'
# create filename for new summary file # FORMAT is $ROOTDIR/summary."todays_date" SUMMARY_FILE="$ROOTDIR/summary.`date | cut -d: -f1 | tr ' ' _`"
# move previous summary file mv $ROOTDIR/summary* $ROOTDIR/old.summary for directory_name in $DIRECTORIES do ls -lR $directory_name >> $SUMMARY_FILE done diff $ROOTDIR/old.summary $SUMMARY_FILE
14-3. The main difference is the addition of the s in the executable fields for the owner and the group. This means that the passwd program is both setuid and setgid.
# which passwd /usr/bin/passwd # ls -l /usr/bin/passwd -r-sr-sr-x 1 root sys 20240 Nov 23 1991 /usr/bin/passwd
14-4. Methods exist using symbolic and numeric permissions chmod 4??? filename set uid chmod 2??? filename set gid Where ??? is replaced with the normal numeric permissions and filename is the name of the file chmod u+s filename set uid chmod g+s filename set gid
14-5.
bash$ i_am The real uid is 201 The effective uid is 201 The real gid is 1 The effective gid is 1
David Jones (11/30/94) Page 308 Solutions to Exercises Appendix A
# chown root i_am # chmod u+s i_am # ls -l i_am total 80 -rwsr-xr-x 1 root other 5600 Oct 2 19:47 i_am
bash$ i_am The real uid is 201 The effective uid is 0 The real gid is 1 The effective gid is 1
14-6.
# chgrp root i_am # ls -l i_am # chmod g+s i_am total 80 -rwsr-sr-x 1 root root 5600 Oct 2 19:47 i_am
bash$ i_am The real uid is 201 The effective uid is 0 The real gid is 1 The effective gid is 0
14-7. The find command has a switch that can very easily find all the setuid and setgid programs. Add the following to the fs_check program just before the diff command.
find / -perm -4000 -print >> $SUMMARY_FILE find / -perm -2000 -print >> $SUMMARY_FILE
Chapter 15
15-1. The type of things you’ll find in the kernel include device drivers, network code, memory management code, process management code, inter-process communication code, code to boot the system, file system code (for all the various file systems), character and block I/O code etc.
Many systems will divide the different sections into separate directories.
David Jones (11/30/94) Page 309 Solutions to Exercises Appendix A
15-2. The following command will give a good indication though it is probably going to pick up a number of object files in addition.
pol:/usr/src/linux# find . -name \* -print | wc -w 1108 pol:/usr/src/linux#
15-3. On this particular system the MANPATH variable wasn’t set but examining the manual page for the man command revealed that the manual pages (in one format) were stored in the directory /var/share/catman. Under this directory there were a number of directories corresponding to the various different sections of the manual.
Taking a listing of the contents of the correct directory gives a list of the system calls that are documented in the manual pages. bash$ ls /var/share/catman/2 access.Z getgroups.Z msgop.Z setpgid.Z sys3b.Z acct.Z getmsg.Z munmap.Z setpgrp.Z sysfs.Z adjtime.Z getpid.Z nice.Z setsid.Z sysinfo.Z alarm.Z getrlimit.Z open.Z setuid.Z termios.Z brk.Z getsid.Z pause.Z shmctl.Z time.Z chdir.Z getuid.Z pipe.Z shmget.Z times.Z chmod.Z intro.Z plock.Z shmop.Z uadmin.Z chown.Z ioctl.Z poll.Z sigaction.Z ulimit.Z chroot.Z kill.Z priocntl.Z sigaltstack.Z umask.Z close.Z link.Z priocntlset.Z signal.Z umount.Z creat.Z lseek.Z profil.Z sigpending.Z uname.Z dup.Z memcntl.Z ptrace.Z sigprocmask.Z unlink.Z exec.Z mincore.Z putmsg.Z sigsend.Z ustat.Z exit.Z mkdir.Z read.Z sigsuspend.Z utime.Z fcntl.Z mknod.Z readlink.Z stat.Z vfork.Z fork.Z mmap.Z rename.Z statvfs.Z wait.Z fpathconf.Z mount.Z rmdir.Z stime.Z waitid.Z fsync.Z mprotect.Z semctl.Z swapctl.Z waitpid.Z getcontext.Z msgctl.Z semget.Z symlink.Z write.Z getdents.Z msgget.Z semop.Z sync.Z
15-4 to 15-6. Both are system specific tasks read your manual pages.
15-7. diff abc.doc def.doc > patch.file patch abc.doc < patch.file > newabc.doc
15-8 & 15-9. The process here will depend on your system.
David Jones (11/30/94) Page 310 Solutions to Exercises Appendix A
Chapter 16
16-1. Questions for you to answer for your particular system.
16-2. a) 0 5 * * * rm /tmp/* b) No time specified so we’ll stay with 5 am. (Sunday is 0 on this system) 0 5 * * 3 /root/weekly.job c) 0 15,18,21 1-5 * * /root/summary
16-3. Again some questions for you to answer for your system.
16-4. 0 7 2,4,6,8,10,12,14,16,18,20,22,24,26,28,30 * * /root/passwdck | mail root
On some systems the redirection to the mail command is not necessary as the system automatically sends output of commands run by cron as mail.
16-5, 16-6. Again questions for you to answer on your system but you should be able to follow the steps in the study guide and the documentation for you system.
16-7. Simply cat /etc/syslog.conf
16-8. *.emerg /dev/console
David Jones (11/30/94) Page 311 Solutions to Review Questions Appendix B
Appendix B
SOLUTIONS TO REVIEW QUESTIONS
There ain’t no answer. There ain’t going to be any answer. There never has been an answer. That’s the answer. -- Gertrude Stein
Chapter 1
1.1. A Systems Administrator is responsible for allowing the users of a system to perform the work the need to complete as quickly, easily and efficiently as possible. The Systems Administrator will have to do anything that fulfills this responsibility.
1.2. The answer to this is open to argument. A good Sys Admin should be • technically sound in all computing fields, • emotionally stable and able to work under pressure, • have good interpersonal skills, and • have strong independent learning skills.
1.3. Much the same as that of a Systems Administrator. Enable the programs that will run on a computer to complete as quickly, easily and efficiently as possible.
1.4. Windows, Windows 95 (Chicago), OS/2, Windows NT, UNIX, and various legacy mini and main frame operating systems that still have a hold in certain niches.
1.5. The kernel is the part of the operating system that must be in main memory at all times.
David Jones (11/30/94) Page 312 Solutions to Review Questions Appendix B
Chapter 2
2.1. a) cd /usr/local b) ls /etc > listing c) wc -w /etc/passwd
2.2. A link is a mechanism that enables a file or directory to be referred to by more than one name.
2.3. Major responsibilities of a Unix shell include • executing commands • interpreting special characters • performing I/O redirection
2.4. a) Display the message the *s are out tonight' b) Creates a file count that contains the number of files in the current directory. c) Produce a directory listing of files and or directories that match the contents of the file file.list.
2.5. a) Any filename (with at least one character and you can't have any less than 1, so every file). b) Any filename that doesn't start with a lowercase character, finishes with a lowercase character and has two characters inbetween (e.g. A34z). c) Any filename that finishes with a * character followed by any other character. The \ tells the shell to treat the * as a normal character. d) Any filename that starts with *?*?abc followed by any other characters. The ' ' tell the shell to treat the * and ? as normal characters.
Chapter 3
3.1. #!/bin/sh
# make sure at least one parameter is passed in if [ "$1" = "" ] then echo ERROR: expect at least 2 numeric parameters exit 1 fi
total=0
David Jones (11/30/94) Page 313 Solutions to Review Questions Appendix B
for number in $* do total=`expr $total + $number` done
echo Total = $total
3.2. #!/bin/sh
# make sure at least one parameter is passed in if [ "$1" = "" ] then echo ERROR: expect at least 2 numeric parameters exit 1 fi
total=0 count=0
for number in $* do total=`expr $total + $number` count=`expr $count + 1`
if [ $count -ne $# ] then echo -n "$number + " else echo -n "$number " fi done
echo = $total
3.3. Some version of the test command may not recognise the more exotic file types (named pipes, character device files etc.)
#!/bin/sh if [ $# -eq 0 ] then echo USAGE: $0 list_of_filenames echo $0 will return what type of files they are exit 1 fi for filename in $* do if [ -d $filename ] then type=directory elif [ -h $filename ]
David Jones (11/30/94) Page 314 Solutions to Review Questions Appendix B
then type="symbolic link" elif [ -c $filename ] then type="character device file" elif [ -b $filename ] then type="block device file" elif [ -p $filename ] then type="named pipe" elif [ -f $filename ] then type="normal file" else type="something very unusual" fi
echo $filename is a $type done
Chapter 4
4.1.
#!/usr/local/bin/bash
choice() # get users decision on what to do # include a bit of error recovery # users choice is returned in the variable response { response=0 while [ $response -lt 1 -o $response -gt 3 ] do echo echo Do you want to echo "1) copy $1 over the top of $2" echo "2) move existing $2 to $2.old and copy over $2" echo "3) cancel the operation" echo -n "Enter you choice (1,2,3) ==> " read response
if [ $response -lt 1 -o $response -gt 3 ] then echo '**** ERROR: You must choose one of 1 2 or 3 ****' fi
David Jones (11/30/94) Page 315 Solutions to Review Questions Appendix B
done }
# # Main Program #
if [ $# -lt 2 ] then echo Usage: $0 source destination echo $0 will copy the source file to destination file exit 1 fi
if [ -r $2 ] then echo The file $2 already exists. choice $1 $2 case $response in 2) cp $2 $2.old;; 3) exit 1;; esac fi
cp $1 $2
4.2.
#!/usr/local/bin/bash
choice() # get users decision on what to do # include a bit of error recovery # users choice is returned in the variable response { response=0 while [ $response -lt 1 -o $response -gt 3 ] do echo echo Do you want to echo "1) copy $1 over the top of $2" echo "2) move existing $2 to $2.old and copy over $2" echo "3) cancel the operation" echo -n "Enter you choice (1,2,3) ==> " read response
if [ $response -lt 1 -o $response -gt 3 ] then
David Jones (11/30/94) Page 316 Solutions to Review Questions Appendix B
echo '**** ERROR: You must choose one of 1 2 or 3 ****' fi done }
# # Main Program #
if [ $# -lt 2 ] then echo Usage: $0 source destination echo $0 will copy the source file to destination file exit 1 fi
# destination is the last parameter destination=`echo $* | cut -d' ' -f$#`
# source is everything but the last parameter source_num=`expr $# - 1` source=`echo $* | cut -d' ' -f1,$source_num`
if [ -r $destination ] then echo The file $destination already exists. # pass in source in " " so it is treated as one variable choice "$source" $destination case $response in 2) cp $destination $destination.old;; 3) exit 1;; esac fi
cp $source $destination
4.3. #!/usr/local/bin/bash
space_used() # accept a username as 1st parameter # return amount of disk space used by the users home directory # in a variable usage { # home directory is the sixth field in /etc/passwd the_home=`grep ^$1: /etc/passwd | cut -d: -f6`
David Jones (11/30/94) Page 317 Solutions to Review Questions Appendix B
# du uses a tab character to seperate out its fields # we're only interested in the first one usage=`du -s $the_home | cut -f1` }
# # Main Program #
while read username max_space do space_used $username
if [ $usage -gt $max_space ] then echo $username has a limit of $max_space and has used $used >> offender fi done < disk.hog
Chapter 5
5.1. The type of question for which there is no correct solution. I wonder how many of you actually attempted it.
5.2. Log book format is a personal decision. However it should be complete and easy to use and retrieve information from.
Chapter 6
6.1. This one’s up to you.
Chapter 7
7.1. The solutions to this question can be taken straight out of Table 7.1.
7.2. Block: a disk block is the smallest unit of information that can be read from a disk drive. Typical sizes range from 512 bytes to 4Kb. Cylinder: a collection of disk tracks that are the same distance from the centre of the disk surface. File System: a) a collection of directory hierarchies. b) the code inside the kernel that performs various file and directory operations.
David Jones (11/30/94) Page 318 Solutions to Review Questions Appendix B
i-node: The index node that is used to store and reference all information about a particular file. One i-node per file. Information contained includes permissions, size, number of links and the actual data of the file.
Device File: The device file is an entry point into a particular device drive. It is characterised by a combination of major and minor device numbers.
Major Device Number: Used by device files to point to the associated device driver.
Minor Device Number: Used by device files to point to the entry point in the particular device driver.
7.3. Two methods used to mount file systems are • the mount command, and • the file system configuration files either /etc/vfstab or /etc/fstab
7.4. Decide on the number and size of partitions to be placed on the disk drive and then partition the drive. Create the appropriate file systems on each partition. Decide on the mount points for the various partitions. Mount the partitions either using the mount command or by modifying the file system configuration files.
7.5. Differences between a hard link and a soft link • hard link is limited to within a partition, soft link isn’t, • with a hard link both files point to the same i-node, • with a soft link the link has its own i-node that points to a file that contains the path of the file it is pointing to.
Chapter 8
8.1. Another question for which there is no one correct solutions. A backup strategy must balance all the characteristics discussed in this chapter.
8.2. Media: the physical medium onto which a backup is placed, tape, disk etc.
Scheduler: the person or program that decides when to perform a backup and what to backup.
David Jones (11/30/94) Page 319 Solutions to Review Questions Appendix B
Transport: the method by which the backup is transferred onto the media.
8.3. See the discussion in the chapter.
8.4. A question for you to complete.
Chapter 9
9.1. See Diagram 9.1.
9.2. You might want to shut a Unix computer down to • install a new device or kernel, • the machine has frozen due to some unrecoverable error, • as a simple maintenance process to force the operating system to perform various tasks it completes during booting.
9.3. Four reasons why a Unix computer may not reboot • a missing kernel (it isn’t there or is in the wrong place), • disk drive error causing the root file system not to be found, • other hardware errors (cpu is burnt out etc), • the kernel is configured incorrectly, • the root file system is corrupt, • there is an error in one of the initialisation scripts
9.4. Steps to be completed to shut a Unix machine down properly include • warn the users that the machine is about to be shutdown (not strictly essential but recommended) • log out all users, • kill any background processes and daemons, • sync the disks making sure that all disk buffers have been flushed, • shut the machine off.
9.5. On a Linux machine you can use the CTRL-ALT-DELETE key combination to reboot a system. If you examine the /etc/inittab file you will see an entry for when this happens. This entry tells init to run the shutdown command.
When you do you use CTRL-ALT-DELETE the script /etc/rc.d/rc.0 is executed. This would imply that it is executed by shutdown. This means that somewhere in shutdown /etc/rc.d/rc.0 should appear.
Try the command strings /sbin/shutdown
David Jones (11/30/94) Page 320 Solutions to Review Questions Appendix B
Doing this the output reveals that there is no /etc/rc.d/rc.0.
However there is call to halt. Perhaps halt executes the script. Try strings /sbin/halt.
Chapter 10
10.1. Terminal configuration files include • those files responsible for starting getty processes (/etc/inittab, /etc/ttys) • files responsible for setting the characteristics of the serial lines connecting the terminals and the computer (/etc/gettydefs) • terminal characteristic database files (/etc/termcap, terminfo)
10.2. The solution to this question will depend on your system.
10.3. Again this will depend on your system but it should resemble one of the systems outlined in the chapter.
10.4. The login process on a terminal involves • /etc/init, The first process run on a UNIX system. It is init's responsibility to ensure that each terminal has a getty process running for it. • getty , Is responsible for setting up the characteristics of the line, displaying the login prompt, getting a username and executing the login process with the username as an argument. • login, and Is responsible for logging the user into the system, obtaining and validating the password, and executing the user's login shell. • a login shell. The user's entry in the /etc/passwd file lists the program to execute as the user's login shell. The shell will in turn execute the appropriate startup scripts.
David Jones (11/30/94) Page 321 Solutions to Review Questions Appendix B
Chapter 11
11.1. It will either be SysV or BSD based.
11.2. lp and lpr are both print spoolers. They are responsible for placing user data destined for the printer into a spooling directory.
11.3. lpd and lpsched are both print daemons. They are responsible for taking information to be printed from the spooling directory and handing it over to the printer.
11.4. /etc/printcap is the BSD printer configuration file.
Chapter 12
12.1. gateway: A machine of some description (sometimes a computer) that acts as the translator between two different networks. It may sometimes perform some type of translation from one network protocol to another or it may simply pass packets from one network onto the other. port: A logical software connection through which TCP/IP protocols communicate. protocol: A specification about how two separate entities may communicate. daemon: A process designed to fulfill a particular type of request. It sleeps waiting for such a request when it arrives it wakes up performs some task and goes back to sleep. router: A machine of some description that forwards network packets to the right direction for them to reach their destination. bridge: A device used to connect to separate networks together (part of what a gateway may be).
12.2. The telnetd daemon may not be running on hades or it may be refusing to take connections from the user’s machine. The telnet program on the machine the users are coming from might be incorrect. The network between hades and the user’s machine might be incorrectly set up in some way, packets may be getting lost while travelling to hades. The user might be using telnet incorrectly (typing in the wrong command or address for hades). The DNS on the user’s machine may not be working correctly and so when the user types telnet hades the DNS is not translating it into an IP address, or perhaps it is translating it into the wrong IP address.
David Jones (11/30/94) Page 322 Solutions to Review Questions Appendix B
12.3. Both TCP and UDP are transport protocols. They provide higher level protocols with services that enable the higher level protocols to send information from one host to another according to different characteristics.
12.4. The machine name aldur may be resolved to its IP address by • a machine looking into its /etc/hosts file, • a machine checking its cache of hostnames and IP addresses, • a machine asking its nameserver for the translation (which then repeats these steps again ad nauseum until the host is resolved or it decides to give up)
12.5. Each well-known Internet protocol will typically have a daemon listening on a specified port for incoming requests. The /etc/services file specifies which protocols belong to which ports and the /etc/inetd.conf specifies which daemons to run for which services.
12.6. TCP/IP uses four layers • network access layer Responsible for the transformation of IP datagrams into a format that can be placed onto the hardware. • internet layer The Internet Protocol is the heart of this layer. Functions IP performs include defining the IP datagram and addressing scheme, and routing datagrams between remote hosts. It provides a well-defined protocol that from the higher layers (transport, application) will appear the same regardless of the hardware. This means that the transport protocols do not have to know about the different types of hardware since the Internet layer hides the hardware dependencies. • transport layer The transport layer is responsible for dividing (and putting back together) the user data into correct size datagrams for network transmission, and providing any additional transport controls such as error detection and reliable delivery. • application layer Protocols in the application layer add additional functionality by building on the transport layer.
David Jones (11/30/94) Page 323 Solutions to Review Questions Appendix B
Chapter 13
13.1. a) For this question you basically assume that everything is setup in the network etc. All that you must do is specify the location and contents of all the files needed by the r commands to operate.
The two files necessay are • /etc/hosts.equiv Which specifies which machines are trusted. • .rhosts Located in each user's home directory and specifies accounts from specified machines which are considered equivalent to the local account even though they have different usernames. I'll be using david as my username.
Gandalf
/etc/hosts.equiv
Should contain the list of machines and users on those machines which are considered trusted. bilbo david panea backup frodo root david backup arnold
/.rhosts
Contains all users who are allowed to remotely login as root. Probably would never have in a real system.
bilbo david frodo david
/home/tideyj/.rhosts
The location of this file depends where Jim's home directory is. I'll assume the above. He wants to be able to come in from bilbo and frodo as the user backup.
bilbo backup frodo backup
David Jones (11/30/94) Page 324 Solutions to Review Questions Appendix B
/home/david/.rhosts
bilbo root frodo root
Any machine (e.g. gandalf) that wishes to be a trusted host of another (e.g. bilbo) i.e. the users want to be able to remote login. That machine (gandalf) must appear in the /etc/hosts.equiv file of the other (bilbo).
Also if a account on the first machine has a different username than the account on the second machine. Then a .rhosts file must exist in the home directory of the account on the second machine.
b) The main reason why this is not a good idea is security. The security mechanism associated with the Berkeley r commands is notoriously insecure.
Chapter 14
14.1. Issues that are important for system security • importance of data and service from machine, • size and ability of the user population, • whether or not he machine is networked, • how much effort is it deemed worthwhile to invest in security, • can the users handle the inconvienience, • the type of machine and operating system being used, • physical security threat.
14.2. Methods which can improve the security of passwords include • shadow passwords, • proactive password programs, • password generators, • password aging, • password cracking, • user education, and • regular checks on the /etc/passwd file.
14.3. Possible problems with internal security of a system include • incorrectly set file permissions, • unauthorised setuid or setgid programs, • a number of buggy system programs that supply security holes
David Jones (11/30/94) Page 325 Solutions to Review Questions Appendix B
14.4. There are a number of things that can be done and are divided into two categories. Get a command-line on the machine and then get root access. • attempt to look over the shoulder of any user possible, • steal a few bags or rifle a few desks to obtain written down passwords, • try a number of known security holes that might get an account including sendmail bugs, bugs in some login programs and various other holes, • use a “network sniffer” program that catches and examines all packets being sent across a network, • change the permisions on various device files, • look for security holes in a number of system programs, • create a setuid root program when and if root access is achieved, • fiddle with the /etc/passwd file
Chapter 15
15.1. See Review Question 1.5
15.2. The Systems Administrator has a number of responsibilities related to the kernel including • ensuring the system has a valid kernel to boot with, • configuring the kernel to represent the system’s make up, • modifying the kernel code, and • compiling the kernel.
15.3. Both the patch and diff commands are related to patch files. (see the chapter for more explanation).
Chapter 16
16.1. Refer to the description in the appropriate section.
16.2. It will append the words this day is output of date command here to the file date.log. It will do this for every minute of every hour of the first day of January, June and December as long as that first day is a Sunday, Monday, Tuesday, Wednesday or a Thursday (assuming weekdays start with Sunday as 0).
David Jones (11/30/94) Page 326 Solutions to Review Questions Appendix B
16.3. The BSD disk quota system works on a per user, per file system basis. Every individual user can have individual quotas on every different file system. The quotas and current disk usage of each user is kept in a file called quotas (or maybe quotas.user) in the root directory of every file system that has quotas installed. Quotas are checked by code within the kernel of the operating system.
16.4. The syslog system provides a central place for the collection of status and error information about the system.
David Jones (11/30/94) Page 327 Index
bridge ...... 197 Special Characters C " ...... 42 # ...... 50 cancel ...... 187 #! ...... 48 captoinfo...... 181 $ ...... 42, 51 case ...... 57 $HOME/.rhosts...... 221 chgrp...... 34 & ...... 42 chmod ...... 34, 35 ' ...... 42 chown...... 34, 36 .cshrc ...... 101 compress...... 152 .exrc...... 101 cut ...... 64 .login ...... 101 D .mailrc ...... 101 .profile ...... 101 daemon...... 163 .rhosts ...... 221 datagrams...... 197 /etc/exports...... 224 DCE ...... 172 /etc/fstab ...... 134, 226 dd ...... 149 /etc/gettydefs ...... 175 Device Files...... 118 /etc/group...... 102, 107 diff ...... 258 /etc/hosts...... 204 disable ...... 187 /etc/inetd.conf ...... 213 Disk Controller...... 118 /etc/init ...... 175, 321 DNS ...... 204 /etc/inittab...... 159, 175 Domain Name Service ...... 204 /etc/passwd...... 102 DTE ...... 172 /etc/printcap...... 188, 190 dump ...... 145 /etc/profile...... 101 E /etc/rc...... 160 /etc/resolv.conf...... 206 echo...... 70 /etc/shadow ...... 102 echo escape chaaracters ...... 70 /etc/termcap ...... 181 enable...... 187 /etc/ttys ...... 176 esac ...... 57 /etc/vfstab...... 134 ethernet ...... 197 ; ...... 42 eval...... 41 ;; ...... 57 ex ...... 78 < ...... 40 exec ...... 156 <<...... 40 exit ...... 55 > ...... 40 expect...... 111 >>...... 40 expr...... 64 \ ...... 42 F ` ...... 40 {} ...... 51 file...... 64 | ...... 40 file permissions...... 30 File Systems ...... 121 A file types...... 31 accept...... 187 for ...... 61 Aliases ...... 99 fork...... 156 ARP...... 200, 209 fsck...... 136 awk...... 81 FTP ...... 200 Full Regular Expressions ...... 77 B G biod...... 212, 224 block ...... 117 gateway ...... 197 Blocks...... 124 getty ...... 157, 175, 321 Boot Block ...... 124 grep...... 64 Bootstrap Program ...... 155 gzip ...... 152
David Jones (11/30/94) Page 328. Index
H newfs...... 132 NFS...... 222 halt ...... 167 nfsd ...... 212, 224 hard link ...... 127 head...... 64 O Home Directories ...... 100 octet...... 201 host address ...... 201 host name...... 201 P hosts.equiv ...... 220 packets ...... 197 I Passwords...... 97 paste...... 64 ICMP...... 200 patch ...... 258 if ...... 56 Perl...... 82 inetd...... 212 portmap...... 224 infocmp...... 181 ps ...... 72 infocmp -C...... 181 init...... 156 R I-Node...... 124 RARP ...... 200 IP 200 rcp...... 219 K read ...... 69 reboot ...... 168 kernel...... 20 Regular Expressions ...... 75 kill...... 71 reject ...... 187 L restore ...... 145 rlogin...... 219 LAN...... 196 routed ...... 212 Limitied Regular Expressions...... 76 router...... 197 Linux ...... 19 routing...... 197 login ...... 175, 321 rpc.lockd...... 224 Login names ...... 96 rpc.statd...... 224 Login Process...... 175 RS-232 ...... 172 lp ...... 187 rsh...... 219 lpadmin...... 187 rwhod ...... 212 lpc...... 188, 193 lpd ...... 188 S lpmove...... 187 set gid ...... 32 lpq ...... 188, 192 set uid ...... 32 lpr...... 188, 193 Shell...... 38 lprm...... 188, 193 Shell Functions...... 67 lpsched...... 187 shift...... 55 lpshut...... 187 shutdown...... 166 lpstat...... 187 Signals ...... 71 ls -l ...... 30 SMTP...... 200 M smtpd ...... 212 soft link ...... 128 man...... 45 sort ...... 64 man pages ...... 44 Startup Files ...... 100 mkfs...... 132 sticky bit...... 32 mknod...... 120 strings ...... 64 mount ...... 134, 226 stty ...... 177 mount point...... 133 su ...... 108 mountd...... 224 sudo...... 111 mt ...... 150 SuperBlock...... 124 N swapper ...... 156 symbolic link ...... 128 named...... 212 syslog ...... 274 ncheck ...... 127 System Calls...... 122, 252 network protocol ...... 196
David Jones (11/30/94) Page 329. Index
T U tail ...... 64 UDP ...... 199, 200 tar...... 147 UID ...... 98 TCP ...... 199, 200 umask...... 34, 36 TCP/IP...... 197 uniq...... 64 Telnet ...... 200 until...... 63 telnetd...... 212 utmp...... 274 TERM...... 179 V test...... 58 TFTP ...... 200 vi 21, 78 tic ...... 181 W timed...... 212 tr ...... 64 WAN...... 196 trap ...... 72 while ...... 62 tty ...... 121, 177 wtmp ...... 274
David Jones (11/30/94) Page 330.