Unix Directory Structure at NERC
iTSS
November 2005 Contents
1 Introduction 3
2 The UNIX directory tree 4 2.1 Pseudo Filesystems ...... 5 2.2 Device Files ...... 6 2.3 Links ...... 8 2.3.1 Hard links ...... 8 2.3.2 symbolic or soft links ...... 8 2.4 Mounting lesystems ...... 8 2.4.1 Mounting on boot ...... 9 2.5 File Sharing ...... 10 2.6 Comparison with Windows lesystems ...... 11
3 The NERC Directory Structure 13 3.1 Local Structure ...... 15 3.2 Network Structure ...... 15 3.2.1 The /users directory ...... 16 3.2.2 The /data directory ...... 17 3.2.3 The /packages or /nerc/packages directory ...... 17 3.2.4 The /nerc directory ...... 19 3.3 Other common mount points ...... 20 3.4 Singleton machines ...... 20
2 Chapter 1
Introduction
When learning about UNIX or Linux for the rst time you will often hear that “everything is a le unless it is a process”. This may seem odd as your keyboard certainly doesn’t look like a le, neither does your monitor, and what about directories? Well, in a very broad sense, the “everything is a le” statement is true. Directories are just les containing a list of other les and things like keyboard and monitor are accessed in the same way as les and appear as special les in the directory tree. In order to read input from the keyboard for instance, it is done by reading from a le. To write a message to the console you write to a le. Thus, les are absolutely central to understanding a UNIX system.
3 Chapter 2
The UNIX directory tree
Let’s take a look at the basic directory structure of a typical UNIX system. The hierarchical nature is fairly evident with a single root (or trunk) which branches into many directories and subdirectories. The base of the hierarchy is the “/” directory which known as the “root” directory. Below this is the rest of the le system. Here is a list of directories found in the root of a Debian system: bin common programs shared by the system administrator and the users boot the kernel and startup les, possibly boot loader con guration dev les that describe hardware etc system con guration les home user home directories, mainly for stand-alone machines lib library les media contains common mount points for CD and oppy (Debian) mnt general purpose mount points net mount point for remote le systems opt used for 3rd party software proc special lesystem containing system information root system administrator home directory (Linux) sbin programs used by the system and sysadmin scratch mount point for scratch space (Arbitrary) sys special lesystem for interacting with the kernel tmp temporary space available to the system and users usr subtree of programs, libraries, documentation for the users
4 iTSS 2.1. PSEUDO FILESYSTEMS
var storage of variable les such as logs, mail, print spools etc. Underneath /usr you will nd a similar tree to this containing bin, sbin, lib etc. This is historical back when fast storage was expensive. It allowed for the /usr directory tree to be mounted from slower storage unit (likely disk) whilst the essentials on the root partition could be on fast storage (such as drum). Nowadays it is quite unusual to split /usr from the root partition and on Solaris, you will nd /bin is a symbolic link to /usr/bin. This lesystem layout will vary slightly between Solaris, Linux and Irix but not so much that you wont be able to nd your way around. It is worth men- tioning that there is an attempt to produce a standard hierarchy called FHS which should allow software and users to predict the location of installed les and directories.
2.1 Pseudo Filesystems
On all Unix systems you will come across “pseudo” les. These are les that are not related to storage space. An example would be /dev/null, known as the bit bucket. Anything written to this le is simply thrown away. Very useful for discarding unwanted output. Sometimes you will come across entire lesystems that dont exist on disk. The most obvious of these is /proc. This virtual lesystem documents kernel and process information and can sometimes be used to tune a running kernel. Here is the “content” of /proc/cpuinfo from my desktop: processor : 0 vendor_id : GenuineIntel cpu family : 15 model : 2 model name : Intel(R) Pentium(R) 4 CPU 2.40GHz stepping : 7 cpu MHz : 2399.994 cache size : 512 KB fdiv_bug : no hlt_bug : no f00f_bug : no coma_bug : no fpu : yes fpu_exception : yes cpuid level : 2 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe cid bogomips : 4751.36 This shows that I have a 2.4GHz Pentium 4 clocked at the correct speed and with half a megabyte of cache. I can also tell from this that the processor is capable of hyperthreading but this is obviously disabled (the CPU would look like two indidual cpus to the kernel so there would be further information for processor 1). Lots of interesting information is available through /proc and it is used by common system tools such as ps and top as we shall see later in the course. The
5 iTSS CHAPTER 2. THE UNIX DIRECTORY TREE
Solaris version of /proc contains similar information but it is in a less easily readable format. There are many other virtual lesystems available for Linux. /dev is some- times a dynamic virtual lesystem. This is because /dev was becoming very large as Linux distributions were lling it with device les that described every type of hardware likely to be found on a desktop system. If we can make this lesystem dynamic we can generate the correct device le as we need it. When you come across this lesystem on Linux it is known as udev (there is also a lesystem called devfs which is now obsolete). Other examples of special lesystems include sysfs which is used as an API to the kernel. Along with udev, this is very useful for interacting with hotplug devices that may require additional kernel modules and rmware. tmpfs is a lesytem that exists in memory only which is ideal for /tmp. All contents of a tmpfs lesystem are lost during a power cycle. Its also possible that you will encounter the selinuxfs lesystem which exposes the API of SELinux to userspace.
2.2 Device Files
One of the smarter ideas which the unix designers had was to make all les consist of a stream of characters. There are no structures in this stream - no record lengths or block lengths or bu er lengths, no nonsense about xed length record les and variable length record les. There is just a stream of characters. One of these characters is the newline character (ASCII NL) which is used as a record separator by text applications but unix, itself, does not care about that ( though a printing utility certainly will). Those of you have never had to struggle with data descriptors and such things on IBM mainframes or VAX’s will probably never realise just how good an idea this was. Having made this simpli cation, they then took the next step and realised that devices such as disks, tapes, terminals and so on can also be represented as streams of characters, at least as far as users are concerned. Inside the kernel, of course, there are device drivers which have to understand about SCSI commands and magnetic tape blocks and so on but this is kept hidden from the user so that, for example: echo ’Hello’ > output.txt writes “Hello” into a le called output.txt, and echo ’Hello’ > /dev/tty1 writes “Hello” onto the rst terminal. A moment’s thought should convince you that what actually happens behind the scenes is quite di erent in these two cases. /dev/tty1 is actually an interface between user-land and the terminal device driver which is part of the kernel. This may seem like ngernail-painting but it is very useful if you want to write software which is independent of the details of your hardware. In fact, all user-mode software is hardware-independent, by default, and you have to go to extraordinary lengths to make a program dependent on a particular hardware device.
6 iTSS 2.2. DEVICE FILES
If you look under the /dev directory you will nd “device les” which repre- sent all the various devices attached to your machine. It is mostly pretty obvious which device les are an interface to which hardware devices but I probably need to say a few words about the Solaris (SPARC) disk naming convention which is very general (It was designed for very large systems with hundreds, even thou- sands, of disks.) but appears unnecessarily convoluted for a small workstation. The device les which represent the disks on a Solaris machine can be found in two directories - /dev/dsk and /dev/rdsk. The names of the les in these two directories are the same and are something like c0t1d0s2. Here, c0 means controller 0 (the rst controller), t1 means SCSI target 1, d0 means LUN 0 or device 0. LUNs are not commonly used except in some RAID systems and tape libraries. s2 means slice (partition) 2. There are up to 8 partitions on a Solaris disk (s0 - s7). They are de ned using the format program. By convention, s2 is always a pseudo partition covering the whole disk - do not remove or change this ever.
Linux uses a simpler system. Disks are either /dev/hdx or /dev/sdx de- pending on whether the kernel sees them as IDE or SCSI targets. Serial ATA (SATA) disks are treated like SCSI as they employ the SCSI instruction set; the same is true for USB attached storage. Soon parallel ATA (PATA), also known as IDE, will be a deprecated format. The rst disk to be found by the kernel on boot will be labeled “a”, the second “b” and so on. The important thing to remember here is that they are labeled on boot. The label bears little relation to the SCSI target or whether the disk is master or slave on the ATA bus. The rst SCSI-type storage device to be found by the kernel will be /dev/sda and the rst PATA device will be labeled /dev/hda. Therefore /dev/sda1 signi es partition 1 on the rst disk. As the label bears little relation to the target numbering, we must be careful as the following example illustrates.
Consider that you have a server with two disks set up in a mirror con gu- ration for resilience. These disks are SCSI so the two disks are known to the kernel as /dev/sda and /dev/sdb. Suppose that the rst disk develops a fault. It is reasonable to expect that you might remove the rst disk and run with a degraded mirror on just one disk. However, when the machine comes up with only one disk, that disk will be /dev/sda no matter which disk failed. If you then “hot add” a new disk into the running system, this disk will become /dev/sdb. This in itself is nothing to worry about but when you are diagnosing problems with disks, remember that /dev/sda is not necessarily the lowest SCSI target. In the case above, it may be that /dev/sda is a higher SCSI target than /dev/sdb (normally the disks are “discovered” by the kernel during boot in the order that they exist on the bus). If you are unlucky to su er a second disk problem on this host (with no reboot in between) you must remember to check the corespondence between disk device and physical SCSI target before pulling disks out. This is most easily done by examining the boot log.
Returning to Solaris on SPARC, it was mentioned above that there are two device les for each partition. A “raw” device and a “block” device found in /dev/rdsk and /dev/dsk respectively. Always use the block device when mounting partitions, and in most other situations you will use the raw devices.
7 iTSS CHAPTER 2. THE UNIX DIRECTORY TREE
2.3 Links
In a strict mathematical sense, the unix le structure is not a simple, hierarchical tree because of links - there are two kinds of links in the unix le store.
2.3.1 Hard links To create a hard link use the command ln patha pathb.This de nes pathb as an alternative name for patha. patha and pathb do not need to be in the same directory. They do have to be on the same lesystem. The two names are completely equivalent If you edit either patha or pathb, the changes will be seen in both pathb and patha. If you remove one name, the le remains and can be accessed using the other name. You cannot de ne hard links to directories (This is not allowed due to the danger of loops occurring when traversing the le store recursively).
2.3.2 symbolic or soft links To create a symbolic link, use the command ln -s patha pathb. Again pathb is an alternative name for patha but the two names are not equivalent, pathb is a pointer to patha. If patha is removed, pathb remains but points to nothing (a dangling link). You can create symbolic links to directories as well as les. Symbolic links can point to another le system or even another computer. Be- cause of this, they are the great kludge in unix le stores. The example which I quoted earlier in Solaris (/bin pointing to /usr/bin) is typical. Almost always when someone uses a symbolic link, it is a “quick x”. Symbolic links can be nested though there is limit in the depth of nesting, again to catch loops.
2.4 Mounting lesystems
When talking about hard links in section 2.3.1, we found that these can only ex- ist within a single lesystem. A lesystem (in UNIX terms) is a single partition. This de nition becomes slightly more fuzzy when we use more advanced volume management tools but the basic premise still holds. Each lesystem resides on a single (sometimes arbitrary) slice of storage medium. On Windows (and some other operating systems) it is possible to refer to each lesystem by name (Windows uses a drive letter). However, on UNIX- like operating systems, we use the mount command to link all the lesystems together so that they look like a uni ed homogenous tree containing the entire le store. As an example, we may have our root partition on /dev/sda1 and our var partition on /dev/sda2 we would then: mount /dev/sda2 /var
So that everything in the var partition now exists in the /var directory. A lesystem can be mounted at di erent times in di erent places. This concept is used when repairing a system that cannot be booted. The host is booted from a rescue image and the lesystems on the local disks can then be mounted on arbitrary mount points and repaired.
8 iTSS 2.4. MOUNTING FILESYSTEMS
The next step is obvious - extend this concept to disks attached to other computers and this is what NFS (Network File System) is all about. mount livcomms:/local1/data /data Note the machine name followed by colon syntax. This is very common in NFS commands. You can mount, on your machine, le systems which reside on a disk attached to another machine - the NFS server. Your machine is the NFS client. I have introduced two important words here. Also, a single machine can be an NFS server and an NFS client (for di erent le systems) at the same time.
2.4.1 Mounting on boot Obviously, when a machine starts up, it has a table of mounts (both local disk mounts and NFS mounts) to be performed as part of the boot process, before users are allowed to login. These are listed in a le called /etc/fstab (/etc/vfstab on Solaris). The exact detail of the layout of this le varies a bit so RTFM! Note also that this le is used to de ne other disk space parameters such as the size and location of swap space. Here is an example of a Linux /etc/fstab # /etc/fstab: static file system information. # #
9 iTSS CHAPTER 2. THE UNIX DIRECTORY TREE
As you may be suspecting, the warning about the way that Linux names its disks in section 2.2 can come back to haunt us here. This is particularly noticeable on large le servers with many SCSI disks attached. Adding another disk can throw out all the device names and /etc/fstab will have to be rejigged. To get around this problem, it is possible to “label” each partition and this is the default method on the Redhat distributions. The label is part of the lesystem and it is a simple matter to add a label to an existing partition: /sbin/tune2fs -L root /dev/sda1 /sbin/tune2fs -L boot /dev/sda2 /sbin/tune2fs -L var /dev/sda3 or when creating the lesystem: /sbin/mkfs -t ext3 -L scratch /dev/hda7 and then your fstab may contain lines like this: LABEL=root / ext3 defaults 0 1 LABEL=boot /boot ext3 defaults 0 2 LABEL=var /var ext3 defaults 0 2 Similar utilities for labeling other lesystems such as XFS, exist. Note it is not possible to label the swap partition. Managing disks and partitions can be made a lot easier by using Logical Volume Management. There are a number of free and commercial systems available. The commonest one on Linux is LVM2. This is available when in- stalling a number of Linux distros including Redhat and Debian. I will not go into detail here, there is plenty of information available on the web.
2.5 File Sharing
File sharing on Linux and UNIX is usually done using NFS. This is a venerable application and compatible with just about any version of UNIX, Linux or BSD. Other applications for le sharing exist but this is by far the most widely used. To share a le system from a Solaris host you would use the share(1) command and /etc/dfs/vfstab. On Linux and most other systems, you would use the exportfs(1) command and the /etc/exports le. The /etc/exports le on a typical Linux leserver might look like: # exports file for Bush file server # NERC master and Bush domain exports below /local/master 192.171.136.0/24(ro,sync) /local/bush/domain 192.171.136.0/24(ro,sync) bufiles(rw,sync) # experimental (for now) switch to nerc-bush.ac.uk domain /local/nerc-bush.ac.uk/domain 192.171.136.0/24(ro,sync) bufiles(rw,sync) # application packages shares below /local/packages 192.171.136.0/24(ro,sync) budbase(rw,sync) # user shares below /local/users 192.171.136.0/24(rw,sync) /model/users 192.171.136.0/24(rw,sync) /poll/users 192.171.136.0/24(rw,sync)
10 iTSS 2.6. COMPARISON WITH WINDOWS FILESYSTEMS
/ifetrop/users 192.171.136.0/24(rw,sync) # data areas below /model/data 192.171.136.0/24(rw,sync) /poll/data 192.171.136.0/24(rw,sync) /ifetrop/data 192.171.136.0/24(rw,sync) # /var/spool/mail 192.171.136.0/24(rw,sync) /cache/condor_ckpt 192.171.136.0/24(rw,sync) Make changes to this le and then export everything in it by /usr/sbin/exportfs. Solaris is similar, here is an example of /etc/dfs/vfstab, note that long lines may be split using the “\” character. share -F nfs -o [email protected] -d "mail spool" /var/mail share -F nfs -o [email protected],rw=livcomms:livhome \ -d "/nerc structure" /local/nerc-liv.ac.uk share -F nfs -o [email protected],rw=livcomms -d "/nerc structure" \ /local/master # share -F nfs -o [email protected] -d "Solaris packages" \ /local1/packagesshare -F nfs -o [email protected] /local1/data # share -F nfs -o [email protected] -d "Home dirs" /local/users Exporting all shares listed in this le can be done by using the shareall(1) command. There are one or two things that need to be known before plunging into an implentation of NFS. 1. NFS does not do le locking. File locking is provided by an additional piece of software. Normally this is not a problem but for certain le access patterns it can be a real performance killer 2. NFS does not export any metadata associated with a le. Modern lesys- tems allow for extra le attributes to be stored alongside the le itself. These are mostly used for access control list (ACL) type functionality. Such information is not visible on les exported by NFS. These comments relate to NFSv3. NFSv4 provides more functionality and se- curity but is still not widely deployed.
2.6 Comparison with Windows lesystems
This is a bit of a mine eld - a unix le system looks very like a Windows le system (especially NTFS) but there are all sorts of di erences once you get into it. There are the obvious di erences which everyone knows about: case sensitivity line terminators path component seperator
11 iTSS CHAPTER 2. THE UNIX DIRECTORY TREE
special characters There are the ones which rarely matter (but which salesmen and the com- puter press shout about!) such as whether the maximum le size and/or the maximum le system size is 2 GB or 4 TB or whatever. Or what is the maxi- mum pathname length or the maximum lename length. When these do come up, you really have to check the documentation for the particular versions of unix and Windows which you are concerned with. Then there are the oddities such us in unix you can delete a read-only le if you have write permission for its directory. In Windows, you can’t! Sometimes, you can understand why these apparent oddities came about if you remem- ber that Windows started out as a single-user system (a Personal Computer) whereas the unix people were thinking about multi-user systems from the very beginning. Sometimes, there is no obvious explanation and you just have to accept that they are di erent! Then there are the di cult ones:
A unix le has an owner and a group associated with it and there are Read, Write and eXecute permissions for each of owner, group and other
A Windows le has 4 attributes - read-only, archive, hidden, system. More signi cantly, there are Access Control Lists which govern precisely who is allowed access to the le
These don’t map at all well into one another even at the conceptual level - for example, Windows identi es executable les by an extension (.exe, .com, .cmd or .bat). There is no equivalent of the unix executable permission bit. unix has no concept of le name extensions - people (especially people who have a Windows background) do use le names with extensions (e.g. prog.c) but that is just for their own convenience. Dot is just another character to unix with no special signi cance. If you are interested in detailed comparison of windows and UNIX lesys- tems, the Samba documentation is a good source of information.
12 Chapter 3
The NERC Directory Structure
The UNIX and Linux team of iTSS provide a set of directories on UNIX-like systems so that NERC users and NERC-speci c utilities have a uni ed structure within which to operate. The requirements for this structure are seen to be: Increased leverage of skills This is particularly relevant for those of us work- ing at many sites. We don’t want to have to spend half-an-hour guring out where things are every time we turn up at a new site! However, there are obvious and real bene ts for supporters and users who only work at one site as well e.g. transferring software. You don’t want to spend time hacking scripts from colleagues to work on your particular workstation con guration. Simplicity It must be understood easily by users and supporters alike (Cer- tainly not just systems group.) - we should avoid “clever” tricks as a matter of design. We should also have lots of README les to explain what each directory is for and the like. Small sites, especially, must not be made excessively complicated. Multi-vendor (architecture) support However, we are not trying to make all unix’s look like Solaris. That would be beyond our resources and would be rather silly anyway. (If people want a SUN they should buy a SUN!) All we intend is that people will nd the same basic structure and environment on any NERC unix system. Integrated workstation network We must think in term of the network (normally a single site LAN) and not individual workstations. Our networks should be capable of being scaled up to several hundred workstations (not several thousands as in a large university campus) - NFS and LDAP are applicable on this scale. This implies things like servers for application software and communications. Maintainability The design has to include provision for “test” versions run- ning alongside “production” versions and mechanisms for non-cataclysmic upgrades. Consistent directory structures Users should see the same directory struc- ture on all workstations. They should not have to know (at the nuts and
13 iTSS CHAPTER 3. THE NERC DIRECTORY STRUCTURE
bolts level) where particular software (e.g. An application package) re- sides. In fact it is desirable that they don’t know and support sta can move such software around or introduce a new version without having to tell users to change their scripts or other code.
Minimum of modi cations to vendor software We have always been quite clear on this one. In particular, for unix, the / and /usr les systems are likely to be completely replaced on an operating system upgrade so are best regarded as “belonging to the vendor”. We keep out of them as much as possible!
Flexibility The scheme cannot be too rigid - it must be capable of adaptation to local requirements, else it will simply be rejected. It is also important to realise that we are not claiming that our way is the only way or even that it is the best way. It often seems to be the case here that there are several, apparently equally good, ways of doing something. Nonetheless it seems right to make a choice and, unless there are overriding reasons for changing that choice, asking that all sites should fall into line. I should also point out that this is not a case of “us or them” - it is perfectly possible to have some software installed on a machine following our package scheme and other software installed using a suppliers defaults for pathnames. For example, our normal practice with the Legato Net- worker distributed backup software is to do the server installation using Legato’s own install tools but install the client software using our con- ventions under /nerc/packages/networker. The server install involves lots of special les such as the jukebox software (a kernel device driver) which would be di cult to relocate. There is no possibility of ever hav- ing multiple software versions on the server and there is normally only one server on a site. The client software on the other hand is relatively simple, consisting of a few executables and other les which can be easily relocated to any pathname, and NFS-mounting one copy of the software on many clients is a big time-saver when the software has to be upgraded to a new version.
In terms of the actual directory structures, we have designed and imple- mented using NFS mounts, a network-wide directory tree (a virtual directory structure, if you like) which is visible from anywhere on the network. Within this, every le has a unique (i.e. network-unique) pathname. This is the “net- work” name. Of course, les still have their “local” pathnames as well but only on the machine to which the relevant disk is physically attached. Users generally use only “network” names. In fact, they should be discour- aged from using “local” names. One de nition of support sta is that they are the people who have to know and understand both! That is, they have to understand how pathnames in the local le structure map into the “network-wide” le structure created by NFS- mounting the “local” directory trees. This is what we are going to discuss in the next few sections.
14 iTSS 3.1. LOCAL STRUCTURE
3.1 Local Structure
This is what is present on your own workstation before starting NFS. First, there are the root and /usr partitions - these contain (to simplify) unix. Sometimes there will also be a /var partition and/or /tmp. /var contains “ les of variable size” such as spool areas and logs. It is a good idea to have these in a separate partition in case they get too big. (It is very bad news in unix if the root le system gets 100% full!) /tmp is for temporary les (scratch area). (/var and /tmp can be directories within the root partition on a small work- station.) These are all essentially supplied by the vendor or distributor. We strongly recommend that you don’t add or or change any les in here. If you must (and you will to a limited extent) please keep a note and make a copy of the les elsewhere. The problem is that an OS upgrade often involves a total replacement of these le systems! Additional lesystems (e.g. add-on disks) we mount on /local, /local1, /local2,. . . (the numbers being SCSI targets) or /raid01,. . . . - generally some- thing which is meaningful or memorable in local hardware terms. (But remem- ber, when choosing names for mount points that the names may well have a longer life than the hardware beneath them - it can be confusing to have a mount point with a name of /local007 (meaning Controller 0, SCSI target 0, partition 7) which, after a disk reorganisation or an upgrade is mounted on Con- troller 3, SCSI target 6! Below these /local directories we have users, packages and data directories : users Below this are group directories and then the actual user home directo- ries. packages Below this are “architecture” directories and then the top-level di- rectories for the various packages or applications. data Divide up by projects or whatever the users want and are willing to pay for.
Next we are going to discuss the “network” directory structure as opposed to the “local” structure.
3.2 Network Structure
This is what the users should see and understand - it is, therefore, very simple! It is also independent of which machine you are logged into. There are four parts to this and we are going to discuss these separately :
/users : home directories
/packages : software packages
/data : shared data areas
/nerc : infrastructure.
15 iTSS CHAPTER 3. THE NERC DIRECTORY STRUCTURE
3.2.1 The /users directory Home directories are de ned in the le /etc/passwd or in the directory container for users (eg LDAP). The home directory is usually of the form /users/
/users/
geophys external itss ccs
automounter
geophys external itss ccs
servA:/localX/users servB:/localY/users/ccs
Figure 3.1: Showing the relationship between network and local structures for /users
At some sites, the mounts are triggered at the level of individual users rather than groups. This creates more NFS tra c overhead. The table which contains the mapping from the “network directory struc- ture” (eg /users/itss) to the “local directory structure” (eg servA:/localX/users/itss) is the automounter map auto users. This is typically an LDAP object but may sometimes be a NIS map. Note that most operating systems have their own idea of where users should have their home directories. Linux puts users in /home by default. The reason we use /users is purely historical but it has the advantage of seperating the networked home directories from the purely local ones. Be careful when creating users that you create the home directory of the correct sort.
16 iTSS 3.2. NETWORK STRUCTURE
3.2.2 The /data directory This is technically very similar to the /users structure. It is used for data directories or common data areas. Another use of data areas is to provide world- A readable or world-writable shared areas ; for example, with LTEX it is necessary to provide an area for automatically created fonts, generated as the system is used, (with continuing use, a cache of fonts is built up) and this is done by A linking the area where LTEX wants to put the fonts to a /data/fonts directory which is world-writable and world-readable. This is better than putting the font cache in the package directory itself since the package directory (software) can then be mounted read-only. The table which maps /data/geophys to servA:/localX/data/geophys is the automounter map called auto data. This is generally an LDAP object which applies to the entire network.
/data/
geophys images fonts www
automounter
geophys images fonts www
servA:/localX/data servB:/localY/data
Figure 3.2: Showing the relationship between network and local structures for /data
3.2.3 The /packages or /nerc/packages directory This is where we install application packages which are shared by all the worksta- tions on the network. There is a completely separate /packages/... structure (set of application software) for each architecture. That is, /nerc/packages/oracle on an SGI is physically quite di erent from /nerc/packages/oracle on a SUN. Obviously the idea is that each platform gets appropriate executables. The
17 iTSS CHAPTER 3. THE NERC DIRECTORY STRUCTURE
$NERCARCH environment variable is used to select the appropriate architecture version of a package. It is de ned when the automounter is started on a partic- ular workstation. Presently, it can be one of: solaris meaning Solaris on SPARC solx86 meaning Solaris on Intel linux meaning Linux on Intel irix meaning SGI Irix on MIPS We have also done the following architecture versions at various times in the recent, or not so recent, past: rs6000 meaning IBM AIX ultrix meaning DECstation (MIPS CPUs) hpux meaning HP 9000/700 convex meaning ConvexOS cray meaning UNICOS sun4 meaning SunOS 4.1 ip12 meaning SGI IRIX 4 alpha meaning COMPAQ TRU64 (formerly known as Digital unix or OSF/1) and we still have the code but it would need more or less tidying up if anyone wanted to use it again. In reality, I expect we could do any modern unix-like system. $NERCARCH started out as the result of the command uname -m but is now best regarded as a “NERC architecture” which is related to but not necessarily exactly the same as the vendor “architecture”. Note, for example, that it is always lower case. A new architecture (e.g. x86-64) is implemented when one needs a new set of application software for some reason, generally when the format of executable les has changed. The mount table can be either a local le (/etc/auto packages/$NERCARCH) which is speci c to one computer or an LDAP object (auto packages/$NERCARCH) in which case it applies to the entire network. We have changed our mind several times about whether this structure is /nerc/packages/... or /packages/... Currently, it is /nerc/packages for everything except linux - we could not get /nerc/packages to work with the linux automounter due the con ict in the /nerc/... trigger. This problem is probably more easily solvable now that Linux has support for direct automount maps. Mostly the /nerc/packages/... structures contain third-party application software. Examples are Oracle, SAS, StarO ce,. . . However, two packages are rather di erent /nerc/packages/utilities This contains a collection of useful “Open Source” programs and utilities. We maintain this and periodically install a new version.
18 iTSS 3.2. NETWORK STRUCTURE
/nerc/packages/local This is a similar idea but for the local support people.
Both /nerc/packages/utilities and /nerc/packages/local are in the de- fault $PATH.
3.2.4 The /nerc directory This small structure usually resides on a disk attached to the LDAP or NIS master server of each domain and is NFS-mounted (via the automounter) on each unix computer. The directories /nerc/etc and /nerc/bin contain a small number of use- ful programs. They are both in the default $PATH but /nerc/etc is common to all architectures and /nerc/bin has a di erent version for each architec- ture so it can contain binary executables whereas /nerc/etc can only contain architecture-independent executables such as shell scripts. Note how the correct architecture-dependent directories (bin and man) are mounted here. If you are working on a linux box, you get the linux version of /nerc/bin; if you are work- ing on a SUN, /nerc/bin is the Solaris version containing Solaris executables. This is not the normal unix convention for directories called “etc” and “bin” but it has become a NERC de facto standard! (Normally an “etc” directory contains con guration les and a “bin” directory contains executables and there is no provision for multiple architectures). To increase resilience, there is a local copy of the /nerc/etc and /nerc/bin directories under /etc/nerc/.... This copy is redone by the rc.nerc startup script as part of the boot process.
/nerc/man contains “man” page les for various utilities, mostly located in /nerc/bin or /nerc/etc.
/nerc/scripts contains les mostly of interest to systems group sta - this is our toolbox if you like! Be aware that we are likely to update this whenever it suits us so don’t place any of your les in here. The les (incp scripts and so on) used for installing new machines as described in our procedures (see http://www-wl.itss.nerc.ac.uk/procs/ ) can be found here under /nerc/scripts/systems/$NERCARCH.
/nerc/
motd.* les or “Message of the Day” les which are displayed at login. motd.common is applicable to all machines in the domain. motd.
19 iTSS CHAPTER 3. THE NERC DIRECTORY STRUCTURE
On the other hand, be aware that cshrc is executed every time a new shell is started so you should keep it “lean and fast”. yp If you are using NIS then this directory contains the text les which are the sources for the NIS maps for the domain. More about this in the next section. ldap But if you are using LDAP then this directory contains LDAP spe- ci c con guration and scripts. startups This directory controls which machines run which startup scripts. The startup scripts themselves can be found in /nerc/bin/rc.d and /nerc/etc/rc.d. If there is a le in the startups directory of the same name as one of the scripts, the script will be run, at boot time, on each of the machines listed (one per line) in the le. An empty startup le means that the script is run on all machines in the do- main. It will be clear that when we designed all this, we were very careful (paranoid even) about keeping directories which are maintained by central sta separate from directories maintained by local sta so that we don’t stand on each others feet. This seems to be less of an issue these days but, even so, it was probably a good design decision.
3.3 Other common mount points
Some sites have found the following useful /scratch Used for a scratch storage area. It is usually cleared out every week- end by a cron job. Probably this is not included in the backup scheme. /working Similar to /scratch/ but with a less rigourous regime. This is often used to store “working” copies of raw data which is copied from CD’s or the Internet. /diskx Disk Extender at POL. The rationale for adding these directory structures is often to have di erent backup schedules. Many sites have a requirement for keeping huge amounts of data online but it is not necessary to back it up or at least not have regular backups.o
3.4 Singleton machines
I just want to mention that we have a simple scheme involving symbolic links which we sometimes use instead of the automounter on “singleton” unix ma- chines such as the NERC nance system server which is deliberately not part of a LDAP/NFS network; this server does not share a passwd le with other machines ; it does not mount les from or serve les to other machines using NFS. It is quite deliberately a world on its own. This is generally not a desirable scenario for scienti c machines due to the maintenance overhead. It is simply a matter of using symbolic links to link /nerc to /local/master and similar so that our tar les, scripts and procedures will work in the same way on these machines.
20