Administering Linux in Production Environments Itinerary

Administering Linux in Production Environments

Itinerary

Administering Linux in § Introduction Production Environments § Production Environment Features • Recent Kernel Developments • Filesystems: Mundane and Advanced • Disk Striping and RAID

Administering • Parallel Processing and Clustering Linux in Æleen Frisch Production • Enterprise Networking Features Environments [email protected] § Deployment Examples www.aeleen.com • File and Print Servers • Enterprise User Authentication x Copyright © 1999-2001, e ponential Consulting, LLC Exponential Consulting, LLC • Beowulf Compute Servers North Haven, Connecticut, USA • Linux and Databases • Linux as an Office PC 2

3 4

What is a Production System? Commercial Applications

§ Real world § Major applications are available now: § System is a tool • Databases: Oracle, Sybase, DB2, etc. § “Money” is involved • Computational chemistry: Gaussian 98 Administering Administering Linux in Linux in • CAE: MSC:Nastran Production Production Environments Environments • Others § Keeping up

5 6

Dressing for Success

§ Tuxedo vs. Business Suit Recent § ILM: 11/15/2001 Administering Administering Kernel Linux in Linux in Production Production Environments Environments Developments

5 6

7 8

2.4 Kernel 2.4: Numbers

§ Feature Lists: § 64 GB memory • lwn.net/2001/0111/a/wwo2.4.php3 § >2GB files • January 2001 Linux Magazine § 16 Ethernet adapters Administering Administering Linux in Linux in Production Production § 10 IDE controllers Environments Environments § SMP support • Support for tons of processes Copyright © 1999-2001, Copyright © 1999-2001, Exponential Consulting, LLC Exponential Consulting, LLC • Scheduler improvement § Billions of users/groups 7 8

9 10

Current Linux Limitations 2.4: I/O

§ Modes: § Memory size: 64 GB • Separate block device and file I/O § File size: 2 TB • Raw devices § Devices: § Filesystem size: 2 TB (VFS limitations) • I2O Administering Administering Linux in Linux in • USB Production § Filesystem block size=Memory page Production Environments Environments • Firewire (IEEE1394) • 4KB (x86) • PC Card • 16KB (IA-64) • Infrared

11 12

2.4: Networking 2.4: VFS and Related Facilities

§ Better multiprotocol support § Single buffer for file caching § Rewritten network layer (firewalls, • Eliminates synchronization problems IP masquerading): § Multiple mount points

Administering Administering Linux in • Packet filtering Linux in Production Production Environments Environments • Network address translation § LVM in kernel § ATM and others § RAID rewrite Copyright © 1999-2001, Copyright © 1999-2001, Exponential Consulting, LLC § Can now mount NFS3 shares Exponential Consulting, LLC

11 12

13 14

2.4 Bonus Features 2.4.1516

§ Memory “Management” § ext3 • New swap space size “recommendations” • “experimental” • Fixed around 2.4.10 § InterMezzo filesystem

Administering Administering Linux in Linux in Production Production Environments Environments

12.1 13’

15 16

Itinerary

Filesystems: § The VFS § Local Filesystems § Journaling Filesystems Administering Mundane and Administering Linux in Linux in Production Production Environments Advanced Environments § Network Shared Filesystems § Logical Volume Manager

14 15

17 18

Spelling 101 Virtual File System (VFS)

§ Filesystem vs. file system § Kernel subsystem/layer § Provides a consistent interface for low-level file I/O Administering Administering Linux in Linux in Production Production Environments Environments § Filesystem need only implement the required functionality using that interface, and it is automatically supported

16 17

19 20

VFS Details FilesystemData Structures

§ Structured as an indirection layer § Superblock § Specifies low-level entities (objects) ... • FS metadata: label, block size, size, # inodes, … • Inodes, Files, Directories, Superblock, § Inodes Administering Administering Linux in Linux in Production Production Environments • Extended attributes Environments • Properties: file type, owners, permissions, times, #links, size, ... § ... and required/optional methods for each one • Data or Disk addresses or Single/double indirect Copyright © 1999-2001, Copyright © 1999-2001, Exponential Consulting, LLC Exponential Consulting, LLC § Directory

18 19 • = File that maps file names to inodes

21 22

VFS in Action 2.4.15+

§ Command/application invokes system call § Journal block device module: designed § VFS looks up filesystem type in kernel table to add generic journaling capabilities to the VFS § Kernel redirects call to FS-supplied method Administering Administering Linux in Linux in Production Production Environments § Method runs and accesses disk Environments • Device drivers issue needed I/O requests § Method returns descriptor to desired object Copyright © 1999-2001, Copyright © 1999-2001, Exponential Consulting, LLC Exponential Consulting, LLC • Descriptor contains pointers to functions for accessing that object as well as related data 20 (e.g.: mounted filesystem, file, inode, dentry) 21

23 24

Filesystems for Local Disks Special filesystems

§ Many, many supported types: § procfs: /proc • ext2 • minix § Pseudo-device: /dev/pts § devfs: Administering • CD-ROM: iso9660, MS Joliet extensions, Administering Linux in Linux in Production v 2.4: udf for DVD Production Environments Environments • /dev/hda => /dev/ide0/disk0/... • ufs, fat, vfat , umsdos, ntfs, sysv , affs, adfs, hfs, • devfsdto support old device names hpfs, qnx4, ... Copyright © 1999-2001, Copyright © 1999-2001, Exponential Consulting, LLC v 2.4: ufs nextstep extensions, efs, ramfs, jffs, cramfs Exponential Consulting, LLC

22 23

25 26

The ext2 Filesystem Ext2 FS Tools

§ 2GB files and 4TB filesystem § fsck.ext2 § 255 character filenames § mke2fs § SetGID directory group ownership inheritance § e2label Administering Administering Linux in Linux in Production • Selectable at mount time Production § dumpe2fs Environments Environments § Variable block sizes § tune2fs § Performance optimizations: § resize2fs Copyright © 1999-2001, Copyright © 1999-2001, Exponential Consulting, LLC Exponential Consulting, LLC • Read-ahead • Data block allocation and pre-allocation 24 25

27 28

The mke2fs Command dump2efs

dumpe2fs 1.18, 11-Nov-1999 for EXT2 FS 0.5b, 95/08/09 Filesystem volume name: Last mounted on: § -b block-size Filesystem UUID: 03d2f865-390a-4a5a -9162 -8f2fc902d3e2 Filesystem magic number: 0xEF53 Filesystem revision #: 1 (dynamic) • Default: 1024 bytes Filesystem features: (none) Filesystem state: not clean Errors behavior: Continue § -i bytes/inode or -N #inodes Filesystem OS type: Linux Administering Administering Inode count: 264928 Linux in • Default: 1 inode per 4096 bytes Linux in Block count: 528948 Production Production Reserved block count: 26447 Environments Environments Free blocks: 260025 § -m reserve% Free inodes: 184468 First block: 0 Block size: 4096 • Default: 5% Fragment size: 4096 Last mount time: Sun Dec 3 09:33:59 2000 Copyright © 1999-2001, Copyright © 1999-2001, Exponential Consulting, LLC Exponential Consulting, LLC Last write time: Mon Jan 29 17:41:57 2001 § -L label Mount count: 12 Maximum mount count: 20 Last checked: Wed Sep 6 18:54:01 2000 § -c or -l bad-block-file Check interval: 15552000 (6 months) Next check after: Mon Mar 5 17:54:01 2001 26 27 Reserved blocks uid : 0 (user root) Ò -f fragment-size Reserved blocks gid : 0 (group root)

29 30

The tune2fs Command Contributed ext2 Tools

§ -l to list superblock info § ext2fs defrag § Modify attributes (mount read-only?) § ext2fs resize • -c max-mounts § ext2fsed Administering Administering Linux in Linux in Production • -C mount-count Production § ext2undelete Environments Environments • -i check-interval[dmw] • -e error-behavior § See: “Filesystems-HOWTO” (section 6)

31 32

Journaling Filesystems Journaling Linux Filesystems

§ A single inherent advantage over § ReiserFS: 2.4.1pre7 and later (SuSE 6.4) “traditional” UNIX filesystems • www.namesys.com § ext3: coming in standard 2.4.15

Administering Administering • e2fsprogs.sourceforge.net/ext2.html Linux in Linux in Production Production • www.zip.com.au/~akpm/linux/ext3/ext3-usage.html Environments Environments § SGI’s XFS: 1 May 2001 • 64-bit; streaming video

Copyright © 1999-2001, Copyright © 1999-2001, • oss.sgi.com/projects/xfs Exponential Consulting, LLC Exponential Consulting, LLC § IBM JFS: 28 June 2001 • 64-bit 30 31 • oss.software.ibm.com/developerworks/opensource/jfs

33 34

ext3 Reiser Filesystem

§ mke2fs’s–j option § Designed for performance, especially on § Specify separate journal device: -J device directories with lots of files, as well as efficient usage of disk space § Convert existing with tune2fs –j Administering Administering Linux in • Space considerations Linux in § Tools: Production Production Environments Environments § Interoperability with/as ext2 • mkreiserfs [-b n] n must be 4 • reiserfsck

32 33

35 36

Reiser mount Options A Testimonial

§ mount -o remount,resize=new-size >From Source Forge http://ftp.sourceforge.net/ has 850GB storage, half of which is reiserfs, half is ext2. Both filesystems have been running flawlessly for > 4 months of production (actually longer, but wasn't reiserfs before). That server pushes § mount -o conv between 15Mbit and 50Mbit/sec, and pulls/syncs about 2-5Mbit/sec, 24x7. Reiser 3.5 to 3.6 Administering • Administering reiserfs also powers the CVS tree filesystem for cvs-mirror.mozilla.org Linux in Linux in (also tokyojoe.sourceforge.net), which is the one and only anonymous CVS Production Production checkout point for mozilla. That server has run flawlessly under very heavy Environments Environments § mount -o notail load since its birth. I don't get involved in kernel politics, but as a production filesystem, reiserfs is ok in my book.

34 35

37 38

XFS JFS

§ mkfs -t xfs –l size=6000b /dev/hde1 § mkfs -t jfs –s 8 /dev/hde1

§ mount -t xfs -o logbufs=8,logbsize=32768 ... § logredo device

Administering Administering Linux in Linux in Production Production Environments Environments

36 37

39 40

Comparison Journaling and dump 100% Released Use as Boot FS

In std. kernel § ext3: yes Sparse files Dyn Block size Max. size Ext.

Interop • Long history . Inode NFSv3 Resize RAID ACLs LVM Attr § ReiserFS: no . . Administering Administering Linux in ext3 ~N 2TB 4K Y N Y Y Y Y Y Y Y N n/a Linux in Production Production § XFS: yes (xfs{dump,restore}) Environments Environments Reiser Y 2TB 4K Y Y Y Y Y Y Y 4 N 4 n/a § JFS: no

XFS Y 2TB 4K Y Y Y Y Y Y N Y Y Y Y

41 42

Mounting Remote Filesystems Linux NFS

§ NFS § Kernel-based NFSv2 § Samba: smbfs § Kernel-based NFSv3 § User space NFSv2 Administering § AFS Administering Linux in Linux in Production Production Environments § ncpfs Environments

39 40

43 44

NFS Components Other exports Options

§ Daemons: § sync, async • portmap, rpc.mountd, rpc.nfsd § root_squash, all_squash

v rpcinfo - p [host] § anonuid=n1, anongid=n2

Administering Administering Linux in • rpc.lockd, rcp.statd Linux in Production Production Environments Environments

§ /etc/exports file

Copyright © 1999-2001, Copyright © 1999-2001, Exponential Consulting, LLC • dir host(options) ... Exponential Consulting, LLC /data2 dalton(rw ) pascal(ro ) henry (rw,all_squash) *.vader .com(rw,sync) 41 /data3/new 192.123.12.0/255.255.255.0( ro ) 42

45 46

Useful mount Options UID Mapping

§ bg retry=n § rpc.ugidd § map_daemon /etc/exports option § intr or soft § retrans=n Administering Administering Linux in Linux in Production Production Environments § nosuid Environments § rsize=8192,wsize=8192 (max. for v2)

43 44

47 48

NFS and Security Sharing Filesystems: Samba

§ Samba is the solution for sharing filesystems and § Limit access to specific machines; avoid printers between UNIX and Windows systems: blanket exports • www.samba.org § Export read-only whenever practical Administering Administering § Books: Linux in Linux in Production Production Environments § Don’t export group-writeable files Environments • Gerald Carter with Richard Sharpe, Teach Yourself Samba in 24 Hours (SAMS, Indianapolis, 1999); § Don’t export system files (incl. executables) ISBN: 0-672-31609-9

Copyright © 1999-2001, Copyright © 1999-2001, • Robert Eckstein, David Collier-Brown and Peter Kelly, Exponential Consulting, LLC § Don’t use the insecure option (allows access Exponential Consulting, LLC Using Samba (O’Reilly, 2000); ISBN: 1-56592-449-5 from any port) 45 46

49 50

Samba Samba CIFS § Implements SMB protocol under UNIX § Features: • Server Message Block is the native protocol for Microsoft networking file/printer sharing: • Filesystem sharing • Printer sharing

Administering Administering Linux in Linux in • Master browser Production Production Environments Environments • Domain security • Primary domain controller (alpha)

47 48

51 52

Samba Daemons Samba with inetd

§ smbd (TCP) § For inetd control, modify /etc/inetd.conf • Provides sharing services netbios-ssn stream tcp nowait root /usr/sbin/smbd smbd netbios-ns dgram udp wait root /usr/sbin/nmbd nmbd § nmbd (UDP):

Administering Administering Linux in • Handles NetBIOS name server requests Linux in § and /etc/services: Production Production Environments Environments netbios-ssn 139/tcp § Execution options: netbios-ns 137/udp • Standalone: -D

49 50

53 54

The Samba Configuration File global Section

§ /etc/smb.conf [global] hosts allow = vala, pele • [global] section hosts deny = lilith • Exported filesystem sections valid users = dagmar, @chem, @phys, @bio, @geo invalid users = root, admin, bin, system, daemon Administering • [homes] section: defaults for user home Administering max log size = 2000 KB Linux in Linux in Production Production Environments directories Environments username map = /etc/smbusers v Actual services created on the fly Map file entries: linux = translation (usually Windows)

Copyright © 1999-2001, Copyright © 1999-2001, Exponential Consulting, LLC § testparm: verify configuration file structure Exponential Consulting, LLC aefrisch = aeleen sysadmin=Administrator chem = @chemistry 51 52’

55 56

Defining Shares Connecting to Samba Shares

[chemdir] Define a directory for export. § Run net use as usual on the Windows system: path = /chem/data/new Local path to be shared . • net use s: \\dalton\chemdir comment = New Data Description of filesystem. read only = no Filesystem is not read-only. • net use x: \\dalton\\homes

Administering case sensitive = yes Filenames are case sensitive. Administering net use x: \\dalton\username Linux in Linux in Production force group = chemists Map all users to this group. Production Environments Environments read list = dagmar, @chem, @phys Users/groups w/ read access. write list = @chem Write access list. browseable= no Exclude from browse lists. Copyright © 1999-2001, Copyright © 1999-2001, Exponential Consulting, LLC admin users = chavez Administrative users. Exponential Consulting, LLC

53’ Group=NIS netgroup (&) or UNIX group (+) 55

57 58

User Home Directories Samba Utilities

[homes] § smbstatus comment = Home directories writeable = yes § smbrun valid users = %S § smbclient

Administering Administering § smbtar Linux in Linux in Production Production Environments § Effect of map files: Environments § GUI admin tool: swat • \\server\Administrator ⇒ \\server\sysadmin • Runs within a browser Copyright © 1999-2001, Copyright © 1999-2001, Exponential Consulting, LLC Exponential Consulting, LLC

54’ 56

59 60

Mounting Windows Filesystems Passwords and Samba

§ Add to [global] section: § smbmount encrypt passwords = yes • Two versions: smbfsvs. standalone security = user v smbmount //dalton/chem –c 'mount /mnt …' v smbmount //dalton/chem /mnt § Run mksmbpasswd.sh to create initial Samba Administering Administering Linux in Linux in Production Production password file: Environments Environments § mount -t smbfs \ cat /etc/passwd| /path/mksmbpasswd.sh -o username=aefrisch,password=xxx \ //dalton/chem2 /chem v Owner: root Copyright © 1999-2001, Copyright © 1999-2001, Exponential Consulting, LLC Exponential Consulting, LLC v Mode: 600 v Directory mode: 500 57 58

61 62

More Authentication Options Advanced Filesystem Features

§ Authentication by a different server: § Larger than 1 disk partition security = server password server = host encrypt passwords = yes • Expandable on the fly § Local (UNIX) authentication: § Distributed across network Administering Administering Linux in Linux in Production security = user Production Environments encrypt passwords = no Environments • Load balancing § Windows domain participation: security=domain Copyright © 1999-2001, Copyright © 1999-2001, § Faster I/O Exponential Consulting, LLC workgroup = domain Exponential Consulting, LLC password server = pdc bdc1 bdc2 encrypt passwords = yes § Fault tolerant 59 § Samba server as the PDC 60

63 64

Logical Volumes Linux Logical Volumes

§ Dynamically-resizable filesystem consisting § Logical Volume Manager (lvm) of multiple, independent disk partitions • Still developing (physical volumes), upon which a virtual § Veritas Volume Manager Administering structure is overlaid: Administering Linux in Linux in Production Production • $$$$ Environments • Volume groups/virtual disks, divisible into Environments • Logical volumes/virtual partitions, which hold

61 62

65 66

LVM LVM Commands

§ www.sistina.com/products_lvm.htm § pv : create, change, display, move, scan § Maximum filesystem size: 2TB § vg: create, change, display, ck, cfgbackup/restore, export, extend, reduce, • Limits: 99 VGs, 256 LVs remove, split, merge , scan Administering Administering Linux in Linux in Production Production Environments § Maximum logical volume: 256 GB with Environments § lv : create, change, display, extend, reduce, standard 4MB physical extent size remove, rename, scan • e2fsadmin (requires resize2fs) Copyright © 1999-2001, Copyright © 1999-2001, Exponential Consulting, LLC Exponential Consulting, LLC v Resize filesystem and its underlying logical volume in a single operation 63 64

67 68

LVM Example LVM Snapshots

§ Set partition type to 0x8E (fdisk) § Designed to ensure consistent backups § pvcreate /dev/sdb1 /dev/sdc1 • lvcreate --size nm --snapshot --name snap1 \ § vgcreate vg1 /dev/sdb1 /dev/sdc1 /dev/my_vg/homes

Administering Administering Linux in /dev/vg1/group Linux in • mount /dev/my_vg/snap1 /somewhere Production • Production Environments Environments • /etc/lvmconf/vg1.conf • Back up /somewhere § lvcreate -L 2g -n biolv -r 8 -C y • umount /somewhere

Copyright © 1999-2001, Copyright © 1999-2001, Exponential Consulting, LLC • 8 read-ahead sectors, contiguous Exponential Consulting, LLC § mke2fs … /dev/vg1/biolv § Uses standard VMM copy-on-write 65 § mount /dev/vg1/biolv /somewhere 66

69 70

Distributed Filesystems History

§ Various focuses: § AFS • Handle client network file access and outages § Coda gracefully and seamlessly

Administering Administering § Aura Linux in v Goal: remote files are indistinguishable from local files Linux in Production Production Environments • Distribute network I/O among various servers Environments § InterMezzo v High availability

67 § Lots of overlap 68

71 72

InterMezzo Concepts

§ www.inter-mezzo.org § Fileset: directory subtree (“folder collection”) • Basic unit served § Designed for high availability • Currently is the entire filesystem

Administering § A palimpsest on Coda Administering § Change log Linux in Linux in Production Production Environments • Designed to be simpler Environments • Filesystem-like journaling to track modifications § Replication • Server to (duplicate) server Copyright © 1999-2001, Copyright © 1999-2001, Exponential Consulting, LLC Exponential Consulting, LLC • Client to server after reconnect § Concurrent conflict handling 69 70 • Current: detect and die

73 74

Components Installation

§ Lento: cache manager and file server § Kernel support • User space daemon § Kernel loopback support § Presto: kernel module (intermezzo.o) § Initial RAM disk for booting Administering Administering Linux in Linux in Production Production Environments Environments § Group InterMezzo (GID: 4711) § mkizofs [-t ext3] -r fsetname -j /dev/hda n Copyright © 1999-2001, Copyright © 1999-2001, Exponential Consulting, LLC Exponential Consulting, LLC • ReiserFS and XFS: maybe

71 72

75 76

Converting Existing ext2 Configuration: Common

§ mount -t ext2 -o loop /tmp/cache /izo0 § /etc/intermezzo/serverdb § mkdir -p /izo0/.intermezzo/name/db § /etc/intermezzo/fsetdb § chgrp -R InterMezzo /izo0/.intermezzo clientA Administering § chmod 700 /izo0/.intermezzo Administering Linux in Linux in Production Production Environments § touch /izo0/.intermezzo/name/{kml,lml,last_rcvd} Environments § tune2fs -j /tmp/cache § /etc/fstab § umount /izo0 /tmp/fs0 /izo0 intermezzo Copyright © 1999-2001, Copyright © 1999-2001, loop,fileset ="test", prestodev=/dev/intermezzo0, Exponential Consulting, LLC Exponential Consulting, LLC mtpt=/izo0,cache_type=ext3,noauto 0 0

§ /etc/conf.modules 73 74 alias char-major-185 intermezzo

77 78

More Initial Setup Configuration: Server

§ # dd if=/dev/zero of=/tmp/fs0 bs =1024 count=10k § /etc/intermezzo/sysid # mkizofs -F /tmp/fs0

§ # mknod /dev/intermezzo0 c 185 0 Administering Administering Linux in Linux in Production # chmod 700 /dev/intermezzo0 Production Environments Environments

§ # mkdir /izo0

75 76

79 80

Configuration: Client Checking the Configuration

§ /etc/intermezzo/sysid § config_check

Administering Administering Linux in Linux in Production Production Environments Environments

77 78

81 82

Global File System (GFS) GFS Example

§ www.sistina.com/products_gfs.htm § Network-shared storage • Asynchronous journaling

Administering Administering Linux in • Intelligent locking mechanisms Linux in Production Production Environments v DMEP device/memexpd Environments • Large files (64-bit addressing)

Copyright © 1999-2001, § Requires supported host-bus adapter Copyright © 1999-2001, Exponential Consulting, LLC Exponential Consulting, LLC • High speed interconnect like Fibre Channel 79 • Storage is placed directly on the network 80

83 84

Setting up GFS More ...

§ Select hardware approach § Kernel configuration: lots of patches • FC=>Multiported SCSI=>Network Block Device • GFS & hardware (FC, for example) v Future: Gigabit Ethernet TCP/IP storage § Build & distribute GFS software • DMEP hardware=>IP daemon Administering Administering Linux in Linux in § Configure Production § Plan configuration: Production Environments Environments • Lock server • Topology

85 86

Configuring GFS Configuring ...

§ Pool: • Abstraction for § gfsconf shared storage devices • Lock sources • Subpools for • Callback ports devices of Administering Administering • Timeouts Linux in same type Linux in Production Production Environments Environments • Clients • STOMITH methods v “Shoot the other machine in the head” Copyright © 1999-2001, Copyright © 1999-2001, Exponential Consulting, LLC Exponential Consulting, LLC § Failover for memexpd 83 § Cluster information device (cidev): shared partitions 84 • Still potential bottleneck which holds metadata

87 88

Architecture

Also 2-layered: • GLOCKS at filesystem level (lock caching) Disk Striping • Modules to talk to devices

Administering Administering and Linux in Linux in Production Production Environments Environments RAID

85 86

89 90

RAID Software RAID Levels

§ Redundant Array of Inexpensive Disks § Linear § Choices § 0: Disk striping for performance • Software • Best large transfer I/O bandwidth Administering Administering Linux in Linux in Production Production Environments • Hardware Environments • No loss of storage capacity v Controller § 1: Disk mirroring v Standalone device Copyright © 1999-2001, Copyright © 1999-2001, • Best data redundancy Exponential Consulting, LLC Exponential Consulting, LLC • Good performance on small transfers 87 88

91 92

More RAID ... RAID 5

§ 4: Disk striping with parity disk • Best fault-tolerant sequential file access • Not vulnerable to single disk failures Administering Administering Linux in Linux in Production • Parity disk is a bottleneck for writes Production Environments Environments § 5: Disk striping with rotating parity block

93 94

Hybrid Levels Configuring RAID

§ RAID 0+1 § Enable kernel support • Mirroring of a striped disk: two striped disks which are • May need patches mirrors of one another. Data is striped across each stripe v ftp.kernel.org/pub/linux/daemons/raid set, and the same data is sent to both striped disks. Thus, v Red Hat installs for you Administering this RAID variation provides both I/O performance Administering Linux in Linux in Production advantages and fault tolerance. Production § Special files: /dev/ mdx Environments Environments § RAID 1+0 § Configuration file: /etc/raidtab • Striping across mirror sets: Similar in intent to RAID 1+0, • mkraid device

Copyright © 1999-2001, it provides equivalent performance advantages and slightly Copyright © 1999-2001, Exponential Consulting, LLC better fault tolerance in that it is easier to rebuild the RAID Exponential Consulting, LLC § Persistent superblock device after a single disk failure (since the data on only one • Automatic detection of RAID entities disk is affected). 91 92 • raidstart/raidstop to control manually

95 96

Kernel Kernel Support

§ make xconfig=>Block Devices • Multiple devices driver support • Autodetect RAID partitions Administering Administering Linux in Linux in Production Production • RAID levels Environments Environments v Linear v RAID-0

92.1 93

97 98

Sample /etc/raidtab Entries RAID 0+1

raiddev /dev/md0 raiddev /dev/md0 raiddev /dev/md0 raiddev /dev/md2 raid-level 0 raid-level 1 raid-level 0 raid-level 1 nr-raid -disks 2 nr-raid -disks 2 nr-raid -disks 2 nr-raid -disks 2 chunk-size 64 persistent-superblock 1 chunk-size 64 persistent-superblock 1 persistent-superblock 1 device /dev/sdc1 persistent-superblock 1 raid-disk 0 device /dev/sdc1 device /dev/md0 device /dev/sdc1 device /dev/sdb1 raid-disk 0 raid-disk 0 Administering raid-disk 0 Administering device /dev/sdd1 Linux in raid-disk 1 Linux in raid-disk 1 device /dev/md1 Production device /dev/sdb1 Production Environments Environments raid-disk 1 raid-disk 1 raiddev /dev/md1 raid-level 0 nr-raid -disks 2 chunk-size 64 Copyright © 1999-2001, Copyright © 1999-2001, Exponential Consulting, LLC Exponential Consulting, LLC persistent-superblock 1 device /dev/sde1 raid-disk 0 device /dev/sdf1 94 95 raid-disk 1

99 100

RAID 5 with a Hot Spare General RAID Considerations

raiddev /dev/md0 § RAID 0 stripe size matters for performance raid- level 0 nr-raid-disks 5 • Best value high dependant on typical I/O transfer size persistent-superblock 1 device /dev/sdc1 v Defaults are poor for large I/O operations raid- disk 0 Administering device /dev/sdd1 Administering • No substitute for testing (trial and error) Linux in Linux in Production raid- disk 1 Production Environments device /dev/sde1 Environments § Underlying filesystem block size = 4KB raid- disk 2 device /dev/sdf1 • mke2fs -b 4 ... raid- disk 3 Copyright © 1999-2001, device /dev/sdg1 Copyright © 1999-2001, Exponential Consulting, LLC Exponential Consulting, LLC § Don't overload controllers raid- disk 4 device /dev/sdh1 § Spend the money if you have it space -disk 0 96 97 • RAID 5 overhead ~ 23% !!

101 102

Configuring Compute Servers

§ Parallel program execution Parallel • SMP: shared memory • Distributed parallel: Beowulf and others Administering Processing Administering v Simulates shared memory for discrete systems Linux in Linux in Production Production Environments Environments • Can be combined and Clustering § Clusters • High availability Copyright © 1999-2001, Copyright © 1999-2001, Exponential Consulting, LLC Exponential Consulting, LLC

98 99

103 104

Beowulf Two Extremes

§ Stone Soupercomputer (Oak Ridge): § www.beowulf.org stonesoup.esd.ornl.gov § www.extreme-linux.org § An idea, obsession, religion Administering Administering Linux in Linux in Production Production Environments Environments

100 101

105 106

The Other End of the Spectrum Creating a Beowulf System

§ The Hive (NASA goddard): § Hardware/software installation newton.gsfc.nasa.gov/thehive/ § Interconnect • Ethernet or better

Administering Administering § Kernel changes Linux in Linux in Production Production Environments Environments • Generally integrated into kernel source tree • Channel bonding § Parallel computing environment Copyright © 1999-2001, Copyright © 1999-2001, Exponential Consulting, LLC Exponential Consulting, LLC • Administrative setup

101’ 102’ § Modified (or parallel-ready) applications

107 108

Appropriate Hardware System Setup

§ Memory System configuration (e.g., DHCP) § I/O bandwidth • Static addressing • Disk § Distributed filesystem choice

Administering Administering Linux in • Network Linux in Production Production Environments Environments § CPU § Physical labeling: § Disk • Node name

109 110

Interconnection Configuration

§ Cluster/farm vs. to outside world § Setup vs. ongoing • Switches • Security on “world” node § Automounting Administering Administering Linux in Linux in Production Production § Distributing configuration files Environments § Channel bonding Environments § Interprocess communication § Topology • rsh

102.4 102.5

111 112 Evaluating Pre-Packaged Systems Example

§ Do the math to compute price-performance § Box with 4 Compaq EV5s plus proprietary • Whole system price vs. sum of parts interconnect for $40K v Discrete systems from commodity vendors versus v NICs Administering Administering Linux in Linux in § Compaq DS20 (2 EV6s with shared memory), Production v Switch or hub Production Environments Environments v Cabeling listing at around $35K v Time? • Weight price by processor speed Copyright © 1999-2001, Copyright © 1999-2001, Exponential Consulting, LLC Exponential Consulting, LLC v Pre-packaged systems often use older processors

104 105

113 114

Other Considerations SMP

§ Power § Enable in kernel: § Air conditioning • 2.2.x § Space • Symmetric multi-processing support: yes Administering Administering Linux in Linux in • Memory type range register (MTRR): yes Production Production Environments Environments • RTC support: yes • Advanced power management: no

115 116

Clustering Linux Virtual Server (LVS)

§ Multiple servers appear as a single system to § Not designed for single application/job users (one IP address) performance § Some load balancing § Purpose is to combine multiple systems to be

Administering Administering • Implemented via a designated server Linux in presented as a single computing resource to Linux in Production Production • Fairly simple algorithms Environments users: Environments • Linux Virtual Server (LVS) § www.LinuxVirtualServer.org

117 118

LVS High Availability Linux

§ High-Availability Linux Project: linux-ha.org • Compare to fault tolerance: quick recovery vs.

Administering Administering never failing Linux in Linux in Production Production Environments Environments • Multiple servers • Redundant communications channels • Shared disks (or distributed filesystem) Copyright © 1999-2001, Copyright © 1999-2001, Exponential Consulting, LLC Exponential Consulting, LLC • Resource groups: everything needed for some service/application to work 110.1 111

119 120

Current Components Commercial Products

§ heartbeat: detects failed servers § TurboLinux Cluster Server § mon: service monitoring daemon • $995 for 2 nodes; $1995 for unlimited § fake: provides IP-address takeover • www.turbolinux.com Administering Administering Linux in Linux in § SGI/SuSE: FailSafe (in progress) Production § Architecture for a more elaborate facility Production Environments Environments § Legato Cluster § LifeKeeper (SteelEye)

112’ 113

121 122

Piranha Project (Red Hat) Piranha Options

§ Free! § Routing: § ≡LVS • Network Address Translation (NAT) • Direct

Administering Administering Linux in Linux in • IP Tunneling Production § Control daemon Production Environments Environments § Scheduling § GUI administration tool • Round robin § Failover vs. Clustering Copyright © 1999-2001, Copyright © 1999-2001, • Fewest connections Exponential Consulting, LLC Exponential Consulting, LLC • With/without weighting (assigned, load averages) § sources.redhat.com/piranha 114’ 115

123 124

Typical Piranha Setup Piranha Components

§ One routing node § IPVS and other kernel code • Supports one or more virtual servers: § lvs daemon: manages routing table v IP address,protocol,port triple § nanny daemon: monitors a server, updates Administering • Visible from the “external” world Administering Linux in Linux in routing info as appropriate Production Production Environments • Connected to a private network holding the real Environments servers § pulse daemon: handles failovers § Multiple servers § piranha: GUI configuration/management tool Copyright © 1999-2001, Copyright © 1999-2001, Exponential Consulting, LLC • Do real work Exponential Consulting, LLC § /etc/lvs.cf: configuration file • Static data only! 116 117

125 126

Standard Services Review

§ DNS Enterprise • Dynamic v www.technopagan.org/dynamic/ddns-primer.html • Load balancing Administering Networking Administering Linux in Linux in v www.cs.twsu.edu/~hcvillia/acads/project Production Production Environments Environments § DHCP Features § Automounting

127 128

High Speed Networking MLPPP

§ 100 mbps Ethernet: all many sites need § Multilink PPP: used to combine two or more PPP links into one, higher bandwidth virtual connection • All mass-market chipsets supported • Any connections can be used: modems, ISDN (WANs) • Issues with dual speed switches • Linux implementation is basic: www.linux-mp.terz.de

Administering v Autonegotiation works best if both ends have it Administering Linux in Linux in § Configuration Production Production Environments Environments • Kernel patches § Gigabit Ethernet: common chipsets supported • Updated pppd • Start multiple lines:

Copyright © 1999-2001, Copyright © 1999-2001, pppd /dev/ttyS0 multilink Exponential Consulting, LLC Exponential Consulting, LLC wait for ppp0 interface to be up as usual § www.beowulf.org/ linux/drivers/ pppd /dev/ttyS1 multilink mp -join ppp0 mp -nonp noip wait for ppp0 interface to be up as usual 120 121 pppd /dev/ttyS2 multilink mp -join ppp0 mp -nonp noip

129 130

Virtual Private Networks TCP Wrappers

§ Adds (primarily) host-level access control to § VPNs use the public Internet as the inetd-based network services communications link between sites • Data is tunneled in encrypted form § tcpd replaces service executable in /etc/inetd.conf: #service socket prot wait? user program args telnet stream tcp nowait root /usr/sbin/tcpd telnetd Administering • Protocols: Administering Linux in v Linux in Production PPTP: many security problems Production § /etc/hosts.allow and .deny control access Environments v IPSec: emerging standard Environments • allow v ssh: See David Sifry, “Creating VPNs with Linux,” Linux : telnetd : LOCAL, .mycomp.com, 192.100.43 Magazine, Spring 1999. • deny: ALL : ALL

Copyright © 1999-2001, • Virtual Private Server (VPS): Linuxcare Copyright © 1999-2001, Exponential Consulting, LLC Exponential Consulting, LLC§ Logs to syslog facility • www.strongcrypto.com § VPN is combined with masquerading 122 for use behind firewalls 123

131 132

Tripwire Setting Up and Using Tripwire

§ www.tripwiresecurity.com § Perform immediate after a clean • Originated with COAST/CERIAS installation/upgrade • Free and commercial versions for Linux • Database security (read-only or cryptographic signatures as well) Administering Administering Linux in • “2.0 adds many enhancements” Linux in Production Production § Configure system, specifying files to check Environments Environments and ignore § What it does: • Features to handle log files and the like

Copyright © 1999-2001, • Documents a known system state Copyright © 1999-2001, Exponential Consulting, LLC Exponential Consulting, LLC § Set up and enable automatic periodic v Multiple cryptographic signatures for every item monitoring • Many algorithm choices § Look and and act upon the reports 124 • Compares current state to the stored state 125

133 134

Other Monitoring Tools UPS Configuration

§ saint: www.wwdsi.com/saint § Connect UPS device to serial port

§ nmap : www.insecure.com/nmap • Non-shared IRQ! • Cable with 10 Ohm resistor (from manufacturer)

Administering Administering § Run powerd (or other daemon) Linux in Linux in Production Production Environments Environments • Master: powerd /dev/ttyS0 –port n • Client: powerd –host dalton n § Add inittab entries for power-related events: Copyright © 1999-2001, Copyright © 1999-2001, Exponential Consulting, LLC Exponential Consulting, LLC • powerfail • powerfailnow 126 129 • powerok

135 136 Stronger File Permissions: POSIX Access Control Lists

§ More fine-detailed control of file access File Servers • Specifiable on a per-user/per-group basis • Default ACLsflow from the directory location Administering Administering Linux in Linux in Production Production • acl.bestbits.at Environments Environments § Limits total number & size § ACLs and NFS/Samba Copyright © 1999-2001, Copyright © 1999-2001, Exponential Consulting, LLC Exponential Consulting, LLC

130 131

137 138

Example Enabling ACLs

u::rwx § Entries are applied as follows: § Patch and build kernel g::rwx • User owner uses u:: • Set development option o:--- • Specified users use u:'s § Get/build utilities Administering u:chavez:rw- • Group members use all applying g:'s Administering Linux in Linux in Production (incl. g:: if applicable) Production Environments g:chem:r-x Environments § Patch and rebuild ext2fs utilities v Not accumulated! m:r-x • Everyone else uses o: § mount ... -o acl

Copyright © 1999-2001, • Mask (m:) sets maximum access level Copyright © 1999-2001, Exponential Consulting, LLC Exponential Consulting, LLC § Use setfacl and getfacl to set permissions except for o access § Do frequent backups 132 133

139 140

Adding ACL Capability Backing Up ACLs

* § ACLs: • getfacl -R --skip-base / > backup.acl setfacl --restore=backup.acl

Administering Administering Linux in Linux in Production Production • aget -sdR -e base64 / > backup.ea Environments * Environments * aset –B backup.ea * § Files with ACLs: Copyright © 1999-2001, Copyright © 1999-2001, Exponential Consulting, LLC Exponential Consulting, LLC • star H=exustar … > file.tar * • star -p -x < file.tar 134 134.1 • ftp.fokus.gmd.dc/pub/unix/star

141 142

Encryption General Setup Process

§ Steganography § Patch/configure/build kernel • Filesystem hidden in the low bits of each byte of an audio file § Get/build utilities • ban.joh.cam.ac.uk/~adm36/StegFS/ Administering Administering § Patch/rebuild standard FS tools Linux in Linux in Production § Encrypted filesystems: Production Environments Environments v Loop Device mechanism: EncryptionHOWTO.sourceforge.net/

v PPDD: linux01.gwdg.de/~alatham/ppdd.html

v TCFS: tcfs.dia.unisa.it 135 136

143 144

LPRng

§ Enhanced BSD lpd Print Servers § Better networking support • Smarter clients

Administering Administering Linux in Linux in Production Production § www.lprng.com Environments Environments

137 137.1

145 146

lpc Printcap Entries

hp: :lp=/dev/lp0 :cm=HP Laser Jet printer :lf=/var/log/lpd.log :af=/var/adm/pacct :filter=/usr/local/lib/filters/ifhp

Administering Administering :tc=.common Linux in Linux in Production Production Environments Environments laser: :oh=10.0.0.0/24 :lp=painters@matisse

.common: :sd=/var/spool/lpd/%P 137.2 137.3 :mx=0

147 148

LPRng Capabilities CUPS

§ Bounce queues § Common UNIX Printing System § Printer pools § www.cups.org § Local and/or remote filtering

Administering Administering Linux in § Program-generated printcap file Linux in Production Production § Network-based printing Environments § Access control: lpd.perms Environments REJECT SERVICE=X NOT REMOTEHOST=*.ahania.com § Separates job processing and device spooling REJECT SERVICE=X REMOTEHOST=dalton,hamlet § Compatible commands Copyright © 1999-2001, ACCEPT SERVICE=C SERVER REMOTEGROUP=printop Copyright © 1999-2001, Exponential Consulting, LLC LPC=topq,hold,release Exponential Consulting, LLC ACCEPT SERVICE=R,M,C REMOTEUSER=chavezPRINTER=test 137.4 REJECT SERVICE=* PRINTER=test 137.5

149 150

cupsd.conf Security

ServerName painters.ahania.com Server name. ServerAdmin [email protected] CUPS admin’s email address. § Access control ErrorLog / var/log/cups/error_log Log file locations. § Authentication AccessLog /var/log/cups/access_log PageLog / var/log/cups/page_log Printer accounting data. § Encryption LogLevel info Log detail (debug, warn, error). Administering MaxLogSize 1048571 Rotate log files when current > this. Administering Applies to class named checks. Linux in Linux in Production PreserveJobFiles No Don’t keep files after job completes. Production Encryption Always Always encrypt. Environments Environments RequestRoot / var/spool/cups Spool directory. AuthType Digest Require valid user account and password. User lp Server user and group owners. AuthClass Group Restrict to members of the finance group. Group sys AuthGroupName finance

Copyright © 1999-2001, TempDir / var/spool/cups/tmp CUPS temporary directory. Copyright © 1999-2001, Exponential Consulting, LLC Exponential Consulting, LLC Order Deny,Allow MaxClients 100 Max. client connections to server. Deny From All Deny all access... Timeout 300 Printing timeout period in seconds. Allow From 10.100.67.0/24 Except from this subnet. Browsing On Let clients browse for printers. 137.6 ImplicitClasses On Implicit classes are enabled. 137.7

151 152

Sharing Printers via Samba Printing to a Windows Printer

[laser4] printable = yes comment = LaserWriter on dalton § Standard remote system printcap entry: public = yes picasso:lp=:rm =vala:rp=picasso: postscript = yes printer name = laz4 printer driver = Microsoft driver name § Using smbprint: gaughin:sd=dir:lp=/dev/null:if=/usr/sbin/smbprint:af=file: [global] Administering Administering Linux in load printers = yes Linux in • Place a .config file in the specified spool directory Production printcap name = /etc/printcap.samba Production Environments printing = bsd | sysv | aix | hpux Environments containing password: [printers] server= zoas writeable = no service=matisse path = /tmp password="pwd" auto services = laz4 laz5 monet Copyright © 1999-2001, Copyright © 1999-2001, Exponential Consulting, LLC • Note that this makes all printers in the specified file available! Exponential Consulting, LLC

138 139

153 154

Font Management Summary

§ XFree86Config FontPath Enterprise User § fonts.dir and fonts.scale files • mkfontdir Authentication

Administering Administering Linux in • type1inst Linux in Production Production Environments § Font server: xfs Environments § Ghostscript Fontmap file

139.1 /OctavianMT-Roman /OctavianMT ; 140

155 156

User Authentication PAM

§ UNIX standard mechanisms § Pluggable Authentication Modules • Data files: passwd and group • Linux and Solaris 7 • Shadow password file § Per-application (service) configuration files in

Administering v MD5 Administering Linux in Linux in /etc/pam.d Production Production Environments Environments • Actual authentication performed by modules: § PAM shared libraries (.so files)

141 142

157 158

PAM Configuration Files Example: rlogin

#%PAM-1.0 § Entry format: type severity path args auth required /lib/security/pam _securetty .so auth sufficient /lib/security/pam _rhosts_auth.so • type: purpose auth required /lib/security/pam _pwdb.so shadow nullok v auth – user authentication auth required /lib/security/pam _nologin .so account required /lib/security/pam _pwdb.so v account – account attributes/controls account required /lib/security/ pam_time.so password required /lib/security/pam _cracklib.so Administering v session – pre/post service activities (logging to syslog) Administering password required /lib/security/pam _pwdb.so Linux in Linux in Production v password – causes password change if applicable Production nullok use_ authtok md5 shadow Environments Environments session required /lib/security/pam _pwdb.so • severity: how results affect outcome v required: failure => access denied

Copyright © 1999-2001, v requisite: immediate required Copyright © 1999-2001, Exponential Consulting, LLC Exponential Consulting, LLC v sufficient: success => access granted immediately Will this work?? v optional: result used only if nothing else is deterministic 143 • path args: path to module and arguments to it 144

159 160

Example: su PAM Modules

#%PAM -1.0 § pam_deny (account, auth, passwd, session) auth required /lib/security/ pam_rootok.so auth required /lib/security/ pam_wheel.so group=admins § pam_permit (account, auth, passwd, session) auth required /lib/security/ pam_pwdb .so shadow md5 nullok account required /lib/security/ pam_pwdb .so • Deny/allow all access by always returning failure/success password required /lib/security/ pam_pwdb .so (respectively). These modules do not log, so stack them Administering session required /lib/security/ pam_pwdb .so Administering Linux in Linux in with pam_warn to log the events. Production Production Environments Environments § pam_warn (account, auth, passwd, session) • Log information about the calling user and host to syslog.

Copyright © 1999-2001, Copyright © 1999-2001, § pam_access (account) Exponential Consulting, LLC Exponential Consulting, LLC • Specify system access based on user account and originating host/domain as in the widely-used logdaemon 145 146 facility. Its configuration file is /etc/security/access.conf.

161 162

More... And More...

§ pam_pwdb (account, auth, passwd, session) § pam_unix (account, auth, passwd, session) § pam_cracklib (passwd) • Two modules for verifying and changing user passwords. • Password triviality checking. Needs to be stacked with When used in the auth stack, the modules check the entered pam_pwdb or pam_unix. See the separate discussion user password. When used as an account module, they below.

Administering determine whether a password change is required or not (based Administering Linux in Linux in § pam_pwcheck (passwd) Production on password aging settings in the shadow password file); if so, Production Environments they delay access to the system until the password has been Environments • Another password checking module, checking that the changed. proposed password conforms to the settings specified in • When used as a password component, the modules update /etc/login.defs (discussed previously in this chapter).

Copyright © 1999-2001, the user password. In this context, the shadow (use the Copyright © 1999-2001, Exponential Consulting, LLC Exponential Consulting, LLC § pam_env (auth) shadow password file) and use_authtok options are useful; the latter forces the modules to set the new password to one • Set/unset environment variables with a PAM stack. It uses provided by a previous module in the stack and should the configuration file /etc/security/pam_env.conf. 147 accordingly be set when a password checking module is used. 148

163 164

Still More... More Again...

§ pam_issue (auth) § pam_limits (session) § pam_motd (session) • Sets user process resource limits (root is not affected), as specified in its configuration file, /etc/security/limits.conf. • Display an issue or message of the day file at login. The issue file is displayed before the username prompt, and the This file contains entries of the form: Administering Administering Linux in message of the day file is displayed at the end of a Linux in Production Production name hard/soft resource limit-value Environments successful login process. Environments § pam_krb4 (auth, passwd, session) where name is a user or group name or an asterisk • Interface to Kerberos user authentication. (indicating the default entry). The second field indicates Copyright © 1999-2001, Copyright © 1999-2001, Exponential Consulting, LLC § pam_lastlog (auth) Exponential Consulting, LLC whether it is a soft limit, which the user can increase if desired, or a hard limit (the upper bound which the user • Adds an entry to the /var/log/lastlog file which contains cannot exceed). The final two fields specify the resource in 149 data about each user login session. 150 question and the limit assigned to it.

165 166

More and More... Some More...

§ pam_listfile (auth) § pam_nologin (auth) • Deny/allow access based on a list of usernames in an • Prevents non-root logins if the file /etc/nologin exists, the external file: contents of which are displayed to the user. § pam_rhosts (auth) auth required pam _listfile.so onerr =fail sense=allow \ Administering Administering • Performs traditional /etc/rhosts and ~/.rhosts password-free Linux in file=/etc/ ftpusers item=user Linux in Production Production authentication for rsh and rlogin sessions between Environments Environments § pam_mail (auth, session) networked hosts (see chapter 8). • Displays a message indicating whether the user has mail. § pam_rootok (auth) The default mail file location (/var/spool/mail) can be • Allow root access without a password. Copyright © 1999-2001, Copyright © 1999-2001, Exponential Consulting, LLC changed with its dir argument. Exponential Consulting, LLC § pam_securetty (auth) § pam_mkhomedir (session) • Prevents root access unless the current terminal line is 151 • Creates the user’s home directory if it does not already 152 listed in the file /etc/securetty. exist, copying files from /etc/skel to the new directory.

167 168

...The Final Two More on PAM

§ pam_time (account) § The other service:

• Restrict access by time of day by user, group, tty auth required /lib/security/pam_warn.so and/or shell. auth required /lib/security/pam_deny.so …

Administering § pam_wheel (auth) Administering Linux in Linux in Production Production Environments • Designed for the su facility, this module prevents Environments § Module configuration files stored in root access to any user who is not a member of a /etc/security specified group (group=name option), which defaults to GID 0. You can reverse the logic of • Example: time.conf specifies hours when users Copyright © 1999-2001, Copyright © 1999-2001, Exponential Consulting, LLC the test to deny root access to members of a Exponential Consulting, LLC may access defined PAM services specific group by using the deny option along 153 with group. 154

169 170

PAM Futures SmartCards

§ More modules § Hardware token • Debugged modules • CryptoCard: www.cryptocard.com

Administering Administering Linux in § New syntax for severity: Linux in § Card plus reader Production Production Environments return-val=action [, return -val=action [,…]] Environments • MUSCLE: Mvmt. for use of smart cards in Linux env. • Example: success=ok,open-err=ignore,authtok_a v www.linuxnet.com/smartcard/tutorial.html • Schlumberger cards Copyright © 1999-2001, Copyright © 1999-2001, Exponential Consulting, LLC Exponential Consulting, LLC • Some with biometric authentication integrated into the reader 155 156 • Radius standard: PAM module

171 172

Networking and Authentication LDAP

§ Beyond the single system § Lightweight Directory Access Protocol • rdist, rsynch § Protocol for accessing general directory • NIS services (namespace)

Administering Administering Linux in • LDAP Linux in • DAP, X.500 Production Production Environments Environments • Global structure is defined but little used

173 174

Basics Example User Data

§ Directory not database dn: uid=chavez,o= SomeCo ,c=US § More object classes • Entries (objects) and attributes uid: chavez § More attributes cn: Rachel Chavez • First name objectClass: posixAccount • Last name Administering Administering objectClass: account Linux in § Distinguished name: Linux in • Email address Production Production Environments • cn=Aeleen Frisch,ou=People,dc=ahania,dc=com Environments gecos: Rachel Chavez • Kerberos data • uid=chavez,o= SomeCo,c=US userpassword: {crypt} xxxxxxxxxx • Phone number loginShell: /bin/tcsh § RDN • Picture Copyright © 1999-2001, Copyright © 1999-2001, uidNumber: 278 Exponential Consulting, LLC Exponential Consulting, LLC • Whatever gidNumber: 250 homeDirectory: /home/chavez 158.1 159 LDIF format

175 176

Deployment Process Linux Software

§ Design § DB manager: GNU gdbm (www.fsf.org) or BerkeleyDB (www.sleepycat .com). § LDAP server: § Transport Layer Security (TLS/SSL) libraries • Open LDAP slapd (LDAP 3) (www.openssl.org). § The Cyrus SASL libraries (asg.web.cmu.edu/sasl). Administering • Netscape Directory Server (LDAP 3) Administering Linux in Linux in Production v Replication servers Production # rpm -qa| egrep-i '(db|sasl|ssl)' Environments Environments § Configure db-3.1.17-13 gdbm-1.8.0-225 • /etc/openldap/{slapd,ldap}.conf sdb-2001.1.18-0 Copyright © 1999-2001, Copyright © 1999-2001, Exponential Consulting, LLC§ Migrate existing data into LDAP databases Exponential Consulting, LLC sdb_en-2001.1.18-0 cyrus-sasl-1.5.24-5 openssl-0.9.6-21 160 160.1 openssl-devel-0.9.6-21

177 178

slapd.conf Starting the Server

# /etc/ openldap /slapd .conf § /etc/init.d/ldap start include /etc/ openldap/schema/core.schema pidfile / var/run/ slapd .pid argsfile / var/run/ slapd .args § Enabling configuration:

Administering Administering • E.g.: SuSE /etc/rc.config: Linux in database ldbm Linux in Production Production Environments suffix "dc=ahania, dc=com" Environments START_LDAP="yes“ rootdn " cn =Manager, dc= ahania , dc=com" # encode with slappasswd - h '{MD5}' -s secret -v -u rootpw {MD5}Xr4ilOzQ4PCOq3aQ0qbuaQ== Copyright © 1999-2001, Copyright © 1999-2001, Exponential Consulting, LLC directory /var/lib/ ldap Exponential Consulting, LLC § Populate database/migrate data • PADL Perl scripts 160.2 160.3

179 180

Clients Client Configuration File

§ gq, web2ldap, kldap, … § /etc/ldap.conf or /etc/openldap/ldap.conf § Netscape # /etc/openldap/ ldap .conf URI ldap:// bella. ahania.com Administering Administering Linux in Linux in BASE dc= ahania ,dc=com Production Production Environments Environments

161 161.1

181 182

System Level Clients Schema for Authentication

§ LDAP-aware applications: § /etc/openldap/schema/*.schema • sendmail and Postfix objectClass Schema file • Apache top core

Administering Administering person core Linux in Linux in Production Production Environments § Using for authentication: Environments organizationalPerson core • pam_ldap inetOrgPerson core account cosine Copyright © 1999-2001, • nss_ldap Copyright © 1999-2001, Exponential Consulting, LLC Exponential Consulting, LLC posixAccount nis shadowAccount nis

161.2 161.3 inetLocalMailRecipient inet*mailrecipient

183 184

nss_ldap pam_ldap

§ ldap.conf § rlogin nss _base_ passwd ou=People,dc= ahania ,dc=com nss _base_shadow ou=People,dc= ahania ,dc=com #%PAM-1.0 auth required /lib/security/pam_securetty.so nss _base_group ou=Group,dc=ahania,dc=com auth required /lib/security/pam_nologin.so Administering Administering Linux in Linux in auth sufficient /lib/security/pam_rhosts_auth.so Production Production auth sufficient /lib/security/pam_ldap.so Environments Environments § /etc/nsswitch.conf auth required /lib/security/pam_unix.so auth required /lib/security/pam_mail.so passwd: files ldap account sufficient /lib/security/pam_ldap.so shadow: files ldap Copyright © 1999-2001, Copyright © 1999-2001, account required /lib/security/pam_unix.so Exponential Consulting, LLC Exponential Consulting, LLC password sufficient /lib/security/pam_ldap.so password required /lib/security/pam_unix.so strict=false session required /lib/security/pam_unix.so debug 161.4 § getent passwd 161.5

185 186

Limiting Access by Host and/or Group Security: Access Control

§ ldap.conf # Allow user access to hosts # Only this group can access thi s host § slapd.conf pam _check_host_ attr yes pam _groupdn cn=dalton.ahania .com,... pam _member_attribute uniquemember # simple access control: read-only except passwords access to dn=".*,dc=ahania,dc=com" attr=userPassword by self write § Directory entries: Administering Administering by dn=root,ou=People,dc=ahania,dc=com write Linux in # List of allowed hosts # List of allowed users on the lo cal host Linux in Production Production by * auth Environments dn: uid =aefrisch,ou =People,... dn: cn=dalton .ahania .com,... Environments objectClass : account objectClass : ipHost objectClass : posixAccount objectClass : device access to dn=".*,dc=ahania,dc=com" ... objectClass : groupOfUniqueNames by self write

Copyright © 1999-2001, host: milton.ahania .com cn: dalton Copyright © 1999-2001, by * read Exponential Consulting, LLChost: shelley .ahania.com cn: dalton.ahania.com Exponential Consulting, LLC host: yeats .ahania.com uniqueMember: uid=chavez ,ou=People,dc=... uniqueMember: uid=carter,ou=People,dc=... 161.6 161.7

187 188

Security: TLS389 and SSL636 Security: Other

§ Key generation: § Kerberos cd /usr/ ssl/cert openssl req - newkey rsa:1024 -keyout slapd _key. pem \ § SASL -x509 - days 365 - out slapd_cert.pem openssl rsa -in slapd _key. pem - out slapd _key. pem Administering Administering Linux in chown ldap.ldap *.pem (if appropriate) Linux in Production Production Environments chmod 600 sl*. pem Environments § Statup: slapd -h "ldap:/// ldaps:///"

Copyright © 1999-2001, Copyright © 1999-2001, Exponential Consulting, LLC § slapd.conf Exponential Consulting, LLC TLSCipherSuite HIGH:MEDIUM:+SSLv2 TLSCertificateFile /usr/ssl/certs/slapd_cert.pem 161.8 TLSCertificateKeyFile /usr/ssl/certs/slapd_key.pem 161.9

189 190 Helpful Installation and Configuration Tools

§ KickStart (RedHat) Compute Servers • Automated installation/reinstallation • Ghost, Drive Image

Administering Administering Linux in Linux in § cfengine (www.iu.hioslo.no/cfengine) Production Production Environments Environments • System monitors and corrects itself by comparing to a stored “healthy” state

162 171’

191 192

(Some) cfengine Capabilities Pieces (1.6)

§ Check and configure the network interface. § cfengine § Edit text files for the system and for all users. § cfd § Make and maintain symbolic links, including multiple links from a single command. § cfrun

Administering Administering Linux in § Check and set the permissions and ownership of files. Linux in § cfengine.conf Production Production Environments § Delete junk files which clutter the system. Environments § cfd.conf § Systematic, automated mounting of NFS filesystems . § Checking for the presence of important files and filesystems .

193 194

A Simple cfengine.conf Actions

control: § mountables, mountinfo, mountall, umount, actionsequence= ( tidy links ) addmounts access = (chavez root) § directories, files, links, required § copy, tidy, editfiles, disable Administering links: Administering Linux in Linux in Production /bin -> /usr/bin Production § shellcommands, processes Environments Environments § netconfig, broadcast, resolve, defaultroute tidy: § checktimezone /tmp pattern=* age=7 recurse=inf Copyright © 1999-2001, Copyright © 1999-2001, § mailcheck Exponential Consulting, LLC Exponential Consulting, LLC § module

171.3 171.4

195 196

Examples 1 Examples 2

files: copy: /etc/security mode=600 owner=root group=sys recurse =inf action=fixall $( masteretc)/hosts.denydest=/etc/hosts.deny o=root mode=0644 /usr/bin checksum=md5 action=warnall $( masteretc)/ntp.drift dest =/etc/ntp.drift mode=644 $( masteretc)/shells dest=/etc/shells mode=644 /home recurse=inf include=*. txt action=compress Administering Administering Linux in Linux in /private acl=secure1 action= fixall Production Production Environments linux:: Environments $( masteretc)/rc .config dest=/etc/ rc .config o=root mode=644 acl: { secure1 method:overwrite dbdevhost.Hr03:: Copyright © 1999-2001, Copyright © 1999-2001, fstype:nt Exponential Consulting, LLC /devel dest=/backup/devel server=dalton r=1 backup=false Exponential Consulting, LLC user:chavez:rwx:allowed user:mark:all:allowed user:toreo:read:allowed group:dummy:all:denied 171.5 171.6 }

197 198

Classes Class Examples

§ OS ultrix, sun4, sun3, hpux, hpux10, aix, solaris, osf, irix4, irix, irix64 § myclass.solaris.Monday.Hr01:: freebsd, netbsd, openbsd, bsd4_3, newsos, solarisx86, aos, nextstep , bsdos, linux, debian , cray , unix_sv, GnU, NT § sun4|ultrix|osf:: § host § host group name § myhosts.aix.!vader:: Administering Administering Linux in § day of the week Linux in Production Production § December.Day31.Friday:: Environments § hour (Hr02) Environments § minute (Min33) § any:: § 5 minute interval (Min00_05)

199 200

Distributing the Resources PBS

§ Batch processing § Portable Batch Scheduler • Multiple queues across many servers • Designed for maximal resource usage • Priorities • www.pbspro.com Administering • Resource limits Administering Linux in Linux in § Intelligence and control reside in the Production Production Environments • Access control Environments scheduler, not in the queue attributes • Authentication § Site-definable scheduling via configuration

Copyright © 1999-2001, • Logging and accounting Copyright © 1999-2001, and/or hooks for routines Exponential Consulting, LLC Exponential Consulting, LLC § Separate pre-staging vs. running vs. cleanup 162.1 § Load balancing 162.2

201 202

PBS Implementation Others

§ Master server: pbs_server § Maui Scheduler § Per system: pbs_mom • www.mhpcc.edu/maui § IBM Load Leveler

Administering Administering Linux in Linux in Production Production Environments Environments

162.3 162.4

203 204

Parallel Environments Parallelization Strategies

§ PVM, MPI § Master/worker paradigm is dominant § BSP • Same code for both § DSM schemes § Deciding where to parallelize is the most Administering Administering Linux in Linux in important question Production • Linda Production Environments Environments • Discrete task -based strategies v When free, worker gets the next bit of work to do • Domain decomposition Copyright © 1999-2001, Copyright © 1999-2001, Exponential Consulting, LLC Exponential Consulting, LLC v Worker is preassigned a subset of the entire operation

163 103

205 206

Computing π Numerically Message Passing Interface (MPI)

§ Processes explicitly send data to one another get n as input • MPI_Send and MPI_Receive h = 1.0/double(n); sum=0.0; § www.erc.msstate.edu/labs/hpcl/projects/mpi for (i=1; i<=n; i++) { sum += 4.0 / (1.0 + (h*(i- 0.5))**2); }

Administering mypi = h*sum; Administering Linux in Linux in Production print the result Production Environments Environments

164 165

207 208

MPI Example BSP

§ main routine: § Bulk Synchronous Parallel Model MPI_Init(&argc,&argv ); MPI_Comm_size(MPI_COMM_WORLD, &numprocs); § www.bsp-worldwide.org MPI_Comm_rank(MPI_COMM_WORLD, &myid); if (myid==0) { § Consists of a small number of operations Administering input parameter n } Administering Linux in Linux in designed to distribute data during a parallel Production MPI_Bcast(&n, 1, MPI_INT, 0, …); Production Environments Environments h = 1.0/double(n); sum=0.0; calculation for (i=myid +1; i<=n; i+=numprocs) { sum += 4.0 / (1.0 + (h*(i- 0.5))**2); } • Structured for asynchronous communication mypi = h*sum; Copyright © 1999-2001, Copyright © 1999-2001, Exponential Consulting, LLC MPI_Reduce(&mypi, &pi, Exponential Consulting, LLC • Read/write data from remote process’ memory MPI_DOUBLE,MPI_SUM,0, …); without its participation if (myid==0) { print the result } • Send data to a remote process’ queue 166 MPI_Finalize(); 167

209 210

Simulating Shared Memory Master

• master function: § Linda get n out("pi-int", n); • www.lindaspaces.com out("pi-task", n); sum=0.0 § Provides a virtual shared memory (VSM) worker(); Administering Administering for (j=0; j<=n; j++) Linux in • High level operations that can be ignorant of Linux in Production Production in("pi-sum", j, ?sum_j); Environments the specifics of communication Environments sum += sum_j; } print sum return; Copyright © 1999-2001, Copyright © 1999-2001, Exponential Consulting, LLC Exponential Consulting, LLC • For faster performance, change j to ?k 168 169 v Why??

211 212

Worker OpenMP

§ www.openmp.org • worker function: while (1) { § Standard set of compiler directives for rd("pi-int", ?n); in("pi-task", ?i); shared-memory parallelism if (i==0) { break; } Administering Administering Linux in out("pi-task", i-1); Linux in • Serial code is unmodified Production Production Environments h = 1.0 / double(n); Environments sum = 4.0 / (1.0 + (h*(i-0.5))**2); mypi = h * sum; out = ("pi-sum", n, mypi); } Copyright © 1999-2001, Copyright © 1999-2001, Exponential Consulting, LLC return; Exponential Consulting, LLC

170 107

213 214

OpenMP Example

!$ omp parallel do do 50 k=1, ntop arr(k)=a(k)+b(k-1)*c(k)/(k+1) Databases 50 continue !$omp end parallel do

Administering Administering Linux in Linux in Production sum=0.0 Production Environments Environments !$omp parallel default(shared) !$omp & reduction(+:sum) lots of computations Copyright © 1999-2001, $!omp end parallel Copyright © 1999-2001, Exponential Consulting, LLC Exponential Consulting, LLC

108 172

215 216

Databases mySQL

§ mySQL § Good for medium size applications § PostgresSQL • If you need to save money … § Limitations include: § Commercial servers: Administering Administering • Limited SQL Linux in Oracle, Sybase, DB2, … Linux in Production Production v No subqueries: Environments Environments select sum(total) from pmts where trx_id in (select trx_id from trx_hdr,org where country>1 and paid=0 and ship>’06/01/1999’ and join)

173 174 § www.mysql.com

217 218

mySQL Tools Database Programming Interfaces

§ xmysql: web.wt.net/~dblhack § Perl § xmysqladmin • MySQL • www.tcx.se/Contrib/ • DBI and DBD drivers

Administering Administering v DBD::ODBC Linux in § kmysql: www.xnot.com/kmysql Linux in Production Production Environments Environments • Win32:ODBC § More in Tck/Tk, Python, …

175 176

219 220

Office Applications

§ Applixware Office PC § Star Office • Microsoft response

Administering Administering Linux in Linux in Production Production Environments Environments § Others • WordPerfect

177 178

221 222

Word Processing Word Processing

• Convert 1 page test file: • Import 10 page sample file: • Import 187 page manual chapter: • Paragraph styles: • Character styles: Administering • Superscripts: Administering Linux in Linux in Production Production Environments • Footnotes: Environments • Tables: • Sectioning: • Dynamic headers/footers: Copyright © 1999-2001, Copyright © 1999-2001, Exponential Consulting, LLC • TOC/Index: Exponential Consulting, LLC • Equation editor equations: 179 • Manually-constructed equations: 179.1 • Pictures:

223 224

Spreadsheets Spreadsheets

• Convert spreadsheet files: • Convert multisheet workbooks: • Formulas:

Administering • Text color: Administering Linux in Linux in Production Production Environments • Text formatting: Environments • Graphs (visible): • Graphs (correct): Copyright © 1999-2001, Copyright © 1999-2001, Exponential Consulting, LLC Exponential Consulting, LLC

180 180.1

225 226

Presentations Presentations

• Convert current PPT files: • Convert prev. rev PPT files: • MS template:

Administering • Custom template: Administering Linux in Linux in Production Production Environments • Notes pages: Environments • Drawn graphics: • Imported graphics: Copyright © 1999-2001, Copyright © 1999-2001, Exponential Consulting, LLC Exponential Consulting, LLC

181 181.1

227 228

Required Applications: Everyday Required Applications: Specialty

§ File manager § Outlook § Personal/Baby database • Email • Schedule § Image editing • Contacts § Web browser § Vector-based illustration Administering Administering Linux in § Winzip Linux in Production Production § HTML editor Environments § Music (CD, MP3) Environments § Video § 3D drawing § Burn CD § Program development Copyright © 1999-2001, § PDF viewer Copyright © 1999-2001, Exponential Consulting, LLC § PDA interface Exponential Consulting, LLC § Games § Expenses 182’ 182.1 § Fax

229 230 Required Applications: Administrative Select the Right Distribution

§ Database with development environment § Lots of ported, pre-compiled packages § Business finances • SuSE § Web server • Recent RedHat Administering Administering Linux in Linux in • ? Production § FTP server Production Environments Environments § File sharing § Printer sharing Copyright © 1999-2001, Copyright © 1999-2001, Exponential Consulting, LLC Exponential Consulting, LLC § Remote installation facility

182.2 § Scripting 182.3

231

Into the Future

§ “These are Linux’s early days …” • The beginning of the end? • The end of the beginning?

Administering Linux in Production Environments

183