Introduction to Welcome

SANOG IV Workshop 2004 Introduction ¡ ¡ Instructors: Helpers: Hervey Allen Kishor Panth Joe Abley July 2004 Philip Hazel

Presented by Hervey Allen Network Startup Resource Center At what level are we?

How the class runs

What we'll do today 1 2

Schedule How the workshop runs

Morning Evening sessions With your participation!

¡ Optional. Around 08:45-10:45 Class Please ask questions. If you don't understand

¡ 19:30-21:00. 10:45-11:00 Break something due to language, ask us to clarify. ¡ 11:00-13:00 Class Labs We are sharing workstations. This is good. You'll Lunch Open after dinner learn more working in groups. ¡ 13:00-14:00 (19:30) until around At the end of the week we'll have a written test. Afternoon 21:00. If there is class you

¡ 14:00-16:00 Class can use machines not in A Certificate of Attendance will be given as well.

¡ 16:00-16:15 Break/Tea use by those attending

¡ 16:15-18:15 Class class. 3 4 How the workshop runs – practical Outline Part 1

What distribution? We have a server in use for the workshop. IP and Fedora Core 2 installation and Kickstart Notes server name will be given in class. System commands (cp, ls, cd, rm, etc.). Basic editor usage with vi.

The PCs have Internet access, but during class Create and remove user accounts.

we'll often work outside of XWindows. Discuss /etc/passwd, /etc/group, /etc/shadow.

Commands, programs, shells and paths.

The root password on your machine is Use of 'su' command for 'root', and /etc/sudoers.

“SANOG2K4”. Download RPM packages using FTP and install them. We have several people assisting during the week Directory structures – what's where. Learn how to shutdown and restart the server. Initialization levels.

to help with language issues and during Discuss /etc/ and /etc/rc.d/init.d/ (services).

exercises. Practice what we've learned if there is time. 5 6

What distribution? Fedora Core 2 and Kernel 2.6.x

Red Hat, Fedora, SuSE, Debian, Conectiva, Turbolinux, Fedora Core 2 uses Kernel version 2.6.5-1.358 Mandrake, United Linux, Gentoo, Slackware, other? etc? (see /proc/version). There are some changes (refer, also, to /usr/src/linux/Documentation/Changes): Or, do we use FreeBSD, Solaris, OpenBSD, NetBSD, HP/UX, AIX, Mac OS X, SCO, etc? ¡ mount has more options and supports more filesystems.

What do you run? ¡ Modules work differently (/etc/modprobe.conf)

Our response: ¡ Kernel building is a bit different. ¡ Currently Red Hat 9 appeared almost to be a standard ¡ Larger disk (TB) and memory (64GB) support. ¡ If you know Red Hat Linux using commands and text file, then you know enough to use other distributions. ¡ Pcmcia support partially built in to kernel. ¡ ¡ Lots of local as others are likely running RH/Fedora. Updates to NFS support and Quota support. ¡ We want to teach how to scale services and “good ¡ More... 7 network practices” using a reasonable system. 8 Installing Fedora Core 2 Basic Commands

How can you install? Upgrade? cp, cd*, ls, mv, rm y man

¡ (*built in command shell commands). ¡ Using CD/DVD. Where are commands located? ¡ Hard drive partition /bin, /usr/bin, /usr/local/bin, /sbin, /usr/sbin ¡ Floppy/CD/DVD and an ftp/tftp/http/nfs server. ¡ The difference between “sbin”, “bin” and “ /usr”

¡ Floppy/CD/DVD and Kickstart file. If you know DOS: ¡ ¡ NIC PXE boot. cp = copy ¡ cd/chdir = cd/chdir

Install options using the Kickstart graphicl tool. ¡ ls = dir ¡ We might install at the end of the workshop. mv = move (before it was copy and delete/erase ¡ rm = del[ete] and/or erase 9 10

Basic commands continued Some more commands... ¡ ¡ ¡ ¡ ¡ bg groupadd* man tail apropos ¡ ¡ ¡ ¡ Not a command, but we'll practice starting a ¡ bzip groupdel* mount* tar “ctrl-u” ¡ ¡ ¡ ¡ ¡ cat gzip more telinit* date* ¡ ¡ ¡ ¡ detached process. ¡ chgroup* history netstat top hexdump ¡ ¡ ¡ ¡ ¡ chmod ifconfig* nmap touch hwclock* ¡ ¡

To do this you use the “&” symbol the ¡ ¡ after ¡ clear info ping traceroute mkisofs ¡ ¡ ¡ ¡ command you wish to run that opens a separate chown* init* printenv uname ¡ tcpdump ¡ ¡ ¡ window. ¡ exec insmod* ps unset ¡ ¡ ¡ ¡ df kill pwd unzip ¡ ¡ ¡ For example, to open another terminal from within ¡ dmesg ln route* usermod* ¡ ¡ ¡ ¡ du locate rpm* users ¡ ¡ ¡

your terminal under XWindows type: ¡ export lsmod rmdir watch ¡ ¡ ¡ /usr/bin/gnome-terminal & ¡ find lsof rmmod* whereis

¡ ¡ * = root ¡ ¡ gcc mkdir set which ¡ ¡ ¡ ¡ grep “|” (pipe) su whoami 11 12 Looking for more information The vi editor

Not only can you use commands to find information about Why use vi? Why not emacs, xemacs, joe, pico, your system, but you can look inside several files, and you can explore the /proc directory, as well. etc.? (Make note of “pico -w”)

Example of files with useful information: vi exists in almost all distributions of

/etc/motd Linux/Unix/BSD. /etc/issue

/etc/resolve.conf If you have to work on a new machine, then vi /etc/services will almost always be available to you.

/etc/X11/XF86Config In reality, you are likely to use a different editor /etc/modules.conf and with Kernel 2.6 /etc/modprobe.conf for more complex editing, but let's see what we /etc/fstab can do with vi --> 13 14

Basic vi commands Create and eliminate user accounts

Impress your friends... /etc/passwd, /etc/group, /etc/shadow, /etc/sudoers /usr/sbin/adduser --> /usr/sbin/useradd vi fn, vi -r fn, vi + fn, vi +n fn, vi +/pat fn

Open: /usr/sbin/userdel

Close: :w, :wq, :q, :q! /etc/default/useradd

Movement: h,j,k,l y w, W, b, B, :n (+arrow /etc/skel keys) /etc/login.defs

Edit: i, o, x, D, dd, yy, p /etc/profile

chsh, passwd, groupadd*, groupdel*, groupmod*, Search: /pattern, ?pattern, n, N usermod* Note: vi is alias to vim, and view is link to /usr/bin/vim under Fedora 15 16 Note: * requires root/admin access /etc/passwd /etc/shadow

The /etc/password file has the following format: This file is used to hide encoded user passwords.

hervey:x:500:500:Hervey Allen:/home/hervey:/bin/bash Only root can (or should) read this file. i.e.: /etc/shadow has the following format:

userid:pw:UID:GID:name:directory:shell hervey:$1$w!@F62adfk3$LCYjTI3udsd/tGP1pux1:12452:0:99999:7::: Using /etc/shadow the “pw” is represented by an i.e.:

“x”. If the user entry is actually something like a userid:

password encoded with crypt:

service, then the “shell” is represented with Days after Jan. 1, 1970 on which the password was last “/sbin/nologin”. changed: Days until the user can change the password:

Number of days before a password expires:

17 18

/etc/shadow continued /etc/group

In this file group names are specified (no more hervey:$1$w!@F62adfk3$LCYjTI3udsd/tGP1pux1:12452:0:99999:7::: than 8 characters), password, the Group

Days before a password expires that a user is notified: IDentifying number (GID), and a list of group

Days after a password expires that an account is members separated by commas: deactivated: Days since Jan. 1, 1970 that a user has been deactivated: For example: Reserved: mail:x:12:mail,postfix The “x” - If there is a group password, then it is stored in encoded format in /etc/shadow. What are groups used for? 19 20 Commands - programs – shell – path Using the su command

What's a “command” and a “program”? The “su” command is used to become a different Why can't you always run all commands and userid, like root, without having to logout and programs on a system? log back in. How do you “fix” this? To use “su” to become root your userid has to be How do you see how things are configured for a given permission to do this in “/etc/sudoers”. user? You can allow users to run specific privileged ¡ /etc/skel commands using “/etc/sudoers” and “sudo”. ¡ /etc/profile

¡ /home/user/.bashrc You can assign users to the “wheel” group and

¡ /home/user/.bash_profile using “/etc/sudoers” you can allow them to run

¡ set, printenv, export all or some commands. 21 22

More uses for the su command Package installation using RPM

Instead of having to open a root shell, you can run RPM stands for “Red Hat Package Manager”

a privileged command like this: There are several systems to control and install sudo command software (rpm, dpackage/apt-get, yum, source). For example: Each one has its advantages and disadvantages.

sudo less /etc/shadow RPM allows you to install and remove software And, if you wish to open a different user shell and on your server. You can, also, see the files run their login scripts do: associated with a package and where they install. su – userid RPM, however, cannot resolve software What happens if you don't use the “-”? dependencies between software packages. 23 24 Using RPM Usando RPM continued

You can do quite a few things with RPM, but People use RPM differently. See the difference typically you install like this: between “rpm -Uvh” and “rpm -ivh”. Is it rpm -Uvh package-name.rpm really necessary to use the “-h” option? rpm -ivh package-name.rpm The key for anything that trys to organize your Found out if something is already installed: software is does it really function? RPM helps, rpm -qa | grep package-name but it does not solve the problems of knowing what you have installed, where, and what belongs Get more information about an already installed to what software package. RPM: rpm -qi package-name If you don't believe me see how many times you end up using “rpm –nodeps” in the future... ;-) rpm -ql package-name 25 26

Shutdown and restarting a server Run levels

How do you shutdown a Linux box? And, what was “init/telinit 0”?

¡ shutdown -t 60 message Linux works with the concept of run levels. Each level has a meaning. These are: ¡ halt 0 – Shutdown. Never make this “default” ¡ init/telinit 0 1 – Single user mode And, to restart? 2 – Multi-user mode, but without network file system (NFS).

¡ reboot (linked to /sbin/halt) 3 – Multi-user mode with NFS.

¡ shutdown -r now 4 – Not used 5 – Multi-user mode with X11 (Gnome, KDE, XWindows, etc.). 6 – restart. Never make this “default” 27 28 Run levels continued Run levels continued

The level your machine boots in to is defined in To shut down XWindows (Gnome, KDE) and /etc/inittab. work only in a shell (text), you can open a For a server you will almost always want to start in terminal in XWindows and type: run level 3, with the occasional exception going init 3 to “single user” mode (level 1). To restart XWindows type: The basic rule of thumb; “Server, level 3, computer init 5 for personal use, level 5”. Now we are going to see how you can control You can change run levels using “telinit --> init”, what runs in each run level... that is “telinit” is an alias for “init”. 29 30

Services that run in each run level Controlling Services

This is a bit complicated, but the bits and pieces First look at /etc/rc.d/init.d. Here we have the we use with Fedora (System V) are: scripts that control most running services (note

¡ /etc/rc.d/ xinetd!).

¡ /etc/rc.d/rc0.d to rc6.d Let's stop and start “nfslock”.

¡ /etc/rc.d/init.d/ Let's see in which run levels “nfslock” runs:

¡ /etc/inittab /sbin/chkconfig –list | grep nfslock

¡ /sbin/service script option Now we are going to use “chkconfig” to remove

¡ chkconfig “nfslock” as a running service in all run levels.

31 32 Controlling services continued Services summary

First let's go to /etc/rc.d/init.d/rc3.d and rc2.d to Perhaps you've seen /etc/rc.d/rc, rc.sysinit, or explain what happens in these directories. rc.local on other systems. These files, also, control how services run, but in a different After this, let's configure things so that “nfslock” manner. does not run the next time we restart our server The directory structure we have seen is called (in /etc/rc.d/init.d): “System V” and comes from operating system chkconfig –del nfslock structures before Linux was created. Now let's play with “chkconfig”, the rcn.d Understanding how to start and stop services directories, and services to understand what's temporarily and permanently is key to being able happening. to properly run and maintain a Linux server.

33 34

Summary part one Part two

We have seen a bit about how to run a Linux server. Tomorrow we'll discuss much more about the structure of a Linux server. Now, if there's time, we'll return to the command list near the start of this presentation and practice Introduction to Linux Continued using these commands, or read about them using the command “man”.

35 36 Overview* Linux filesystems ¡ Discussion about Linux partition schemes and options - /etc/fstab, dev. ¡

Permissions in Linux. Review commands “chmod” and “chown”. ¡ “ifconfig” to configure network cards and interfaces. ext2, ext3, reiserfs, jfs, hpfs, etc... ¡ Discuss Linux services and how to know what's running. ¡ The current “winner” , ext3 (journaling) ¡ The /proc directory.

¡ Network configuration changes using /etc/sysconfig/network-scripts Structure of partitions: ¡ We'll present /etc/crontab and practice using cron commands. ¡ ¡ The Linux kernel and how to recompile it. / (“root”) ¡ /etc/modules.conf and /etc/modprobe.conf. ¡ /usr ¡ We mention firewalls. ¡

¡ /var Installing compiled software (vs. RPM). ¡ Gnome vs. KDE and XWindows. What are they. Why are they not ¡ /tmp

necessary for a server? ¡

¡ /home Logs and where they exist. We'll inspect logs, noting /etc/syslog.conf. ¡ swap * This is an example of how not to use a slide in a presentation...;-) 37 38

/ (“root”) /usr

The root partition is where critical system files live, Is used for system software like user tools, including the programs necessary to boot the compilers, XWindows, etc. system in to “single user” mode (run level 1). If one has to expand* this partition for additional The idea is that this part of the system does not software, then having it separate makes this grow or change, but rather stays isolated from the possible. rest of the operating system. The default Fedora install looks like this:

¡ /boot (approx. 100Mb)

¡ swap (1 to 2 x installed RAM) *We'll discuss this. We don't always install Linux with a separate /usr partition. ¡ 39 / (the rest of the hard drive) 40 /var /tmp

This is where files and directories that change a lot This is where users and application can save are kept. For example, webserver logs, email temporary files. By default quotas are not directories, print spools, etc. enabled in Linux, thus it's possible for a user, or On a server it is a good idea to have /var in a program, to fill the /tmp partition on purpose or separate partition. If someone, by accident or on by accident. purpose, creates a large amount of email, web Sometimes /tmp resides in /var/usr/tmp, but many activity, etc. then it's possible to fill this Linux applications expect to have access to /tmp. partition. If /var is not a separate partition this Another convention in the Unix world is to use could cause your server to crash or hang. /scratch, or /var/tmp instead of /tmp.

41 42

/home Swap

This is where user directories are kept. Often user's Swap is where virtual memory lives. Swap is it's email is not kept in this partition. own filesystem. If you don't use quotas (common in Linux) then You can run without swap, and your PC may run faster, but this is dangerous if you run out of users can easily fill this partition. memory. An example: a user writes a program that produces There are several opinions about what is the a text file. On accident they create an infinite optimal swap size. This can depend on what type loop that writes text to a file and they don't of services you run (databases need more swap). notice this. This could fill your /home partition The general rule of thumb is that swap size rapidly. should be somewhere between your RAM and twice your server's RAM. 43 44 Mounting filesystems File permissions

If you want to mount a filesystem not listed in There are five categories and three types of permissions /etc/fstab then you need to use the mount that you need to consider. The default file permissions are set with the umask command. command.

First, you need to know what entry in the /dev There are two categories of permissions that relate to the directory describes the device you wish to mount user or group that is going to run a file (setuid, setgid). (a cd, floppy, another hard drive, etc.). In addition you can set a “sticky” bit on directories so that only root and owner can make changes.

You, also, need to know what type of filesystem. The permissions available are “r” (read), “w” (write), and

“x” (execute). For example, mounting a dos formatted floppy: You can assign permissions to world (a), group (g), and ¡ mount -t msdos /dev/fd0 /mnt/floppy user (u). 45 46

File permissions continued Ifconfig

A file belongs to a user. You can assign a file to During the week you are going to use this another user or another group using the chown command a lot. (“CHange OWNer”) command.

ifconfig is used to view the status of and to You can change permissions and/or owners for a group of files or for all files and all files in configure ethernet interfaces. subdirectories using the chmod and chown For example: commands. ¡ You can change directory permissions using the /sbin/ifconfig chmod command. ¡ /sbin/ifconfig eth0 192.188.58.66

netmask 255.255.255.224 As root you can modify file write access using

47 the chattr command. 48 /etc/hosts Running services and ports

In this file you should have, at least, this line: To view all services (UNIX us “-Af”):

127.0.0.1localhost.localdomain localhost ¡ ps -aux | more

For a private network you can use this file instead What tcp ports are they using?

of using a DNS server. Linux looks in /etc/hosts ¡ sudo /usr/sbin/lsof -i

before asking DNS to resolve an IP address. ¡ /bin/netstat -natu

You can change this order in /etc/nsswitch.conf. What starts at boot time? See in /etc/rc.d/init.d

We'll add an entry in /etc/hosts for “noc”. ¡ /sbin/chkconfig –list | more

¡ Don't forget xinet.d and inet.d

49 50

The special directory /proc The /proc directory continued

The /proc directory represents the state of your Some of the files in /proc: machine. It is an abstraction of your kernel and /proc/meminfo Memory usage status. does not exist as a filesystem on your hard drive. /proc/version Kernel version. For example /proc/cpuinfo has information about the cpu (or cpu's) of your machine. You can read /proc/net/dev Information about each network it like this: interface.

¡ cat /proc/cpuinfo /proc/interrupts IRQ usage in your system. But, you need to be root. The “/proc” directory is a /proc/kcore RAM contents (be careful!). security hole.

51 52 More /proc Configuration changes

If you think about it, there are several commands If you want to make sure that your network that simply show what's in /proc. interface configuration remains the same upon rebooting your machine you need to edit the For example, the command is basically the lsmod file /etc/sysconfig/network-scripts/ifcg-eth0 (if same as: it's for eth0). cat /proc/modules If the “default route” cannot be calculated using And, the command ps uses information you can the IP address and the netmask, then you need to see by looking in /proc/processid. use the command:

¡ route add default gw nnn.nnn.nnn.nnn in order to access networks outside your local net. 53 54

Crontab Crontab continued

The “cron” service allows you to automatically run A cron file that shows how a service is going to programs when you want. Fedora uses “anacron” as well run has the following format: (better on personal PCs). Minute Hour Day Month Weekday Command

This is configured in /etc/cron.allow, and /etc/cron.deny, An example: if they exist. 1 4 1 4 * /bin/mail [email protected] < /home/user/joke

Use the command crontab in order to change the files Send an email on the first of April. that control how the crond daemon works.

The process “anacron” is used in Fedora. Anacron Or edit system-wide files in uses /etc/anacrontab and is used for machines /etc/cron.hourly/daily/weekly/monthly. that are not always running (i.e. desktops vs. The cron files have a very specific format. servers). 55 56 The Linux Kernel The Linux kernel continued

The kernel, or heart of Linux, can be adjusted and Recompiling your kernel is a bit of an “art”, but configured as you like. You must install the you can make your kernel smaller, more kernel source to do this. This resides in /usr/src. efficient, and more secure by building a custom If you want to recompile the kernel things have kernel for your hardware. changed a bit with kernel version 2.6.x in Fedora. In the /usr/src/linux2-6.x directory you might do: For much more detailed information go to:

(removes old and “stale” files) make mrproper http://www.kernel.org/ make xconfig (“.config” in configs/kernel-2.6.xx)

make (builds bzImage in arch/i386/boot) And download and read the documentation

make modules_install describing how to compile the kernel.

¡ mv arch/i386/boot/bzImage /boot/vmlinuz-2.6-new

Hint: kernel.org docs are still not “grub” savvy. 57 vi /boot/grub/grub.conf 58

Kernel loadable modules Firewalls

¢ Changed with kernel 2.5.48+. modprobe program was We mention this topic here as firewalls involve several aspects of simplified and now uses /etc/modprobe.conf vs. / Linux: etc/modules.conf to describe the “drivers” (modules) that With a modern you can expect that the will be loaded to support hardware on your PC. kernel already has support for “netfilter” built in.

¢ You can control incoming, outgoing and forwarded network You can view what modules are in use, load a module packets (tcp/udp/icmp) using “iptables” (netfilter).

manually, remove a module, and view module information: In Fedora you initialize iptables as a service in

¡ /sbin/insmod /etc/rc.d/init.d that reads the script /etc/sysconfig/iptables (this is just one method). ¡ /sbin/lsmod logical link to /sbin/insmod To make a set of network filter rules that work well as a

¡ firewall for your server you need to understand network /sbin/modprobe logical link to /sbin/insmod protocols and configuration very well as well as what you are ¡ /sbin/rmmod logical link to /sbin/insmod running and how you wish to control these services.

¡ iptables-save, iptables-restore (ip6tables-restore, etc.) are /sbin/modinfo available to make setting up rules easier. 59 60 Compiled software installation Compiled software installation cont.

It's likely you'll want to install software that's either Download a file fn.tar.gz to /usr/local/src.

not available as an RPM package, or that you tar -xvzf /usr/local/src/fn.tar.gz need to change or reconfigure before installation. cd /usr/local/src/fn-version

In such cases, you need to compile the software ./configure from source code. make It's very typical that software comes as a single

“tar” archive that is compressed (tar.gz or .tgz) make install An example of how to install from source --> This is all you need if it works, but now you don't have any good way to uninstall the software...

61 62

XWindows – Gnome – KDE XWindows – Gnome – KDE cont.

The first thing to understand is that Gnome and Which windowing system is better. There's no KDE use the XWindows graphical subsystem. correct answer to this. Generally KDE programs run in Gnome and To configurar how XWindows runs you specify vice-versa. this in the file /etc/X11/XF86Config. You can configure everything using menu-based For a server you do not need to run, or install, any tools, but if you understand how things work of these. you can edit /etc/X11/XF86Config directly. If you are going to run one (Gnome?), you can run To exit a graphical interface you may need to both (KDE and Gnome), or even more graphical change to run level 3 (or 2 or 1). windowing systems. You can, also, go directly to a terminal using alt-

63 64 ctrl-f2 through f6. Logs – how to know what's Logs continued happening

To configure what services will report events to There are many log files. For example, if you run a be logged see the file /etc/syslog.conf. webserver, like apache, all of the webserver logs

Take a look at the file /var/log/messages. The are likely to be in /var/log/httpd “tail” command is very useful for this. sendmail uses /var/log/maillog.

Get used to using /var/log/messages (among There are multiple software packages to read and others) to resolve problems. For example, automatically generate reports based on log files. problems running a service, type the command: See:

tail -f /var/log/messages ¡ http://nsrc.org/security/index.html#logging while you start and stop the service. to see some examples of available packages. 65 66

Summary part two More resources

The Linux operating system is built in a modular http://www.google.com/linux

manner to ensure stability, and to allow for http://www.linux.org/ extensibility and security. http://www.linuxdocs.org/ Having access to the Linux source code allows for rapid development and the resolution of O'Reilly books (http://www.oreilly.com/)

problems in a timely manner. ¡ “Linux Server Hacks” by Rob Flickenger

With Kernel 2.4, and 2.6, Linux comes closer in http://www.sourceforge.net/ terms of stability and scalability of operating systems like FreeBSD and Solaris when offering http://www.redhat.com/ network services. Hervey Allen - [email protected] 67 68