JPJ8;D@E Sysstat
Dfe`kfi`e^n`k_k_\JpjjkXkkffcZfcc\Zk`fe GIF: The Sysstat tools, featuring sar, iostat, mpstat, and pidstat, acquire system parameters and calculate statistics. BY MARKUS FEILNER AND BASTIAN KAMES The sar set includes the data collector sadc; sa1, which assembles binary data CPU, or by interrupt or APIC source in a system activity file; sa2, which writes activity reports; and sadf, a tool used to display sar data in formats such erver performance is down or as CSV and XML. kSar, which is avail- non-existent. To look for clues, able as a separate project [2], is a conve- Jyou check the logfiles, take a nient graphing tool for sar data. quick look at the /proc directory, and run The Sysstat tools provide a practical, tools such as Vmstat or Top, but to no simple set of building blocks that are The information provided by the tools is avail: time to launch the Sysstat collec- easy to integrate into the everyday life typically grouped into the categories of tion [1]. Sysstat is a group of simple of the network. memory, network, processors, CPU, and Linux command-line tools for perfor- mance analysis and monitoring. Accord- M\ijXk`c\ -d -r the pagefaults, ing to the Sysstat project, the toolset “… Iostat, pidstat, mpstat, and the sar utili- -u the CPU usage, and -w the kernel’s contains various utilities common to ties provide information on system task-switching activity. Sar and iostat many commercial Unixes, and tools you parameters, such as: also include options for displaying data can schedule via cron to collect and his- for network filesystems, and sar lets the torize performance and activity data.” or by device, partition, NFS drive, The Sysstat set collects system infor- process ID, or process name mation, stores it for a period of time, - JXiXe[bJXi and calculates mean values, letting you cess Unix diehards might complain that you query individual system parameters at can gather the same data with the use of specific times for more flexible trouble- swap file usage internal Linux tools. Admins can use shooting. The tools work well with cron Awk to calculate the load average for so that you can take readings of system counts: global, for individual pro- any given period of time. All you need is performance at predefined intervals for a cesses or process trees some shell survival skills and the data flexible, customizable approach to data from the proc filesystem (Figure 2). Al- collection. The Sysstat project defines spawning new processes though this is true, sar makes the task the collection as follows: much simpler. The command Get a Recent Linux for devices, partitions, and network Pidstat lets administrators query and sar -q P0 120 2 filesystems. monitor I/ O load. Because this level of functionality requires a special kernel op- gives you the queue length and load av- tion, older distributions will not support erage. P0 specifies the CPU to monitor, it. In fact, if you have Ubuntu, you will and 120 2 tells sar to take two samples need at least Hardy, and openSUSE fans will need version 11 or a modified kernel for a period of two minutes each, and on system activities related to CPU, to run the recently released version 8 of then display the mean values for 1, 5, memory, interrupts, interfaces, kernel Sysstat by Sébastien Godard and the and 15 minutes. The output is shown in tables, and other factors. team of developers backing him up. Figure 2. 62 ISSUE 95 OCTOBER 2008 062-064_systat.indd 62 13.08.2008 15:47:55 Uhr Sysstat JPJ8;D@E native to time-consuming monitoring with Top, Strace, and other debug- ging tools, and that meant sitting in front of the screen. Today, one of the Sysstat tools makes this unnecessary. Pidstat cre- ates statistics reports for individual Linux tasks. To monitor processes over extended periods of time, you can use pidstat and output the mean values. The Sysstat developers describe how they even =`^li\(1K_\bJXikffcflkglkj ]ifdc\]kkfi`^_k k_\@&Fk_ifl^_glkf]Xe]X`icplejki\jj\[dX`cj\im\i used pidstat to track Z_\Zb`e^k_\`e$Yfo\jXe[Z_\Zb`e^]fijgXd #Xgfn\i]`c\j\im\i]fi(''\dgcfp\\j ilee`e^XYXZblg`e down a memory hole in k_\d`[[c\f]k_\e`^_kXe[j\im`e^[XkXkflj\ijXk/Xd #Xe[X[\jbkfgdXZ_`e\% the Sysstat tools. If you use, say, pidstat -u 20 2 -C If inexplicable hardware problems formance and can even show you multi- processname to monitor an individual occur, you can run ple servers at the same time with an SSH process, the result is the average load for command (Figure 1). The Java tool will two requests within a period of 40 sec- sar -I interrupt run sar -A on a remote server, receive a onds (see Figure 2). full set of sar data, and display the data to investigate individual interrupts. If the for point and click navigation in a tree @ejXe[Flkj CPU load, which you can get by running If you have kernel 2.6.20 or newer, with sar -u, tells you that the server is work- that child windows are synchronized. the CONFIG_TASK_IO_ACCOUNTING ing as hard as it can, this might not Clicking on a sub-item in a tree diagram option built in, you can also use pidstat mean it is time to buy a new machine. will automatically update the graphs for Long-term monitoring might be prefera- all the hosts currently shown. This fea- ture is perfect for administrators wanting with the right support, typing console and the graphical front end. to compare multiple machines. The kSar graphing tool is well suited pidstat -d 2 3 to the task of graphing sar results. kSar @em\jk`^Xk`e^GifZ\jj\j gives you a fast overview of system per- Bugs or loops in active programs often will generate output as shown in Figure bring a server to 3. Pidstat will list the processes generat- its knees. For- merly, Linux ad- or write), along with the data through- mins had no alter- put (in Kbps). =`^li\*1C`eloX[d`e`jkiXkfijZXeefnlj\g`[jkXkkfdfe`kfi@&F =`^li\)18nbj\\djhl`k\c`d`k\[ZfdgXi\[n`k_jXiXe[g`[jkXk% cfX[]fi`e[`m`[lXcgifZ\jj\j% OCTOBER 2008 ISSUE 95 63 062-064_systat.indd 63 13.08.2008 15:47:57 Uhr JPJ8;D@E Sysstat To allow this to happen, you need a Listing 1 shows the /etc/cron.d/sysstat */10 * * * * root 5 recent kernel, the packages from the file on Ubuntu. In the simplest of all /usr/lib/sa/sa1 1 1 Sysstat website, and a matching cron cases, the following two lines in /etc/ 53 23 * * * root 5 configuration. crontab are all you need: /usr/lib/sa/sa2 -A Listing 1: Cronjobs for Sysstat The first entry starts sadc (the System Activity Data Collector), the server dae- 01 # Global variables: mon that collects data every second and 02 # stores its findings under /var/log/sa. The 03 # our configuration file logfile format is saXX, where XX is the 04 DEFAULT=/etc/default/sysstat date. The second part with sar -A just ro- 05 # default setting, overriden in the above file tates the logfiles. 06 ENABLED=false 07 SA1_OPTIONS=““ @fjkXkXe[DgjkXk 08 # Activity reports every 10 minutes everyday Besides sar and pidstat, the Sysstat pack- 09 5-55/10 * * * * root [ -x /usr/lib/sysstat/sa1 ] && { [ -r „$DEFAULT“ age also includes iostat and mpstat. Mp- ] && . „$DEFAULT“ ; stat monitors the processor you pass in 10 [ „$ENABLED“ = „true“ ] && exec /usr/lib/sysstat/sa1 $SA1_OPTIONS 1 to it; iostat is popular with file, database, 1 ; } and mail server administrators as a tool 11 # Additional run at 23:59 to rotate the statistics file disk, partition, and filesystem. 12 59 23 * * * root [ -x /usr/lib/sysstat/sa1 ] && { [ -r „$DEFAULT“ ] For example, && . „$DEFAULT“ ; [ 13 „$ENABLED“ = „true“ ] && exec /usr/lib/sysstat/sa1 $SA1_OPTIONS 60 2 ; } iostat -m -n 10 2 Listing 2: NFS Mount Point Activity monitors an NFS mount point and out- puts the volume of data in megabytes 01 # iostat -m -n 10 2 (Listing 2). For a single partition, you 02 Filesystem: rMB_nor/s wMB_nor/s rMB_dir/s wMB_dir/s rMB_svr/s can do the same by typing iostat -m -x wMB_svr/s rops/s wops/s sda2 10 2, as shown in Listing 3. 03 server:/home 0,03 0,01 0,00 0,00 0,01 0,02 6,53 6,53 8Yjfclk\cp 64 ISSUE 95 OCTOBER 2008 062-064_systat.indd 64 13.08.2008 15:47:57 Uhr