Logging Policies

Logging Policies The accounting system, the kernel, and various utilities all emit data that is logged and eventually ends up on your finite-sized disks. Most of that data has a limited useful lifetime and needs to be summarized, compressed, archived, and eventually thrown away. Logging Policies Logging policies vary from site to site. Common schemes include: Throw away all data immediately Reset log files at periodic intervals Rotate log files, keeping data for a fixed period of time Compress and archive logs to tape or other permanent media. The correct choice for your site depends on how much disk space you have and how security conscious you are. Even sites with an abundance of disk space must deal with the cancerous growth of log files. Whatever scheme that you select, maintenance of log files should be automated with cron. Throwing away log files We do not recommend throwing away all logging information. Sites that are subject to security problems routinely find that accounting data and log files provide important evidence of break-ins. Log files are also helpful for alerting you to hardware and software problems. In general, given a comfortable amount of disk space, data should be kept for at least a month and then discarded. In the real world, it may take this long for you to realize that your site has been compromised by a hacker and that you need to review the logs. If you need to go back further into the past, you can recover older logs from your backup tapes. Some administrators allow log files to grow until they become bothersome, then restart them from zero. This plan is better than keeping no data at all, but it does not guarantee that log entries will be retained for any particular length of time. Average disk usage may also be higher than with other management schemes. Rotating Log Files Most sites store each day’s log information on disk, sometimes in a compressed format. These daily files are kept for a specific period of time and then deleted. If you have sufficient disk space, it is handy to keep the log files uncompressed so that they can be easily searched with grep. One common way of implementing this policy is called rotation. In a rotation system, you keep backup files that are one day old, two days old and so on. Each day, a script renames the files to push older data toward the end of the chain. If a log file is called logfile, for example, the backup copies might be called logfile.1, logfile,2, etc. If you keep a week’s worth of data, there will be a logfile.7 but no logfile.8. Every day, the data in logfile.7 is lost as logfile.6 overwrites it. e.g. Suppose a log file needs daily attention and you want to archive its contents for three days to keep the sample short. The following script would implement an appropriate rotation policy: #!/bin/sh cd /var/log mv logfile.2 logfile.3 mv logfile.1 logfile.2 mv logfile logfile.1 cat /dev/null > logfile chmod 600 logfile Ownership information is important for some log files. You may need to run your rotation script from cron as the log files’ owner rather than as root, or you may need to add a chown command to the script. Archiving log files Some sites must archive all accounting data and log files as a matter of policy, perhaps to provide data for a potential audit. In this situation, log files should be first rotated on disk and then written to tape or other permanent media. This scheme reduces the frequency of tape backups and gives you fast access to recent data. Log files should always be included in your regular backup sequence. They may also be archived to a separate tape series. Separate tapes are more cumbersome, but they impose less of a documentation burden and will not interfere with your ability to recycle dump tapes. If you use separate tapes, we suggest that you use the tar format and write a script to automate your backup scheme. Files NOT To Manage You might be tempted to manage all log files with a rotation and archiving scheme. However, there are three files that you should not touch: /var/adm/lastlog /var/adm/wtmp, and /var/adm/utmp. lastlog records each user’s last login and is a sparse file indexed by UID. It stays smaller if your UIDs are assigned in some kind of numeric sequence. Do not copy lastlog, or it will use all of the disk space that ls –l reports. utmp, really utmpx on our Solaris system, attempts to keep a record of each user that is currently logged in. It is sometimes wrong, usually because a user’s shell was killed with an inappropriate signal and the parent of the shell did not clean up properly. utmpx is world-writable. wtmp, really wtmpx on our Solaris system, attempts to keep a history of user access and administrative information. .

Load more