Special Publication 800-92 Guide to Computer Security Log Management Recommendations of the National Institute of Standards and Technology Karen Kent Murugiah Souppaya GUIDE TO COMPUTER SECURITY LOG MANAGEMENT Reports on Computer Systems Technology The Information Technology Laboratory (ITL) at the National Institute of Standards and Technology (NIST) promotes the U.S. economy and public welfare by providing technical leadership for the nation’s measurement and standards infrastructure. ITL develops tests, test methods, reference data, proof of concept implementations, and technical analysis to advance the development and productive use of information technology. ITL’s responsibilities include the development of technical, physical, administrative, and management standards and guidelines for the cost-effective security and privacy of sensitive unclassified information in Federal computer systems. This Special Publication 800-series reports on ITL’s research, guidance, and outreach efforts in computer security and its collaborative activities with industry, government, and academic organizations. National Institute of Standards and Technology Special Publication 800-92 Natl. Inst. Stand. Technol. Spec. Publ. 800-92, 72 pages (September 2006) Certain commercial entities, equipment, or materials may be identified in this document in order to describe an experimental procedure or concept adequately. Such identification is not intended to imply recommendation or endorsement by the National Institute of Standards and Technology, nor is it intended to imply that the entities, materials, or equipment are necessarily the best available for the purpose. ii GUIDE TO COMPUTER SECURITY LOG MANAGEMENT Acknowledgements The authors, Karen Kent and Murugiah Souppaya of the National Institute of Standards and Technology (NIST), wish to thank their colleagues who reviewed drafts of this document and contributed to its technical content, especially Bill Burr, Elizabeth Chew, Tim Grance, Bill MacGregor, Stephen Quinn, and Matthew Scholl of NIST, and Stephen Green, Joseph Nusbaum, Angela Orebaugh, Dennis Pickett, and Steven Sharma of Booz Allen Hamilton. The authors particularly want to thank Anton Chuvakin of LogLogic and Michael Gerdes for their careful review and many contributions to improving the quality of this publication. The authors would also like to express their thanks to security experts Kurt Dillard of Microsoft, Dean Farrington of Wells Fargo Bank, Raffael Marty of ArcSight, Greg Shipley of Neohapsis, and Randy Smith of the Monterey Technology Group, as well as representatives from the Department of Energy, the Department of Health and Human Services, the Department of Homeland Security, the Department of State, the Department of Treasury, the Environmental Protection Agency, the National Institutes of Health, and the Social Security Administration, for their valuable comments and suggestions. Trademarks All names are registered trademarks or trademarks of their respective companies. iii GUIDE TO COMPUTER SECURITY LOG MANAGEMENT Table of Contents Executive Summary............................................................................................................ES-1 1. Introduction ................................................................................................................... 1-1 1.1 Authority................................................................................................................ 1-1 1.2 Purpose and Scope............................................................................................... 1-1 1.3 Audience ............................................................................................................... 1-1 1.4 Publication Structure ............................................................................................. 1-1 2. Introduction to Computer Security Log Management ................................................ 2-1 2.1 The Basics of Computer Security Logs.................................................................. 2-1 2.1.1 Security Software....................................................................................... 2-2 2.1.2 Operating Systems..................................................................................... 2-4 2.1.3 Applications................................................................................................ 2-4 2.1.4 Usefulness of Logs..................................................................................... 2-6 2.2 The Need for Log Management............................................................................. 2-7 2.3 The Challenges in Log Management..................................................................... 2-8 2.3.1 Log Generation and Storage ...................................................................... 2-8 2.3.2 Log Protection............................................................................................ 2-9 2.3.3 Log Analysis............................................................................................. 2-10 2.4 Meeting the Challenges....................................................................................... 2-10 2.5 Summary............................................................................................................. 2-11 3. Log Management Infrastructure................................................................................... 3-1 3.1 Architecture........................................................................................................... 3-1 3.2 Functions............................................................................................................... 3-3 3.3 Syslog-Based Centralized Logging Software......................................................... 3-5 3.3.1 Syslog Format............................................................................................ 3-5 3.3.2 Syslog Security .......................................................................................... 3-7 3.4 Security Information and Event Management Software ......................................... 3-9 3.5 Additional Types of Log Management Software................................................... 3-10 3.6 Summary............................................................................................................. 3-11 4. Log Management Planning........................................................................................... 4-1 4.1 Define Roles and Responsibilities ......................................................................... 4-1 4.2 Establish Logging Policies..................................................................................... 4-3 4.3 Ensure that Policies Are Feasible.......................................................................... 4-7 4.4 Design Log Management Infrastructures............................................................... 4-9 4.5 Summary............................................................................................................. 4-10 5. Log Management Operational Processes.................................................................... 5-1 5.1 Configure Log Sources.......................................................................................... 5-1 5.1.1 Log Generation .......................................................................................... 5-1 5.1.2 Log Storage and Disposal.......................................................................... 5-2 5.1.3 Log Security............................................................................................... 5-4 5.2 Analyze Log Data.................................................................................................. 5-5 5.2.1 Gaining an Understanding of Logs............................................................. 5-5 5.2.2 Prioritizing Log Entries ............................................................................... 5-6 5.2.3 Comparing System-Level and Infrastructure-Level Analysis....................... 5-7 iv GUIDE TO COMPUTER SECURITY LOG MANAGEMENT 5.3 Respond to Identified Events................................................................................. 5-8 5.4 Manage Long-Term Log Data Storage .................................................................. 5-9 5.5 Provide Other Operational Support...................................................................... 5-10 5.6 Perform Testing and Validation ........................................................................... 5-10 5.7 Summary............................................................................................................. 5-11 List of Appendices Appendix A— Glossary ........................................................................................................A-1 Appendix B— Acronyms ......................................................................................................B-1 Appendix C— Tools and Resources....................................................................................C-1 Appendix D— Index ..............................................................................................................D-1 List of Figures Figure 2-1. Security Software Log Entry Examples ................................................................ 2-3 Figure 2-2. Operating System Log Entry Example ................................................................. 2-4 Figure 2-3. Web Server Log Entry Examples ........................................................................