Information Technology / Security & Auditing Mohammed Pathan Able to propagate quickly and change their payload with each infection, Automatic Defense Against polymorphic worms have been able to evade even the most advanced intrusion detection systems (IDS). And, because zero-day worms require only seconds Automatic Defense Against Zero-day Polymorphic in Worms Communication Networks to launch flooding attacks on your servers, using traditional methods such as Zero-day Polymorphic Worms manually creating and storing signatures to defend against these threats is just too slow. in Communication Networks Bringing together critical knowledge and research on the subject, Automatic Defense Against Zero-day Polymorphic Worms in Communication Networks Mohssen Mohammed • Al-Sakib Khan Pathan details a new approach for generating automated signatures for unknown polymorphic worms. It presents experimental results on a new method for polymorphic worm detection and examines experimental implementation of signature-generation algorithms and double-honeynet systems. If you need some background, the book includes an overview of the fundamental terms and concepts in network security, including the various security models. Clearing up the misconceptions about the value of honeypots, it explains how they can be useful in securing your networks and identifies open-source tools you can use to create your own honeypot. There’s also a chapter with references to helpful reading resources on automated signature generation systems. The authors describe cutting-edge attack detection approaches and detail new algorithms to help you generate your own automated signatures for polymorphic worms. Explaining how to test the quality of your generated signatures, the text will help you develop the understanding required to effectively protect your communication networks. Coverage includes intrusion detection and prevention systems (IDPS), zero-day polymorphic worm collection methods, double-honeynet system configurations, and the implementation of double- honeynet architectures. K15460 ISBN: 978-1-4665-5727-7 90000 www.crcpress.com 9 781466 557277 www.auerbach-publications.com K15460 cvr mech.indd 1 4/23/13 10:09 AM Automatic Defense Against Zero-day Polymorphic Worms in Communication Networks OTHER INFORMATION SECURITY BOOKS FROM AUERBACH Asset Protection through Security Awareness Guide to the De-Identification of Personal Tyler Justin Speed Health Information ISBN 978-1-4398-0982-2 Khaled El Emam Automatic Defense Against Zero-day ISBN 978-1-4665-7906-4 Polymorphic Worms in Communication Information Security Governance Simplified: Networks From the Boardroom to the Keyboard Mohssen Mohammed and Al-Sakib Khan Pathan Todd Fitzgerald ISBN 978-1-4665-5727-7 ISBN 978-1-4398-1163-4 The Complete Book of Data Anonymization: Information Security Policy Development for From Planning to Implementation Compliance: ISO/IEC 27001, NIST SP 800-53, Balaji Raghunathan HIPAA Standard, PCI DSS V2.0, and AUP V5.0 ISBN 978-1-4398-7730-2 Barry L. Williams The Complete Guide to Physical Security ISBN 978-1-4665-8058-9 Paul R. Baker and Daniel J. Benny Information Technology Control and Audit, ISBN 978-1-4200-9963-8 Fourth Edition Conflict and Cooperation in Cyberspace: Sandra Senft, Frederick Gallegos, and Aleksandra Davis The Challenge to National Security ISBN 978-1-4398-9320-3 Panayotis A Yannakogeorgos and Adam B Lowther Iris Biometric Model for Secured Network Access (Editors) Franjieh El Khoury ISBN 978-1-4665-9201-8 ISBN 978-1-4665-0213-0 Cybersecurity: Public Sector Threats Managing the Insider Threat: No Dark Corners and Responses Nick Catrantzos Kim J. Andreasson ISBN 978-1-4398-7292-5 ISBN 978-1-4398-4663-6 Network Attacks and Defenses: A Hands-on The Definitive Guide to Complying with the Approach HIPAA/HITECH Privacy and Security Rules Zouheir Trabelsi, Kadhim Hayawi, Arwa Al Braiki, John J. Trinckes, Jr. and Sujith Samuel Mathew ISBN 978-1-4665-0767-8 ISBN 978-1-4665-1794-3 Digital Forensics Explained Noiseless Steganography: The Key to Covert Greg Gogolin Communications ISBN 978-1-4398-7495-0 Abdelrahman Desoky ISBN 978-1-4398-4621-6 Digital Forensics for Handheld Devices Eamon P. Doherty PRAGMATIC Security Metrics: Applying ISBN 978-1-4398-9877-2 Metametrics to Information Security W. Krag Brotby and Gary Hinson Effective Surveillance for Homeland Security: ISBN 978-1-4398-8152-1 Balancing Technology and Social Issues Francesco Flammini, Roberto Setola, and Giorgio Securing Cloud and Mobility: A Practitioner’s Guide Franceschetti (Editors) Ian Lim, E. Coleen Coolidge, and Paul Hourani ISBN 978-1-4398-8324-2 ISBN 978-1-4398-5055-8 Electronically Stored Information: Security and Privacy in Smart Grids The Complete Guide to Management, Yang Xiao (Editor) Understanding, Acquisition, Storage, ISBN 978-1-4398-7783-8 Search, and Retrieval Security for Wireless Sensor Networks using David R. Matthews Identity-Based Cryptography ISBN 978-1-4398-7726-5 Harsh Kupwade Patil and Stephen A. Szygenda Enterprise Architecture and Information ISBN 978-1-4398-6901-7 Assurance: Developing a Secure Foundation The 7 Qualities of Highly Secure Software James A. Scholz Mano Paul ISBN 978-1-4398-4159-4 ISBN 978-1-4398-1446-8 AUERBACH PUBLICATIONS www.auerbach-publications.com • To Order Call: 1-800-272-7737 • E-mail: [email protected] Automatic Defense Against Zero-day Polymorphic Worms in Communication Networks Mohssen Mohammed • Al-Sakib Khan Pathan CRC Press Taylor & Francis Group 6000 Broken Sound Parkway NW, Suite 300 Boca Raton, FL 33487-2742 © 2013 by Taylor & Francis Group, LLC CRC Press is an imprint of Taylor & Francis Group, an Informa business No claim to original U.S. Government works Version Date: 20130304 International Standard Book Number-13: 978-1-4665-5728-4 (eBook - PDF) This book contains information obtained from authentic and highly regarded sources. Reasonable efforts have been made to publish reliable data and information, but the author and publisher cannot assume responsibility for the validity of all materials or the consequences of their use. The authors and publishers have attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this form has not been obtained. If any copyright material has not been acknowledged please write and let us know so we may rectify in any future reprint. Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmit- ted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information storage or retrieval system, without written permission from the publishers. For permission to photocopy or use material electronically from this work, please access www.copyright. com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that provides licenses and registration for a variety of users. For organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged. Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe. Visit the Taylor & Francis Web site at http://www.taylorandfrancis.com and the CRC Press Web site at http://www.crcpress.com Dedication To my father, Mohammed Zain Elabdeen Mohammed; my sister, Maali Mohammed Zain Elaabdeen; faculty of Mathematical Science, University of Khartoum, Khartoum, Sudan, I especially would like to thank Dr. Mohsin Hashim and Dr. Eihab Bashier, and Dr. Lorenzo Cavallaro, Information Security Group, Royal Holloway, University of London. Mohssen Mohammed To my father, Abdus Salam Khan Pathan; my mother, Delowara Khanom; and my loving wife, Labiba Mahmud. Al-Sakib Khan Pathan © 2010 Taylor & Francis Group, LLC Contents P REFACE xiii A BOUT THE AUTHORS xvii C HA P TER 1 THE FU N DA M E N TA L C ONCE P TS 1 1.1 Introduction 1 1.1.1 Network Security Concepts 1 1.1.2 Automated Signature Generation for Zero-day Polymorphic Worms 20 1.2 Our Experience and This Book’s Objective 22 References 23 C HA P TER 2 C OM P UTER N ETWORKING 25 2.1 Computer Technologies 25 2.2 Network Topology 26 2.2.1 Point-to-Point Topology 26 2.2.2 Daisy-Chain Topology 27 2.2.3 Bus (Point-to-Multipoint) Topology 27 2.2.4 Distributed Bus Topology 27 2.2.5 Ring Topology 29 2.2.6 Dual-Ring Topology 29 2.2.7 Star Topology 29 2.2.8 Star-Wired Bus Topology 30 2.2.9 Star-Wired Ring Topology 31 2.2.10 Mesh Topology 32 2.2.11 Hierarchical or Tree Topology 32 2.2.12 Dual-Homing Topology 32 © 2010 Taylor & Francis Group, LLC VII VIII CONTENTS 2.3 Internet Protocol 34 2.4 Transmission Control Protocol 34 2.5 IP Routers 35 2.6 Ethernet Switch 35 2.7 IP Routing and Routing Table 36 2.8 Discussion on Router 37 2.8.1 Access Mechanisms for Administrators 37 2.8.2 Security Policy for a Router 38 2.8.3 Router Security Policy Checklist 40 2.9 Network Traffic Filtering 42 2.9.1 Packet Filtering 42 2.9.2 Source Routing 43 2.10 Tools Used for Traffic Filtering or Network Monitoring 43 2.10.1 Packet Capture 44 2.11 Concluding Remarks 44 References 45 C HA P TER 3 I NTRUSION D ETECTION AND P REVENTION SYSTEMS (IDPS S) 47 3.1 Introduction 47 3.2 IDPS Detection Methods 54 3.2.1 Signature-Based Detection 54 3.2.2 Anomaly-Based Detection