Denial of Service Detection and Mitigation for SIP Communincation Networks
Total Page：16
File Type：pdf, Size：1020Kb
Denial-of-Service Detection and Mitigation for SIP Communication Networks vorgelegt von Diplom-Informatiker Sven Ehlert Von der Fakult¨atIV - Elektrotechnik und Informatik der Technischen Universit¨atBerlin zur Erlangung des akademischen Grades Doktor der Ingenieurwissenschaften Dr.-Ing. genehmigte Dissertation Promotionsausschuss: Vorsitzender: Prof. Dr. Jean-Pierre Seifert Pr¨uferder Dissertation: 1. Prof. Dr. Thomas Magedanz 2. Prof. Dr. Erwin Rathgeb Tag der wissenschaftlichen Aussprache: 13.10.2009 Berlin 2009 D 83 Abstract The Session Initiation Protocol (SIP) is the multimedia communication pro- tocol of the future. Used for Voice-over-IP (VoIP), Internet Multimedia Subsystem (IMS) and Internet Protocol Television (IPTV), its concepts are based on mature and open standards and its use is increasing rapidly within recent years. However, with its acceptance as a mainstream communica- tion platform, security concerns become ever more important for users and service providers. In this thesis we identify different attacks on SIP-based networks with the focus on Denial-of-Service attacks (DoS) flooding attacks. We evaluate SIP infrastructure for DoS attack possibilities and demonstrate a completely new attack which utilises a combination of the SIP and Domain Name Service (DNS) system. We propose three different DoS detection and mitigation schemes, including one to handle this particular SIP DNS attack. We also provide a first step into Distributed DoS mitigation by introducing a firewall pinholing scheme. Distributed DoS mitigation is only marginally addressed by current research works. We also evaluate the requirements for a self-sufficient and scalable SIP security framework, where attack countermea- sures can be evaluated and tested. We use this framework for our solutions and validate their effectiveness for DoS mitigation. With these solutions, gen- eral SIP networks will be more robust against flooding DoS and Distributed DoS attacks. i ii Deutsche Zusammenfassung F¨ur Multimedia Kommunikation spielt das Session Initiation Protocol (SIP) eine immer wichtigere Rolle. SIP ist ein ausgereifter, offener Standard f¨urIn- ternet Telefonie (Voice-over-IP, VoIP), Internet Multimedia Subsystem (IMS) oder IP-basiertes Fernsehen (IPTV), dessen Verbreitung in den letzten Jahr- en stark angestiegen ist. Mit der breiten Akzeptanz von SIP als etabliertem Internetprotokoll werden Sicherheitsaspekte f¨urAnwender und Serviceanbie- ter immer bedeutender. Diese Dissertation behandelt verschiedene Angriffe auf SIP-basierte Netzwerke mit dem Schwerpunkt Uberlaufangriffe¨ (Denial- of-Service, DoS). Wir evaluieren SIP-Infrastrukturen auf m¨ogliche Angriffs- punkte und stellen zudem einen komplett neuen DoS Angriff auf SIP-Netz- werke vor. Dieser nutzt eine Schwachstelle bei der Zusammenarbeit von SIP mit dem Domain Name Service (DNS) aus. Wir f¨uhrenaußerdem drei verschiedene Sicherheitsmethoden zur Erkennung und teilweisen Ab- schw¨achung von DoS-Angriffen ein, inklusive einer Methode als Gegenmaß- nahme gegen den angef¨uhrtenSIP-DNS Angriff. Weiter pr¨asentieren wir einen ersten Schritt zur Sicherung vor ”Verteilten DoS-Angriffen”. Dieses Thema wurde bislang nur in ersten Ans¨atzenvon der Forschergemeinschaft behandelt. Zus¨atzlich er¨orternwir die Anforderungen an ein autarkes und skalierbares SIP-Sicherheitsframework zum Evaluieren, Testen und Validier- en der vorgeschlagenen Sicherheitsmethoden. Durch die Anwendung dieser Methoden kann in Zukunft ein SIP-Netzwerk besser gegen DoS-Angriffe ab- gesichert werden. iii iv Acknowledgements This thesis was hard work, and definitely not possible without the help of others. In particular: • My supervisors Prof. Dr. Thomas Magedanz and Prof. Dr. Erwin Rathgeb • My girlfriend Trine • My family • My colleagues Dorgham Sisalem, Jens Fiedler and Yacine Rebahi • My students Ge Zhang, Chengjian Wang, Andreea-Ancuta Onofrei, Erkan G¨uler,and Martin Becker You know what you did – a big thanks! v vi Contents 1 Introduction 3 1.1 Motivation . 3 1.1.1 Importance of SIP for Communication Networks . 4 1.1.2 Complexity of SIP Networks . 5 1.1.3 SIP Challenges . 6 1.2 PhD Scope . 6 1.3 Threat Impact and Requirements . 8 1.4 Related Works . 9 1.4.1 General SIP Security Related Work . 9 1.4.2 SIP DoS Related Work . 9 1.5 Methodology . 11 1.6 Major Contribution . 12 1.7 Thesis Structure . 13 2 Background Information 15 2.1 Multimedia Communication with SIP . 15 2.1.1 SIP Entities . 16 2.1.2 SIP Structure . 19 2.1.3 SIP Message Format . 20 2.1.4 Dialogs and Transactions . 22 2.1.5 Protocol Operation . 25 2.1.6 SIP Security Mechanisms . 28 2.1.7 Session Description Protocol . 30 2.1.8 Real-time Transport Protocol . 31 2.1.9 SIP Alternatives . 32 2.2 SIP Network Application . 32 2.2.1 IP-based Telephony Networks . 33 2.2.2 Next Generation Networks . 33 2.3 Denial of Service Attacks . 35 2.3.1 Motivation for DoS Attacks . 35 2.3.2 DoS Targets . 36 vii viii CONTENTS 2.3.3 Distributed Denial of Service Attack . 43 2.3.4 Mitigation Countermeasure Analysis . 47 2.4 Intrusion Detection Systems . 54 2.4.1 General Overview of Intrusion Detection . 54 2.4.2 Intrusion Detection System Types . 55 2.4.3 Detection Strategies . 58 2.5 Relevant Tools . 59 2.5.1 Netfilter / IPtables . 59 2.5.2 SER . 59 2.5.3 PROTOS Suite . 61 2.5.4 SIPp . 61 3 SIP Security Threats with Focus on DoS 63 3.1 General Threats and Related Works . 63 3.2 SIP DoS Introduction . 65 3.3 SIP DoS by Message Payload Tampering . 66 3.3.1 Example: SIP Message Tampering with SQL . 67 3.4 SIP DoS by Message Flow Tampering . 68 3.4.1 REGISTER Attack . 69 3.4.2 INVITE Attack . 71 3.4.3 BYE Attack . 71 3.4.4 CANCEL Attack . 71 3.4.5 UPDATE Attacks . 72 3.4.6 REFER Requests . 73 3.5 SIP DoS by Message Flooding . 73 3.5.1 Exploitable SIP Resources . 75 3.5.2 Overview of SIP Flooding Attack Scenarios . 77 3.5.3 Attack Amplification . 86 3.5.4 SPIT and Denial of Service (DoS) Attacks . 88 4 DoS Protection Requirement Analysis 91 4.1 Requirements for Payload Tampering Protection . 91 4.2 Requirements for Message Flow Tampering Protection . 92 4.3 Requirements for Message Flooding Protection . 93 4.3.1 Possible Countermeasures Against Memory Exploita- tion Attacks . 93 4.3.2 Countermeasures Against CPU Attacks . 95 4.4 Summary of Operational Guidelines . 95 4.5 Requirements for External Monitoring . 97 CONTENTS ix 5 Security Solution Specification 99 5.1 General Protection Framework . 99 5.1.1 Overview . 99 5.1.2 Filter- and Scanner Node (”Filter”) . 100 5.1.3 Analysis Node (”Analyzer”) . 101 5.1.4 Decision Node (”Decider”) . 102 5.1.5 User Interaction . 102 5.2 General SIP DoS Attack Protection . 103 5.2.1 Background: The SIP State model . 103 5.2.2 Solution Approach: Finite Server Transaction State Machines . 104 5.2.3 Attack Detection and Mitigation . 107 5.3 Distributed SIP DoS Attack Protection . 109 5.3.1 Background: Greylisting . 110 5.3.2 Solution Approach: Firewall Pinholing . 110 5.3.3 Pinholing Parameters . 111 5.4 Combined SIP-DNS DoS Attack Protection . 113 5.4.1 Background: Domain Name Service . 114 5.4.2 DNS Usage in SIP Infrastructures . 115 5.4.3 Scope of the Attack . 115 5.4.4 Basic Prevention Possibilities . 117 5.4.5 DNS Implementations with SIP Servers . 118 5.4.6 Solution Approach: Intelligent Unblocking DNS Cache 119 6 Implementation 123 6.1 VoIP Defender Architecture . 123 6.1.1 Filter . 123 6.1.2 Analyzer . 125 6.1.3 Decider . 126 6.1.4 Component Interaction . 127 6.2 The State Machine Module . 128 6.3 The Pinholing Module . 131 6.4 The DNS Cache . 131 7 Validation and Optimisation 133 7.1 VoIP Defender Validation . 133 7.1.1 Test Bed Setup . 133 7.1.2 Round Trip Time Delay . 134 7.1.3 Throughput . 134 7.2 The State Machine . 135 7.2.1 Test Bed . 135 x CONTENTS 7.2.2 Testing Scenario Setup . 136 7.2.3 Results . 138 7.2.4 Latency Time and CPU, Memory Usage . 140 7.3 Pinholing Validation . 141 7.3.1 Operation Evaluation Scenario . 141 7.3.2 Performance Evaluation Scenario . 142 7.4 DNS Cache Validation . 144 7.4.1 Test Bed Setup . 144 7.4.2 Feasibility of the Attack . 146 7.4.3 Evaluation . 146 7.5 Performance Optimisation . 153 7.5.1 Reducing the Number of Rule Updates in the Pinholing Module . 153 7.5.2 Generally Reducing the Number of Generated Firewall Rules . 156 8 Comparison with Other Approaches 169 8.1 Evaluation Criteria for DoS Defence Systems . 169 8.1.1 Algorithm-related Evaluation Criteria . 170 8.1.2 Framework-related Evaluation Criteria . 171 8.2 Survey of SIP DoS Countermeasure Solutions . 172 8.2.1 Iancu03 . 172 8.2.2 Reynolds03 . 173 8.2.3 Wu04 . 174 8.2.4 Geneiatakis05 . 175 8.2.5 Markl05 . 175 8.2.6 Chen06 . 176 8.2.7 Niccolini06 . 176 8.2.8 Sengar06-1 . 177 8.2.9 Sengar06-2 . 178 8.2.10 Nassar06 . 179 8.2.11 Rebahi07 . 180 8.2.12 Ding07 . 180 8.2.13 Nassar07 . 181 8.2.14 Barry07 . ..