Towards the Detection of Encrypted Peer-to-Peer File Sharing Traffic and Peer-to-Peer TV Traffic Using Deep Packet Inspection Methods August 2009 ! David Alexandre Milheiro de Carvalho Towards the Detection of Encrypted Peer-to-Peer File Sharing Traffic and Peer-to-Peer TV Traffic Using Deep Packet Inspection Methods DISSERTATION Submitted to University of Beira Interior in partial fulfillment of the requirements for the Degree of MASTER OF SCIENCE in Information Systems and Technologies by David Alexandre Milheiro de Carvalho (5-year Bachelor of Science) Network and Multimedia Computing Group Department of Computer Science University of Beira Interior Covilhã, Portugal www.di.ubi.pt Copyright c 2009 by David Alexandre Milheiro de Carvalho. All right reserved. No part of this publication can be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the previous written permission of the author. Title image: Heraldry of the University of Beira Interior. Towards the Detection of Encrypted Peer-to-Peer File Sharing Traffic and Peer-to-Peer TV Traffic Using Deep Packet Inspection Methods Author: David Alexandre Milheiro de Carvalho Student Number: 2274 E-mail: [email protected] Abstract This dissertation is devoted to the study of Peer-to-Peer (P2P) network traffic iden- tification, using Deep Packet Inspection (DPI) methods. The approach followed in this work is based on the analysis of the content of a packet payload, being paid particular attention to the cases where encryption or obfuscation is used. The protocols and applications under study along this dissertation are organized into two main categories: P2P file sharing (BitTorrent, Gnutella and eDonkey) and P2P TV (Livestation, TVU Player and Goalbit). The history of P2P and its major milestones are briefly presented, along with their classification according to the func- tionalities they provide and the network protocol architectures being used by them. Studies on the evolution and current state in the detection of P2P traffic are particu- larly detailed, as they were the main motivation towards the detection of both encrypted P2P file sharing and P2P TV traffic. The detection of Peer-to-Peer traffic is accomplished by using a set of open source tools, emphasizing Snort, Wireshark and Tcpdump. Snort is used for triggering the alerts concerning this kind of traffic, by using a specified set of rules. These are man- ually created, based on the observed P2P traffic protocol signatures and patterns, by using Wireshark and Tcpdump. For the storage and visualization of the triggered alerts in a user friendly manner, two open source tools were used, respectively, MySQL and BASE. Finally, the main conclusions achieved in this work are briefly exposed. A section dedicated to future work contains possible directions that may be followed in order to improve this work. Supervisor: Dr. Mário Marques Freire, Full Professor at the Department of Computer Science, University of Beira Interior. Preface First of all, I would like to thank to my supervisor, Professor Mário Marques Freire, for giving me the opportunity and credit for integrating his dynamic investigation team. During the period when I was working in the MsC thesis, his support, guidance and most important, motivation, were a constant presence whether regarding technical issues or any other matter. He also provided the means so I could perform all the activities, without having limitations of any kind. This work has been partially funded by Fundação para a Ciência e a Tecnologia through TRAMANET Project contract PTDC/EIA/73072/2006. I am also grateful to University of Beira Interior, particularly to the Department of Computer Science and to the Network and Multimedia Computing Group, for providing excellent work conditions and such a pleasant environment for researchers and students. I would also like to express my gratitude to Pedro Ricardo de Morais Inácio and João Vasco Paulo Gomes, both PhD students under the supervision of Professor Mário Marques Freire, for expressing their support for this work. Precious tips about the LATEX formatting system were provided to me by Professor Simão Melo de Sousa, which allowed me to improve the writing of this thesis. He also guided me for several times, allowing me achieve the pretended results, for which I would like to express my sincere gratitude. A special thank you to my mother Maria Deolinda and my brother Luís Miguel, for having faith in me through all these years, not only regarding my academic or professional course, but also in every single personal project in which I was involved in. Finally, I would like to thank to my wife Elisabete for her motivation, support and understanding during this first year of our marriage, in which, unfortunately, I could not be as present as I would like to. For many months, most of my free time was dedicated to this work, abdicating on many opportunities of spending time. For her, my truly gratitude and love. David Alexandre Milheiro de Carvalho Covilhã, Portugal iii Contents Preface iii Contents v List of Figures ix List of Tables x 1 Introduction 1 1.1 Focus . 1 1.2 Problem Definition and Goals . 2 1.3 Thesis Organization . 3 1.4 Main Contributions . 4 2 Peer-to-Peer Systems 5 2.1 Brief Perspective of P2P History . 5 2.2 P2P Definition . 9 2.3 Classification . 10 2.3.1 Functionalities . 10 2.3.2 Architecture . 10 2.4 P2P Traffic Evolution . 20 2.4.1 CAIDA . 20 2.4.2 ipoque . 21 2.5 State of Art in P2P Detection . 27 2.5.1 Legal Issues . 27 2.5.2 Classification of Mechanisms for P2P Traffic Detection . 28 2.5.3 Currently Available DPI Software . 30 2.5.4 Currently Available DPI Hardware . 35 v CONTENTS 3 Experimental Testbed 39 3.1 Introduction . 39 3.2 Lab of the Network and Multimedia Computing Group . 39 3.3 Hardware . 41 3.4 Network Configurations . 42 3.4.1 Firewalls . 42 3.4.2 Traffic Forwarding . 44 3.5 DPI and Network Software . 46 3.5.1 Snort . 46 3.5.2 Barnyard . 51 3.5.3 Apache . 53 3.5.4 MySQL . 53 3.5.5 BASE . 54 3.5.6 Wireshark . 56 3.6 P2P File Sharing Protocols and Applications . 57 3.6.1 BitTorrent Protocol . 58 3.6.2 eDonkey . 59 3.6.3 Gnutella . 60 3.7 P2P TV . 61 3.7.1 LiveStation . 62 3.7.2 TVU Player . 63 3.7.3 Octoshape . 64 3.7.4 Goalbit . 65 3.7.5 Joost . 65 4 P2P Traffic Detection 67 4.1 Introduction . 67 4.2 BitTorrent . 68 4.2.1 BitTorrent Application . 68 4.2.2 Vuze Application . 71 4.3 Gnutella . 76 4.3.1 LimeWire . 76 4.3.2 GTK-Gnutella . 82 4.4 eDonkey . 86 4.4.1 eMule . 86 4.4.2 aMule . 92 4.5 P2P TV . 95 4.5.1 Livestation . 95 4.5.2 TVU Player . 97 4.5.3 Goalbit . 101 vi CONTENTS 5 Conclusions and Future Work 105 5.1 Conclusions . 105 5.1.1 BitTorrent . 106 5.1.2 Gnutella . 106 5.1.3 eDonkey . 107 5.1.4 P2P TV . 108 5.2 Future Work . 109 5.2.1 Combining DPI and Behavior Methods . 110 5.2.2 Mobile P2P . 110 5.2.3 Defeating Encryption . 110 5.2.4 Snort Inline . 111 5.2.5 Snort Performance Measurement . 112 Bibliography 113 Appendix 119 A Snort rules for eDonkey 121 A.1 Client/Server TCP . 121 A.2 Client/Server UDP . 124 A.3 Client/Client TCP . 126 A.4 Extended Client/Client TCP . 130 A.5 Extended Client/Client UDP . 132 A.6 KAD Client/Client UDP . 133 B Snort Rules for Gnutella 139 B.1 General Gnutella TCP . 139 B.2 LimeWire TCP . 140 B.3 LimeWire UDP . 141 B.4 GTK-Gnutella UDP . 143 C Snort Rules for BitTorrent 145 C.1 General BitTorrent TCP . 145 C.2 Vuze Plain Encryption TCP . 146 C.3 External TCP Rules . 147 C.4 General BitTorrent UDP . 148 C.5 Vuze UDP . 149 C.6 External UDP Rules . 150 D Snort Rules for Livestation 151 E Snort Rules for TVU Player 153 E.1 TVU Player UDP . ..