Basic level CDT307 Implementation of data collection tools using NetFlow for statistical analysis at the ISP level Bachelor's Thesis in Computer Science by Daniel Karlström Department of Innovation, Design and Engineering Akademin för Innovation, Design och Teknik Mälardalen University Mälardalens Högskola SE-722 18 Västerås, Sweden 722 18 Västerås Implementation of data collection tools using NetFlow for statistical analysis at the ISP level by Daniel Karlström Supervisors: Stefan Löfgren Mälardalen University, IDT Fredrik Holmqvist Internet 2 Business KB Examiner: Mats Björkman Västerås, May 23rd, 2012 Abstract Defending against Dos- and DDoS attacks is difficult to accomplish; finding and filtering out illegitimate traffic from the legitimate flow is near impossible. Taking steps to mitigate or even block the traffic can only be done once the IP addresses of the attackers are known. This is achievable by monitoring the flows to- and from the target and identifying the attacker's IP addresses, allowing the company or their ISP to block the addresses itself by blackholing them (also known as a null route). Using the IP accounting and monitoring tool “pmacct”, this thesis aims to investigate whether or not the pmacct suite is suited for larger installations when tracking and mitigating DDoS-attacks, such at an Internet Service Provider (ISP). Potential problems are the amount of traffic that need to be analyzed and the computational power required to do it. This thesis also provide information about the pmacct suite at large. The conclusions are positive, indicating it does scale up to handle larger installations when given careful consideration and planning. 1 Sammanfattning Att försvara sig mot DoS-och DDoS-attacker är svårt att åstadkomma; att hitta och filtrera ut illegitim trafik från det legitima flödet är nästan omöjligt. Att vidta åtgärder när en sådan attack upptäcks kan endast göras när IP-adresserna från angriparna är kända. Detta kan uppnås genom att man övervakar trafikflödet mellan målet för attacken och angriparna och ser vilka som sänder mest data och på så sätt identifierar angriparna.. Detta tillåter företaget eller dess ISP att blockera trafiken ifrån dessa IP-adresser genom att sända trafiken vidare till ingenstans. Detta kallas blackhole- routing eller null-routing. Genom att använda redovisnings- och övervakningsprogrammet pmacct syftar denna uppsats på att undersöka hurvida pmacct-sviten är lämpad för större installationer när det gäller att spåra och förhindra DDoS-attacker, såsom hos en Internetleverantör eller dylikt. Potentialla problem som kan uppstå är att mängden trafik som måste analyserar blir för stor och för krävande. Denna avhandling går även igenom pmacct-verktyget i sig. Slutsatserna är lovande, vilket indikerar att den har potential av att kunna hantera sådana stora miljöer med noggrann planering. 2 Acknowledgments I would like to thank Fredrik Holmqvist, Internet 2 Business KB, who was my supervisor at the company, and also to thank the company at large for creating the opportunity for me to make this bachelor's thesis. Fredrik, and others at the company, were always been willing to offer a helping hand and provide information when needed. Many thanks as well to my supervisor Stefan Löfgren at the Department of Innovation, Design and Engineering at Mälardalen University. His guidance helped shape this report, vastly improving it into what you now see before you. Last but not least, I would like to reach out and thank Paolo Lucente, the creator of the pmacct suite, upon which this thesis is based. His advice over continuous mail correspondence regarding configuration issues were essential and his eagerness to help people should be recognized. 3 Table of Contents Abstract..............................................................................................................................................1 Sammanfattning................................................................................................................................2 Acknowledgments..............................................................................................................................3 1 Introduction....................................................................................................................................11 1.1 Thesis background...................................................................................................................11 1.2 Related work............................................................................................................................12 1.3 Problem formulation................................................................................................................12 1.4 Purpose.....................................................................................................................................12 1.5 Software used...........................................................................................................................12 1.6 Organization.............................................................................................................................13 1.7 Limitations...............................................................................................................................13 2 Theoretical background................................................................................................................15 2.1 Internet Protocol......................................................................................................................15 2.2 Apache HTTP Server ..............................................................................................................16 2.3 Autonomous Systems...............................................................................................................16 2.4 NetFlow...................................................................................................................................18 2.5 sFlow........................................................................................................................................19 2.6 IPFIX.......................................................................................................................................19 2.7 Linux........................................................................................................................................20 2.7.1 Ubuntu.............................................................................................................................20 2.7.2 Cron.................................................................................................................................20 2.7.3 Scripting..........................................................................................................................21 2.7.4 Libpcap............................................................................................................................22 2.8 Quagga.....................................................................................................................................22 2.9 MRTG......................................................................................................................................23 2.10 pmacct....................................................................................................................................24 2.11 Baselining..............................................................................................................................24 3 Problem analysis............................................................................................................................25 3.1 Collecting the data...................................................................................................................25 3.1.1 Protocol decision.............................................................................................................25 3.1.2 Choice of NetFlow-collector...........................................................................................25 3.1.3 Network configuration....................................................................................................25 3.2 Storing the data .......................................................................................................................25 3.3 Displaying the data .................................................................................................................26 4 Implementation..............................................................................................................................27 4.1 Choice of method.....................................................................................................................27 4.1.1 Collecting the data..........................................................................................................27 4.1.2 Storing the data ..............................................................................................................29 4.1.3 Displaying the data..........................................................................................................29 4.2 Method criticism......................................................................................................................29 4.3 Solution....................................................................................................................................30 4.3.1 Collecting the data