Design and implementation of an SNMP based data collector that will aid in the design of a multi-vendor network monitoring system. UNIVERSITY OF ZIMBABWE DEPARTMENT OF ELECTRICAL ENGINEERING Submitted by TAKAWIRA DZORO (R1713515) DISSERTATION SUBMITTED IN PARTIAL FULFILMENT OF REQUIREMENTS FOR THE MASTER OF SCIENCE DEGREE IN COMMUNICATION ENGINEERING Supervisor: Dr T. Marisa DECLARATION I hereby declare that the dissertation report entitled “Design and implementation of an SNMP based data collector that will aid in the design of a multi-vendor network monitoring system” is a genuine record of the researcher‟s work carried out in partial fulfilment of the requirements for an MSC Communications engineering degree at University of Zimbabwe under the supervision of Dr. Marisa. Student Signature Date Takawira Dzoro Supervisor Declaration It is certified that the above statement made by the student is correct to the best of my knowledge and belief Supervisor Signature Date Dr T. Marisa i Abstract As network operators in Zimbabwe continue to grow their networks every day, network monitoring is also becoming one of their most challenging issues. The operators use network equipment supplied by different vendors and therefore the monitoring tools that they are currently using are also vendor-specific. Due to the large costs associated with monitoring tools and licenses, many network elements are still being monitored manually which becomes practically impossible as networks grow in size and complexity. Besides purchasing monitoring tools from equipment vendors, some of the solutions implemented by administrators locally include the use of free online ping and traceroute tools. While such solutions can work for a small number of nodes in a single geographical location, they fail to scale to bigger and complex network infrastructures. Purchasing monitoring tools require foreign currency and therefore it creates more overheads and expenses to the company resulting in low returns of investments. This research feels the gap and provides an efficient multi-vendor networking system. SNMP protocol was used to build a data acquisition layer of the tool. It is in this layer where fault information is collected from network elements and delivered to the main application for processing, storage, and dashboard display. The data collection layer was designed to query in real-time, the Network Elements‟ online/offline status, interface connection status, Linkup/link-down status, and link performance utilization. A combination of three switches from different vendors (Huawei, ZTE and Cisco) was used to come up with a prototype network for testing and validating the results. ii Acknowledgements I would like to thank my family for both the financial and emotional support they gave to me, especially my wife Nyaradzo Dzoro for always being there for me. Thank you Dr. Marisa for the guidance and support as my research supervisor. I am also grateful to the lectures in the University of Zimbabwe Faculty of engineering department for the basic theory they shared with me during taught courses which was very essential in implementing my research. A big thank to the University of Zimbabwe institution as a whole for providing the best research environment. Above all, sincere gratitude to the Almighty God for good health and strength to push through all the obstacles. iii Abbreviations API Application Programming Interface CMIP Common Management Information Protocol FCAPS Fault, Configuration, Accounting, Performance, and Security. HTTP Hypertext Transfer Protocol ICMP Internet Control Message Protocol IP Internet Protocol ISO International Organization for Standardization MIB Management Information base NE Network Element NMS Network Monitoring System NNTP Network News Transfer Protocol OID Object Identifier PDH Plesiochronous Digital Hierarchy POP3 Post Office Protocol RAM Random Access Memory RDBMS Relational database management systems RFC Request For Comments RRD Round Robin databases SDH Synchronous Digital Hierarchy SLA Service Level Agreement SMS Short Message Service SMTP Simple Mail Transfer Protocol SNMP Simple Network management Protocol SSH Secure Shell SQL Structured Query Language TCP Transmission Control Protocol UDP User Datagram Protocol iv Listof Figures and Tables Figure 3.1: Net-SNMP installation procedure .............................................................................. 22 Figure 3.2: Net-SNMP installation procedure .............................................................................. 23 Table 3.0: Test snmp agent details ................................................................................................ 23 Figure 3.3: snmptranslate example ............................................................................................... 24 Figure 3.5: snmpgetnext example ................................................................................................. 24 Figure 3.6: snmpbulkget example ................................................................................................. 25 Figure 3.7: snmpstatus example .................................................................................................... 25 Figure 3.8: snmpset example ........................................................................................................ 25 Figure 3.9: snmptrap.config .......................................................................................................... 27 Figure 3.10: snmptrapd.py ............................................................................................................ 28 Figure 3.12: Script to start snmptrap daemon ............................................................................... 29 Table 3.1: snmp-cmds Python Library .......................................................................................... 30 Table 3.2: SNMP manager modules ............................................................................................. 34 Figure 3.13: Wireshark testing sequence ...................................................................................... 37 Figure 3.14: snmpget request trace ............................................................................................... 38 Figure 3.15: snmp get response trace ............................................................................................ 38 Figure 3.16: SNMP connectivity troubleshooting sequence ......................................................... 39 Figure 3.17 : Celery beat schedule ................................................................................................ 40 Figure 3.18: Heartbeat implementation syntax ............................................................................. 41 Figure 3.19: Heart implementation syntax.................................................................................... 42 Figure 3.20: Device update flowchart ........................................................................................... 43 Figure 3.21 Performance instance Creation .................................................................................. 44 v Figure 3.22: Performance data collection ..................................................................................... 45 Figure 3.23: Performance data collection code ............................................................................. 46 Figure 3.24: Performance collection task for Celery .................................................................... 46 Figure 3.25: Device information update flow chart ...................................................................... 47 Figure 3.26: device information update implementation .............................................................. 48 Figure 3.27: Trap handling flowchart ........................................................................................... 49 Figure 3.28 raw varbinds from SNMPTRAPD............................................................................. 50 Figure 3.29: Numeric format of trap dict after parsing ................................................................. 50 Figure 3.30: Textual format of trap dict after parsing ................................................................... 51 Figure 3.31: System architecture .................................................................................................. 52 Figure 3.32: Network prototype .................................................................................................... 53 Figure: 3.33: Network prototype photo ........................................................................................ 54 Figure 4.1: Device Offiline Status Wireshark trace ...................................................................... 55 Figure 4.2: Device Online Status Wireshark trace ........................................................................ 56 Figure 4.3 Device Offline Status .................................................................................................. 57 Figure 4.4 Device Online Status ................................................................................................... 57 Figure 4.5 Device Linkup Status Wireshark trace ........................................................................ 58 Figure 4.6: Device Linkup status .................................................................................................. 58 Figure 4.7 Device Link down Status Wireshark trace .................................................................