A Study on the Cacti-Based Network Traffic Monitoring System Using Libpcap

M.E Thesis

A Study on the Cacti-based Network Traffic Monitoring System Using Libpcap

December, 2011

Graduate School of the PaiChai University Department of Computer Engineering

Major of Multimedia Information Engineering

Xiao Huang

A Study on the Cacti-based Network Traffic Monitoring System Using Libpcap

Principal Advisor Hoe-Kyung Jung

December, 2011

Graduate School of PaiChai University Department of Computer Engineering

Major of Multimedia Information Engineering

Xiao Huang The undersigned have examined the dissertation entitled: “A Study on the Cacti-based Network Traffic Monitoring System Using Libpcap” presented by “Xiao Huang” , a candidate for the degree of Master of Computer Engineering and hereby certify that in their judgment it is worthy of acceptance.

December, 2011

In-June Jo, PH.D

Professor

Department of Computer Engineering

Graduate School of PaiChai University

December, 2011

Dong-Kun Noh, PH.D Professor Department of Computer Engineering Graduate School of PaiChai University

December, 2011

Hoe-Kyung Jung, PH.D Professor Department of Computer Engineering Graduate School of PaiChai University

A Study on the Cacti-based Network Traffic Monitoring System Using Libpcap

Xiao Huang

Department of Computer Engineering Graduate from PaiChai University Daejeon, Korea (Supervised by Professor Hoe-Kyung Jung)

ABSTRACT

Networks are growing rapidly in size and make the networks more complex than before. In LAN(Local Area Network), network congestion, slow-speed and the servers is often attacked and even be paralyzed. In order to keeping our networks quick, reliable, secure and efficient, we need a network traffic monitoring.

Recently, a smart phone can be bound with an email address. We can receive email at hand immediately. Cacti is an open source and freeware monitoring tool and has lots of alarming email plugins.

Libpcap is also an open source library that provides a low level interface to network packet capture systems. Making a Sniffer by

Libpcap to capture packets from NIC(Network Interface Card) and

analyze those packets and to store in DB. Cacti get those data by Perl scripts and use these data to graph and sent emails in special case.

In this thesis, we unite them together and design a system to monitor our network traffic in real time. Executing these programming, our system can get a few choppy continuous graphs and log files and even receive alarming emails by mobile phone.

These results indicate that it is possible to unit Cacti and Libpcap together to monitor our network traffic and this system can achieve our desired goal and it is effective, quick, accurate and real-time.

II Contents Abstract...... I Contents...... III Figures Listed...... V Table Listed...... VI I. Introduction ...... 1 II. Related Work ...... 4 2.1 RRDTool ...... 4

2.1.1 the Concept of RRDTool ...... 4

2.1.2 the Features of RRDTool ...... 4

2.2 Cacti ...... 5

2.2.1 the Concept of Cacti ...... 5

2.2.2 the Features of Cacti ...... 6

2.2.3 Work Principle ...... 6

2.2.4 How to Use Cacti ...... 7

2.2.5 Add PIA (Plugin Architecture) ...... 9

2.2.6 Installing Setting Plugin and Thold Plugin ...... 10

2.3 Libpcap ...... 12

2.3.1 the Concept of Libpcap ...... 12

2.4 Sniffer ...... 13

2.4.1 the Concept of Sniffer ...... 13

2.4.2 Work Principle ...... 13

2.4.3 How to Make a Sniffer ...... 15

2.4.4 Things should be taken into account ...... 18

2.4.4.1 Data Link Type ...... 18

III

2.4.4.2 Network Layer Protocol ...... 20

2.4.4.3 Transport Layer Protocol ...... 20

2.4.4.4 Application Layer Protocol ...... 21

2.4.4.5 Filtering Packets ...... 21

2.4.4.6 Setting a Filter ...... 22 III. Design System ...... 26 3.1 Development Environment ...... 26

3.2 System Structure ...... 27

3.2.1 Capture Packets Block ...... 27

3.2.1.1 Design Ethernet Networks Structure ...... 28

3.2.1.2 Design a Callback Function ...... 30

3.2.1.3 Start a Sniffer Application ...... 32 3.2.2 Counter Block...... 32

3.2.3 Inquiry Block ...... 34

3.2.4 Connect with Cacti Block ...... 35

3.2.4.1 Design a Program to Update a DB table ...... 35

3.2.4.2 Design a Perl Programming ...... 36

3.2.4.3 Create a graph from a Perl Script ...... 37 3.2.5 Set Alarm Block ...... 38

3.2.5.1 Set Threshold Plugins ...... 38

3.2.5.2 Creating a Threshold Templates ...... 40 3.3 Results ...... 42

IV. Result and Analysis ...... 45 4.1 Make a Test ...... 45

4.1.1 Test by TCP ...... 45

4.1.2 TEST by ICMP ...... 48

IV 4.2 Analysis ...... 52

V. Conclusions and Future Works ...... 53 5.1 Conclusions ...... 53

5.2 Future Works ...... 54

References ...... 55 Acknowledgement ...... 57

Figures Listed

Figure 1. Cacti Work Principle ...... 7

Figure 2. Cacti Login Window ...... 8

Figure 3. Thold Plugin ...... 11

Figure 4. Setting Plugin ...... 12

Figure 5. Elements Involved in the Capture Process ...... 15

Figure 6. System Design Flow-Process Diagram ...... 26

Figure 7. Structure of System ...... 27

Figure 8. Data Encapsulation in Ethernet Network ...... 28

Figure 9. Callback Function ...... 30

Figure 10. Start Grabbing Packets ...... 32

Figure 11. Information Recorded in File ...... 33

Figure 12. the Structure of Table ...... 36

Figure 13. Perl Connect to DB ...... 37

Figure 14. Create Graph for ARP ...... 38

Figure 15. Default Alerting Options ...... 39

Figure 16. Emailing Options ...... 39

Figure 17. Cacti Setting(Mail/DNS) ...... 39

Figure 18. SMTP Options ...... 39

Figure 19. Test Message ...... 40

Figure 20. Receive a Email by a Phone ...... 40

Figure 21. Setting Threshold Template ...... 41

Figure 22. High/Low Settings ...... 41

Figure 23. Other Setting ...... 41

Figure 24. Threshold to ARP and UDP ...... 42

VI Figure 25. Email for UDP ...... 42

Figure 26. the First Graph ...... 43

Figure 27. the Data in DB Table ...... 43

Figure 28. the Content of Log File ...... 43

Figure 29. Graph a few Hours Later ...... 44

Figure 30. Update the DB Table by TCP ...... 46

Figure 31. Graph by PuTTY Real Time ...... 47

Figure 32. the Content of Log File by TCP ...... 47

Figure 33. a Alarming Email for TCP ...... 47

Figure 34. Graph for TCP by PuTTY ...... 48

Figure 35. Window PING Messages ...... 49

Figure 36. Update the DB Table by PING ...... 49

Figure 37. Graph by PING Real Time ...... 50

Figure 38. the Content of Log File by PING ...... 50

Figure 39. an Alarm Email for ICMP ...... 50

Figure 40. Graph for ICMP by PING ...... 51

VII

Tables Listed

Table 1. Installing PIA ...... 9

Table 2. Import PIA Data ...... 10

Table 3. Install Setting and Thold Plugins ...... 10

Table 4. Configure PIA ...... 11

Table 5. Function of Looking Device and Opening Device ...... 16

Table 6. Function of Capture Packets ...... 17

Table 7. Define struct pcap_pkthdr ...... 18

Table 8. Function of pcap_close ...... 18

Table 9. Common Data Link Types ...... 19

Table 10. Network Layer Protocol and Ethertype Values ...... 20

Table 11. Transport Layer Protocol ...... 21

Table 12. Function of Setting Filter...... 24

Table 13. Define struct bpf_program ...... 25

Table 14. System Design Environment ...... 26

Table 15. Define struct IP ...... 29

Table 16 Define struct ARP ...... 29

Table 17. the Source of Callback Function ...... 31

Table 18. the Source of Counter Shell ...... 34

Table 19. the Source of Updating Log File shell ...... 35

Table 20. the Source of Connecting to DB by Perl ...... 36

Table 21. TCP/UDP Ports ...... 46

Table 22. ICMP Type ...... 48

VIII I. Introduction

Network traffic monitoring is an important component of network management. With the development of computer network application technology, BT(BitTorrent) downloads, video streaming, IP(Internet

Protocol) phones, P2P(Peer to Peer) point transmission, multicast networks, web online games, multimedia, online video and other applications business is coming out, and also the scale of network is growing rapidly, the number of network data transmission is also a large-scale increasing in size[1]. So, in LAN(Local Area Network), network congestion, network slow-speed often appear. And also, at recently, the server or personal host is often attacked. Lots of important data, information was lost or stolen, even the servers were paralyzed. Network can give us the gospel, and also bring disaster to us.

Therefore, nowadays, network traffic monitoring is necessary in complex network environment; it has great significance for improving network management ability. The anomaly network traffic is network communication failure as the primary performance, and also be a harbinger of a network transmission failures. So it’s necessary to observe and analyze the change of network traffic in time. The potential network problems can be found, and to take preventive measures to expel the coming network problems in time.

Recently, open source and freeware are very popular. Cacti is a great trend monitoring tool and also an open source and freeware. It can graph clearly with collected data. Compare to other monitoring tools, it uses friendly and easily; it can automatically produce daily/weekly/monthly/yearly performance graphs and reports of many important metrics; it also can graph with RRDTool(Round Robin

Database Tool) which collects data by SNMP(Simple Network

Management Protocol), Shell or Perl Scripts etc; Cacti template also can be shared with other users which can save a lot of time; it can easily add plugins to the Cacti to enable the possibility to integrate other free tools. And it stores collected data with RRDs(Round Robin

Databases) which never increases[2,3].

Libpcap(Library Packet Capture) is also an open source library that provides a low level interface to network packet capture systems[4].

Using Cacti and Libpcap, A real-time system of monitoring our network traffic would be designed. Preventing our network from attacking and finding the errors and keeping network environment healthy, fast and stable is the purpose of designing this system.

As a whole, it will contain 5 parts: collecting data; analyzing data; fetching data; graphing with data and the alarm. The first part is making a programming which has the ability to intercept the traffic that pass over a network; the second part is making a Shell programming to analyze the file and count the key words and record these key words in a header file which is used in other programming; the third is making a

2 script to fetch these data from DB; the fourth is making a Cacti template for Cacti displaying; at last, add email plugin to alarm.

Cacti, Libpcap, Perl script, Shell would be studied. It will involve creating Cacti template and add plugins, making a Sniffer, connecting to DB by Perl, and analyzing a file by Shell.

Starting this system, wavy graphs can be graph by Cacti with captured packets; lots of records can be found in log file; emails can be received in abnormal case. It proved that this system is a real-time, friendly-use and easily-use system, and it is possible, effective and accurate to monitor our network traffic.

The organization of this thesis is as follows. Part II will show you the concept of Cacti and the feature of Cacti and how to create a Cacti template for a Perl script, how to add a plugin, and the concept of

Libpcap and Sniffer and how to make a simple Sniffer; Part III will show you how to design and make this system; Part IV will show you how to analyze our test results; conclusion and future works will be discussed in Part V.

II. Related Work

In this part, it will let you know some general information about

RRDTool, Cacti, add PAI(Plugin Architecture Installation) and Sniffer; for example, the concepts, the usages, and the guidelines and so on.

2.1 RRDTool

In this section, it will show you what RRDTool is and the features of RRDTool.

2.1.1 the Concept of RRDTool

RRDtool is a program developed by the Swiss Tobi Oeticker who was already the creator of the famous MRTG(Multi Router Traffic

Grapher) which started as a tiny script for graphing. RRDTool is the open source industry standard, high performance data logging and graphing system for time series data. RRDTool can be easily integrated in shell scripts, Perl, Pytnon, Ruby, Lua or Tcl applications[5].

2.1.2 the Features of RRDTool

Round robin is a technique that works with a fixed amount of data and a pointer to the current element. Think of a circle with some dots plotted on the edge. These dots are the places where data can be stored. Draw an arrow from the center of the circle to one of the dots; this is the pointer. When the current data is read or written, the pointer moves to the next element. As we are on a circle there is neither a

4 beginning nor an end, you can go on and on and on. After a while, all the available places will be used and the process automatically reuses old locations. This way, the dataset will not grow in size and therefore requires no maintenance. RRDtool works with Round Robin Databases

(RRDs). It stores and retrieves data from them[6,7].

RRDtool lets you create a database, store data in it, retrieve that data and create graphs in PNG format for display on a web browser. By default, you can have daily, weekly, monthly and yearly graphs.

2.2 Cacti

In this section, it will show you Cacti’s concept, the Cacti‘s feature, the Cacti’s work principle, how to use Cacti to create a graph for local host form a Perl script, and how to add PIA.

2.2.1 the Concept of Cacti

Cacti is a complete network graphing solution designed to harness the power of RRDTool’s data storage and graphing functionality. Cacti provide a fast poller, advanced graph template, multiple data acquisition methods, and user management features out of the box. All of this is wrapped in an intuitive, easy to use interface that makes sense for LAN-sized installations up to complex networks with hundreds of devices[2].

Cacti is a complete frontend to RRDTool, it stores all of the necessary information to create graphs and populate them with data in

a MySQL database. The frontend is completely PHP driven. Along with being able to maintain Graphs, Data Sources, and Round Robin

Archives in a database, Cacti handle the data gathering[3,8].

2.2.2 the Features of Cacti

Cacti is an open source and freeware monitoring tool. It is a web based PHP/MySQL graphing solution using the RRDTool engine.

It can graph with data which collect by SNMP, Shell or Perl scripts etc. Cacti’s strength lies in the fact that is can be installed and used incredibly easily. You don’t need to be a guru or spend tons of hours on the tool to configure it. Even a beginner can use it very quickly. It is used friendly and allows you to resize your graphs and view data for an arbitrary range. On the very active Cacti forum, Cacti templates can be shared with other users which can save you a lot of time. You can very easily add plugins to the Cacti to enable the possibility to integrate other free tools.

It can automatically produce daily/weekly/monthly/yearly performance graphs and reports of many important metrics.

2.2.3 Work Principle

The work principle of Cacti could be roughly divided into 6 steps[9]. Figure 1 shows the work principle of Cacti. First, collecting data by SNMP, or Perl scripts or Shell; second, storing the collected data in RRDTool and update data and record the device file name corresponding to the rra which is a folder stored RRD file in DB; third, users view the graph of a device; fourth, Cacti looks up the device

6 name in DB; fifth, Cacti uses the file name order the RRDTool graph; sixth, Cacti returns the graph to the user.

Figure 1. Cacti Work Principle

2.2.4 How to Use Cacti

For details about how to install Cacti, please look up the “Cacti

Manual”[10]. It shows you how to install Cacti in Linux environment and in Windows environment etc.

If the Cacti were installed well, you can access Cacti at http://localhost/Cacti from the local computer or from any computer within your LAN network at http://your.server.IP.address/Cacti.

Figure 2. Cacti Login Window

Figure 2 is a screenshot of the login window. The default user name is admin. The default password is admin. It should prompt an automatic password change for the admin account when you log in the first time.

In common, the first step to create graphs for a device is to create a device. Here, local host is considered as the monitored device, then to create a graph for this device, and then to add graph to graph view, at last, to view the graph.

In the following, it shows you how to create a graph from a Perl script which is prepared for this system. If you want to create a graph from a Perl script, you should do it as 3 steps. First, create a data input method to tell Cacti how to call the script and what to expect from it; second, create a data source to tell Cacti how and where the data will be stored; third, create a graph to tell Cacti how the data will be presented in graph form.

8 2.2.5 Add PIA (Plugin Architecture)

The PIA is a set of code changes to core cacti. The Plugin

Architecture for Cacti was designed to be both simple in nature and robust enough to allow freedom to do almost anything in Cacti. Cacti itself is designed nicely enough that integrating into it is fairly easy with very little modifications necessary. Eventually Cacti will come with a standard plugin architecture that will allow you to create addons without the need to modify your installation, but until that time comes

(we are working on it) you will need to follow the directions below.

The first step is to download the plugin architecture. You can get it in either zip or gzip compressed archives; then do it as by Table 1.

Table 1. Installing PIA

tar -zxvf cacti-plugin-0.8.7g-PA-v2.8.tar.gz

cd cacti-plugin-arch

cp cacti-plugin-0.8.7g-PA-v2.8.diff /home/www/html/

cp pa.sql /home/www/html

cd /home/www/html

patch -p1 -N < cacti-plugin-0.8.7g-PA-v2.8.diff

The second step is to import into database. You can do it followed by

Table 2.

Table 2. Import PIA Data mysql -uroot–p*** cacti< /home/www/html/pa.sql

2.2.6 Installing Setting Plugin and Thold Plugin

Setting plugin provides common infrastructure plugin services for

Cacti’s plugin Architecture and thold plugin is for alerting of data found within any graph within Cacti. Before installing a plugin, you need to install the Cacti Plugin Architecture. The first step is to download the plugin of setting. Download them at http://cactiusers.org/downloads, you can get it in either zip or gzip compressed archives; then do it followed by Table 3.

Table 3. Install Setting and Thold Plugins tar zxvf thold-0.4.3.tar.gz mv thold /home/www/html/plugins mkdir /home/www/html/plugins/settings mv settings-0.5.zip /home/www/html/plugins/settings unzip /home/www/html/plugins/settings/settings-0.5.zip mysql -uroot –p*** cacti < /home/www/html/plugins/tholdr/thold.sql

If all goes well, you should have two folders called setting, thold with a few files and possibly a few folders in it. And then the file global.php which locates in var/www/html/include was needed to

10 modify. Put “$plugins[]=’settings’;”and “$plugins[]=’thold’;” to below

$plugins=array() like Table 4.

Table 4. Configure PIA

$database_type = "mysql";

$database_default = "cacti";

$database_hostname = "localhost";

$database_username = "cactiuser";

$database_password = "cactiuser";

$database_port = "3306";

...

$plugins = array();

$plugins[] = 'settings';

$plugins[] = 'thold'; ...

Open the our Cacti main page, click “console” – “plugin management”, and click the “install” button to install them, at last, you can see the main page will appear one more button named “thold” like

Figure 3. The “Mail/DNS” will appear in “console” – “setting” menu.

The Figure 4 shows the tag “Mail/DNS”.

Figure 3. Thold Plugin

Figure 4. Setting Plugin

2.3 Libpcap

In this part, it will show you the concept of Libpcap.

2.3.1 the Concept of Libpcap

Libpcap, an open source and portable packet capture library, is a system-independent interface for user-level packet capture. It was created in 1994 by McCanne, Leres and Jacobson-researchers at

Lawrence Berkeley National Laboratory from the University of

California. It provides a portable framework for low-level network monitoring. Its applications include network statistics collection, security monitoring, network debugging, etc.

Libpcap authors’ main objective was to create a platform-independent API to eliminate the need for system-dependent packet capture modules in each application.

The Libpcap API is designed to be used from C and C++. However, there are many wrappers that allow its use from language like Perl,

Python, Jave, C# or Ruby. Libpcap runs on most UNIX-like operating system (Linux, Solaris, BSD, HPUX…). There is also a windows version named Winpcap. Today, Libpcap is maintained by the Tcpdump

Group[11,12].

12 2.4 Sniffer

In this section, it will show you the concept of Sniffer, work principle, and how to make a simple Sniffer for this system.

2.4.1 the Concept of Sniffer

Since the first message was sent over the ARPANET in 1969, computer networks have changed a great deal. Back then, networks were small and problems were solved using simple diagnostic tools. As these networks got more complex, the need for management and troubleshooting increased. Nowadays, computer networks are usually large and diverse systems that communicate using a wide variety of protocols. This complexity created the need for more sophisticated tools to monitor and troubleshoot network traffic. Today, one of the critical tools in any network administrator tool-box is the Sniffer.

Sniffer, also known as packet analyzers, are programs that have the ability to intercept the traffic that have the ability to intercept the traffic that have the ability to intercept the traffic that passes over a network. They are very popular between network administrators and the black hat community because they can be used for both-good and evil[11].

2.4.2 Work Principle

In the following, main principles of packet capture will be gone through. Packets capture is the action of collecting data as it traces

over a network. Every time a network card receives an Ethernet frame it checks that its destination MAC(Media Access Control) address matches its own. If it does, it generates an interrupt request. The routine in charge of handling the interrupt is the system’s network card driver. The driver timestamps received data and copies it from the card buffer to a block of memory in kernel space. Then, it determines which type of packet has been received looking at the ether-type field of the appropriate protocol handler in the protocol stack. In most cases the frame will contain an IPV4(Internet Protocol Version 4) datagram so the IPV4 packet handler will be called. This handler performs a number of checks to ensure, for examples, that the packet is not corrupt and that is actually destined for this host. If all tests are passed, the IP headers are removed and the remainder is passed to the next protocol handler (probably TCP(Transmission Control Protocol) or UDP(User

Datagram Protocol)). This process is repeated until the data gets to the application layer where it is processed by the user-level application[13,14].

When a Sniffer is used, packets go through the same process described above but with one difference: the network driver also sends a copy of any received or transmitted packet to a part of the kernel called the packet filter. Packet filters are what makes packet capture possible[11]. Figure 5 illustrates the capture process.

Figure 5. Elements Involved in the Capture Process

2.4.3 How to Make a Sniffer

In order to making a Sniffer application, we must use Libpcap functions as below followed.

The first thing needed is getting a network interface device to listen on the network traffics. The functions pcap_lookupdev() would be used. This function would return a pointer to a string containing the name the first network device that is suitable for packet capture[15].

The second thing is opening the device. The function pcap_open_live() would be used[15]. This function would return an interface handler of type pcap_t that will be used later when calling the rest of the functions provided by Libpcap.

Table 5 lists the arguments of pcap_lookupdev() and pcap_open_live().

Table 5. Function of Looking Device and Opening Device

Functions Arguments Notes pcap_lookupdev( ) char *errbuf argument errbuf: Store an error message in case something goes wrong pcap_open_live( ) const char*device, The first argument: a string int snaplen, containing the name of the int promisc, network interface int to_ms, The second: the maximum char *errbuf number of bytes to capture The third: flag decides whether be in promiscuous mode or not The fourth: how many milliseconds should the kernel wait before coping the captured information from kernel space to user space The fitth: the same as the argument of function pcap_lookupdev

The third thing is to start getting packets. For this, there are 3 options to be chosen. The function pcap_next(), pcap_loop(), and pcap_dispatch() would be used[15]. The function pcap_next() would return the first packet that arrives to the network interface card. The function pcap_loop() would be used to collect packets and process them by the callback function which is used to get information from captured packets and defined by user.

16 Table 6. Function of Capture Packets

Functions Arguments Notes pcap_next( ) pcap_t *p, The first argument: struct pcap_pkthdr *h argument p: returned by function pcap_open_live The second: a pointer to a structure pcap_loop( ) pcap_t *p, The first: the same as the unt cnt, above pcap_handler callback, The second: the number u_char *user of captured packets, a negative cnt value would cause it to return only in case of error The third: user define The fourth: set as NULL pcap_dispatch( ) pcap_t *p, The arguments is the int cnt, same as the function pcap_handler callback, pcap_loop(), but when the u_char *user argument to_ms of pcap_open_live() goes out, pcap_dispath() will return at once

The function pcap_dispatch() would is similar to pcap_loop(), but it returns when the to_ms timeout specified in pacp_open_live() elapses.

Table 6 lists the argument of capture packet functions.

Table 7 shows the struct pcap_pkthdr. The argument caplen has usually the same value as argument len except the situation when the size of the captured packet exceeds the snaplen specified in function pcap_open_live().

Table 7. Define struct pcap_pkthdr

struct pcap_pkthdr { struct timeval ts; //timestamp of capture bpf_u int32 caplen; //number of bytes that were stored bpf_u_int32 len; // total length of the packet };

The last thing is to close the pcap handle and release all its resources held. The function pcap_close can do it. Table 8 lists the argument of function of pcap_close.

Table 8. Function of pcap_close

Functions Arguments Notes pcap_close pcap_t *p argument p: returned by function pcap_open_live

2.4.4 Things should be taken into account

When a packet is captured, the only thing that our application has got is a bunch of bytes. Usually, the network interface card driver and the protocol stack process that data for us but when we are capturing packets form our own application we do it at the lowest level so we are the ones in charge of making the data rational. To do that there are some things that should be taken into account.

2.4.4.1 Data Link Type

Although Ethernet seems to be present everywhere, there are a lot of different technologies and standards that operate at the data link

18 layer. In order to be able to decode packets captured from a network interface we must know the underlying data link type so we are able to interpret the headers used in that layer.

The function pcap_datalink() returns the link layer type of the device opened by pcap_open_live(). Libpcap is able to distinguish over

180 different link types. However, it is the responsibility of the user to know the specific details of any particular technology. This means that we, as programmers, must know the exact format of the data link headers that the captured packets will have. In most applications we would just want to know the length of the header so we know where the

IP datagram starts.

Table 9. Common Data Link Types

Data Link Type Pcap Alias Offset(in bytes)

Ethernet 10/100/1000Mbs DLT_EN10MB 14

Wi-Fi 802.11 DLT_IEEE802_11 22 FDDI(Fiber Distributed Data DLT_FFDI 21 Interface) PPPoE(PPP over Ethernet) DLT_PPP_ETHER 20 BSD Loopback DLT_NULL 4

Table 9 summarizes the most common data link types, their names in Libpcap and the offsets that should be applied to the start of the captured data to get the next protocol header.

2.4.4.2 Network Layer Protocol

The next step is to determine what follows the data link layer header. The Ethernet header has a 16-bit field named ethertype which specifies the protocol that comes next. Table 10 lists the most popular network layer protocols and their ethertype value.

Table 10. Network Layer Protocol and Ethertype Values

Network Layer Protocol Ethertype Value

Internet Protocol Version 4(IPv4) 0x0800

Internet Protocol Version 6(IPv6) 0x86DD

Address Resolution Protocol(APP) 0x0806

Reverse Address Resolution Protocol(RAPP) 0x8035

AppleTalk over Ethernet(EtherTalk) 0x809B

Point-to-Point Protocol(PPP) 0x880B

PPPoE Discovery Stage 0x8863

PPPoE Session Stage 0x8864

Simple Network Management Protocol(SNMP) 0x814C

2.4.4.3 Transport Layer Protocol

Once we know which network layer protocol was used to route our captured packet we have to find out which protocol comes next. Table

11 summarizes the most common transport layer protocols, their hexadecimal value and the RFC document.

20 Table 11. Transport Layer Protocol

Protocol Value RFC

Internet Control Message Protocol(ICMP) 0x01 RFC 792 Internet Group Management Protocol(IGMP) 0x02 RFC 3376

Transmission Control Protocol(TCP) 0x06 RFC 793 Exterior Gateway Protocol 0x08 RFC 888 User Datagram Protocol(UDP) 0x11 RFC 768 IPv6 Routing Header 0x2B RFC 1883

IPv6 Fragment Header 0x2C RFC 1883 ICMP for IPv6 0x2A RFC 1883

2.4.4.4 Application Layer Protocol

Assuming that we have got the Ethernet header, IP header, the

TCP header, and then we can get the TCP port numbers from it.

2.4.4.5 Filtering Packets

As we saw before, the capture process takes place in the kernel while our application runs at user lever. When the kernel gets a packet from the network interface it has to copy it from kernel space to user space, consuming a significant amount of CPU time. Capturing everything that flows past the network card could easily degrade the overall performance of our host and cause the kernel to drop packets.

If we really need to capture all traffic, then there is little we can do to optimize the capture process, but if we are only interested in a specific type of packets we can tell the kernel to filter the incoming traffic so we just get a copy of the packets that match a filter expression. The part of the kernel that provides this functionality is the system’s packet filter.

A packet filter is basically a user defined routine that is called by the network card driver for every packet that is gets. If the routine validates the packet, it is delivered to our application; otherwise it is only passed to the protocol stack for the usual processing.

Every operating system implements its own packet filtering mechanisms. However, many of them are based on the same architecture, the BSD Packet Filter or BPF. Libpcap provides complete support for BPF based packet filters.

2.4.4.6 Setting a Filter

Setting a filter involves three steps: constructing the filter expression, compiling the expression into a BPF (Berkeley Packet Fliter) program and finally applying the filter.

BPF programs are written in a special language similar to assembly. However Libcap and tcpdump implement a high level language that lets us define filters in a much easier way. For example,

“src host 203.250.143.158”, it returns packets whose soruce IP address is 203.250.143.158. The full specification can be found in the manual page for tcpdump.

Once we have the filter expression we have to translate it into something the kernel can understand a BPF program. The function int

22 pcap_compile compile the filter expression pointed by str into BPF code. Once we have a compiled BPF program we have to insert it into the kernel calling the function int pcap_setfilter().

Table 12. Function of Setting Filter

Functions Arguments Notes pcap_lookupnet const char *device, the first argument: pointer bpf_u_int32 *netp, to network decive bpf_u_int32 *maskp, the second, third: point to char *errbuf the network interface address and netmask the fourth: stored an error pcap_complipe pcap_t *p, the first: returned by struct bpf_program *fp, function pcap_open_live() char *str, the second: pointer to int optimize, struct bpf_program bpf_u_int32 netmask the third: pointer to tcpdump expression the fourth: optimized for efficiency or not the fifth: returned by function pcap_lookupnet pcap_setfilter pcap_t *p, the first: returned by struct bpf_program *fp function pcap_open_live() the second: pointer to struct bpf_program

Table 12 lists the arguments of function of setting filter. And Table

13 lists the struct bpf_program[16].

24 Table 13. Define struct bpf_program struct bpf_program{ UINT bf_len; //indicates the number of instructions // of the program struct bpf_insn *bf_insns; //a pointer to the first instruction of // the program } struct bpf_insn{ USHORT code; //instruction type and addressing mode UCHAR jt; //jump if true UCHAR jf; // jump if false int k; // generic field used for various purposes }

If everything goes well we can call pcap_loop() or pcap_next() and start grabbing packets.

III. Design System

3.1 Development Environment

Figure 6 shows system flow diagram. This system’s development environment and used program languages are listed on Table 14.

Figure 6. System Design Flow-Process Diagram

Table 14. System Design Environment

Development Note

OS CentOS 5.4 Development language C, Perl, Shell or script Required software Cacti 0.8.7.C, PHP 5.1.6, MySQL 5.0.82SP1, RRDTool 1.2.23, Apache 2.0.63

26 3.2 System Structure

In this part, there are 5 blocks to be discussed. One is to capture packets block; one is to counter block; one is to inquiry block; one is to connect with Cacti block and the last is to set alarm block. Figure 7 shows the structure of system.

Figure 7. Structure of System

3.2.1 Capture Packets Block

In this section, it will show you how to make a simple sniffer for our system. If we want to make a sniffer application, we should do it as following 5 steps for this system. First, we should get a device. Second, once we get a device, we should open it. Third, set the data link type as ether type. Fourth, once we open it, we should tell Libpcap that we want to start to capture the packets. Fifth, once we get a packet, we should analyze this captured packet by callback function. Though

Sniffer application gets lots of data, here, it is regarded as

“UNKNOWN“ type of packets, so it is not necessary to set filter.

3.2.1.1 Design Ethernet Networks Structure

In this application, we only choose IP header, ARP header as consideration. If the packet contains IP header or ARP header, our programming can read the packet and get the packet’s source address and destination address. And for others, it just gets the type of packets or regarded as “UNKNOWN” type of packets. Figure 8 shows the

Ethernet type’s structure in this system. Table 15 defines struct IP and

Table 16 defines struct ARP.

Figure 8. Data Encapsulation in Ethernet Network

28 Table 15. Define struct IP struct ip{ #if __BYTE_ORDER == __LITTLE_ENDIAN unsigned int ip_hl:4; /* header length */ unsigned int ip_v:4; /* version */ #endif #if __BYTE_ORDER == __BIG_ENDIAN unsigned int ip_v:4; /* version */ unsigned int ip_hl:4; /* header length */ #endif u_int8_t ip_tos; /* type of service */ u_short ip_len; /* total length */ u_short ip_id; /* identification */ u_short ip_off; /* fragment offset field */ #define IP_RF 0x8000 /* reserved fragment flag */ #define IP_DF 0x4000 /* dont fragment flag */ #define IP_MF 0x2000 /* more fragments flag */ #define IP_OFFMASK 0x1fff /* mask for fragmenting bits */ u_int8_t ip_ttl; /* time to live */ u_int8_t ip_p; /* protocol */ u_short ip_sum; /* checksum */ struct in_addr ip_src, ip_dst; /* source and dest address */ };

Table 16 Define struct ARP struct ether_arp { struct arphdr ea_hdr; /* fixed-size header */ u_int8_t arp_sha[ETH_ALEN]; /* sender hardware address */ u_int8_t arp_spa[4]; /* sender protocol address */ u_int8_t arp_tha[ETH_ALEN]; /* target hardware address */ u_int8_t arp_tpa[4]; /* target protocol address */ };

3.2.1.2 Design a Callback Function

Once our programming gets a packet and want to get information from it, we should design a function to handle the packet by itself, we call this function callback function. Callback function is designed like this: once get a packet, and then make a judgment whether it was IP packet, or ARP packet, or RARP packet, or SNMP packet, or

UNKNOWN which the left is regarded as; if it is a IP packet, we made a further judgment whether it contained TCP header, or UDP header, or ICMP header, or IGMP header, or UNKNOWN which the left is regard as. Figure 9 shows call back function processing packets. Table

17 shows a part of source for callback function.

Figure 9. Callback Function

30 Table 17. the Source of Callback Function if(ntohs(eth->ether_type)==ETHERTYPE_IP){ ip=(struct ip *)(p+sizeof(struct ether_header )); printf("%s->",ip_to_char((u_char *)&(ip->ip_src))); printf("%s",ip_to_char((u_char *)&(ip->ip_dst))); switch(ip->ip_p) { case IPPROTO_TCP: tcp=(struct tcphdr *)(p+sizeof(struct ether_header )+ip->ip_hl*4); printf(" tcp port :% 6u\n",ntohs(tcp->source)); break; case IPPROTO_UDP: udp=(struct udphdr *)(p+sizeof(struct ether_header )+ip->ip_hl*4); printf(" udp port:%6u\n",ntohs(udp->source)); break; . . . default: printf(" unknow\n"); break; } } else if(ntohs(eth->ether_type)==ETHERTYPE_ARP) { arp=(struct ether_arp *)(p+sizeof(struct ether_header )); printf(" arp:%s->%s\n",ip_to_char((u_char *)&(arp->arp_spa)),ip_to_char((u_char *)&(arp->arp_tpa))); } . . . else{ printf(" unknown\n"); }

3.2.1.3 Start a Sniffer Application

If everything goes well we can call the function pcap_loop() or the function pcap_next and start grabbing packets. It will come out like

Figure 10.

It lists the total number of captured packets, time, size, type, source address, destination address, and port number.

Figure 10. Start Grabbing Packets

3.2.2 Counter Block

In this section, it will show you how to deal with the file in which grabbed packet and recorded. A Shell programming was made to classify the content of file. Once started, it will sleep for one minute, and then to handle the files. The function of Shell programming is to copy Start_File to Copy_File and Log_File, to count the key words in

Copy_File and put the result to Result.h file, to remove Copy_File and

Log_File.

Figure 11. Information Recorded in File

Sniffer output stream was stored in file “Start_File”, the content of

Start_File is like Figure 10; using Shell programming to copy itself to file “Copy_File” in writing way and to file “Log_File” in appending way.

Once get the file Copy_File , we should classify these data by the key words and count them by shell programming. The key words contain “arp” “rarp” “udp” “tcp” “icmp” “igmp” “snmp” “unknown”

“Number”; and then we record the result in a header file named

Result.h which would be used in late by other programming. Figure 11 shows a shell programming how to deal with the stream, and Table 18 shows a part of source of Shell programming.

Table 18. the Source of Counter Shell

# clear the Start_File echo > Start_File echo "long arp = " > Result.h cat Copy_File | grep -c arp >> Result.h echo ";" >> Result.h … echo "long total = " >> Result.h cat Copy_File | grep -c Number >> Result.h echo ";" >> Result.h #clear the Copy_File echo > Copy_File

3.2.3 Inquiry Block

If we want to analyze our network traffic or the numbers of captured packets etc., so we need to look up the log file.

For the captured packets are so many, we design the log file which updated in one day. After one day, it would be cleared up by our Shell programming. Table 19 shows the source of shell programming which updates a log file.

34 Table 19. the Source of Updating Log File shell

t1=$(date +%R) while : do sleep 60s cp Start_File CopyResult t2=$(date +%R) cat Copy_File >> Log_File if [[ "$t1" = "$t2" ]] then cat /dev/null > Log_File fi …

3.2.4 Connect with Cacti Block

In this part, it will show you how to make a DB table, make a Perl programming to connect with DB.

3.2.4.1 Design a Program to Update a DB table

First is that we make a table in MySQL. Figure 12 shows the structure of table named monitoring.

Figure 12. the Structure of Table

The second is that to update table “monitoring” through the field

“interface”, because the initial value of field “interface” will be set as

“eth0”.

The third is that connecting to DB and update table “monitoring”.

We use Perl language to connect to DB to update table.

3.2.4.2 Design a Perl Programming

Now, there is data in table “monitoring”, these data will be changed in one minute. We make a program by Perl language to connecting to

MySQL and get values from table “monitoring” and then pass these data to Cacti and stored in RRDTool. Table 20 shows a part of source of

“multi_traffic.pl”[17]. Do as that and run this Perl script, the result is like Figure 13. If it came out, it means our programming goes well.

Table 20. the Source of Connecting to DB by Perl

use DBI;

36 my $dbh = DBI->connect("DBI:mysql:database=DatabaseName:host=localhost ","UserName","Password", {"RaiseError" => 1}); my $sqr = $dbh->prepare("SELECT arp FROM monitoring"); $sqr->execute(); while(my $ref = $sqr->fetchrow_hashref()) { $ARP=$ref->{'arp'}; } my $sqr = $dbh->prepare("SELECT rarp FROM monitoring"); $sqr->execute(); while(my $ref = $sqr->fetchrow_hashref()) { $RARP=$ref->{'rarp'}; } … print "ARP:$ARP RARP:$RARP UDP:$UDP TCP:$TCP ICMP:$ICMP IGMP:$IGMP SNMP:$SNMP UNKNOWN:$UNKNOWN TOTAL:$TOTAL"; $dbh->disconnect();

Figure 13. Perl Connect to DB

3.2.4.3 Create a graph from a Perl Script

The first step of creating data input method is very important; set the “Input Type” as “Script/Command”, and “Input String” as “per

/scripts/multi_traffic.pl”. The others step we can refer the

“how to use Cacti” in chapter one to create. Figure 14 shows adding graph item for ARP.

Figure 14. Create Graph for ARP

The graph for RARP, TCP, UDP, ICMP, IGMP, SNMP, UNKNOWN, and TOTAL should be created as like creating graph for ARP.

3.2.5 Set Alarm Block

It is designed for alarm, we set the threshold. If the display value is greater or lower the threshold value, it would send an alarm by email. I use a Gmail which is bound with a phone. If our network were normal, we can know them well in real time.

3.2.5.1 Set Threshold Plugins

Select “console”, click “settings” menu, and click ‘Alerting/Thold’ options, in the ’Default Alerting Options’, check this like Figure 15.

Setting the “Emailing Options” is as like Figure 16; Setting “Mail/DNS” menu is as like Figure 17; setting “SMTP options” is as like Figure 18; at last, is to set “DNS options”.

Figure 15. Default Alerting Options

Figure 16. Emailing Options

Figure 17. Cacti Setting(Mail/DNS)

Figure 18. SMTP Options

We can test it as click the ’Send a Test Email’, if it goes well, it would turn out a window like Figure 19. A few minutes later, we can receive a email from ‘[email protected]’. At last, to click ‘save’ to save them. The Figure 20 shows receiving an email by a phone.

Figure 19. Test Message

Figure 20. Receive a Email by a Phone

3.2.5.2 Creating a Threshold Templates

Click ‘console – Threshold Templates – Add’, it would turn out a window like Figure 21. We select the data template as Unix-Multi

Monitoring, and the Data Source as UDP. And click ‘create’, we can set

‘High/Low Setting’ options as we want. If the data value is greater the high threshold or lower the low threshold, the alert would be triggered.

40 Looking at Figure 22, we set the high threshold as 100, low threshold as 10 for testing. We can change them depend on our network.

In the Figure 23 “other setting” option, we set the “re-Alert Cycle” as “Every Hour”, the “Alert-Mail” as “[email protected]”. At last, save them.

Figure 21. Setting Threshold Template

Figure 22. High/Low Settings

Figure 23. Other Setting

Click “console-Management-Devices”, click “Monitoring-MIE” device, and click “create Graphs for these host”, “Auto-create thresholds”, it goes well, the “created threshold for the graph ‘Unix-Multi Monitoring’ using the data Source UDP” will turn out. Come back to the main page,

click ‘thold’ menu, if the value is greater or lower the set value, it would be seen there. We can add threshold for ARP, TCP, ICMP, etc.

The Figure 24 shows the greater than thresholds with ARP and UDP.

The Figure 25 shows receiving an email from ‘[email protected]’ by a phone, for UDP is greater than the thresholds.

Figure 24. Threshold to ARP and UDP

Figure 25. Email for UDP

3.3 Results

After executing our programming, now, we can launch our cacti and view the graph. If everything goes well, the result would like

Figure 26; and check the DB, log file, it looks like Figure 27 and Figure

28.

Figure 26. the First Graph

Figure 27. the Data in DB Table

Figure 28. the Content of Log File

A few hours later, cacti graphed the graph. The graph would look like Figure 29.

Figure 29. Graph a few Hours Later

44 IV. Result and Analysis

4.1 Make a Test

In this part, we will make two tests to test this system. One is testing with TCP by using PuTTy, the other is test with ICMP by using

PING. In these test, we set the threshold to TCP as 800, and to ICMP as

100.

4.1.1 Test by TCP

PuTTy is a free and open source terminal emulator application which can act as a client for the SSH, Telent, Rlogin, and raw TCP computing protocols. SSH, Telent, Rlogin are three ways of doing the same thing: logging in to a multi-user computer from another computer, over a network. These are all connection-oriented service, so they need high reliability. So they depend on TCP for communication

[18,19]. Table 21 shows a few well-known TCP and UDP ports.

Table 21. TCP/UDP Ports

Port TCP UDP Description Status 22 TCP UDP Secure Shell (SSH) — Official used for secure logins, file transfers (scp, sftp) and port forwarding 23 TCP Telnet protocol — Official unencrypted text communications 80 TCP UDP Hypertext Transfer Official Protocol (HTTP) 513 TCP Rlogin Official

In this test, a PuTTy client window was started to connect with our

Cacti server, Figure 30 shows the foreground processing, it shows that the Sniffer application captured the number of packets in less than a minute and table “monitoring” was updated; Figure 31 shows Cacti fetch data from DB in every ten seconds; Figure 32 is a part of log file;

Figure 33 shows an alarming email for TCP. Checked them together and understood that our server was connected with the host which IP address was “203.250.143.245” by using TCP 22 port at 23:09 PM ; and Figure 34 shows the overall trend graph.

Figure 30. Update the DB Table by TCP

Figure 31. Graph by PuTTY Real Time

Figure 32. the Content of Log File by TCP

Figure 33. a Alarming Email for TCP

Figure 34. Graph for TCP by PuTTY

4.1.2 TEST by ICMP

ICMP is one of the core protocols of the Internet Protocol Suite. It is chiefly used by the operating systems of networked computers to send error messages indicating that a requested service is not available or that a host or router could not be reached. ICMP can also be used to relay query message [20, 21]. Table 22 listed few permitted control message.

Table 22. ICMP Type

Type Code Description 0 – Echo Rely 0 Echo reply(used to ping) 8 – Echo Request 0 Echo request

48 PING was used to ping the Cacti server in windows environment.

Figure 35 shows it would send 1000 packets to destination host; Figure

36 shows the foreground processing, it shows that the Sniffer application captured the number of packets in less than a minute and table “monitoring” was updated; Figure 37 shows Cacti fetches data from DB in every ten seconds; Figure 38 is a part of log file; Figure 39 shows an alarm email for ICMP. Checked them together and understood that our server was pinged by a host which IP address was

“203.250.143.245” at 23:22PM; and Figure 40 shows the overall trend graph.

Figure 35. Window PING Messages

Figure 36. Update the DB Table by PING

Figure 37. Graph by PING Real Time

Figure 38. the Content of Log File by PING

Figure 39. an Alarm Email for ICMP

Figure 40. Graph for ICMP by PING

4.2 Analysis

To sum up the above those tests, we can use phone to receive alarming email in real time once the threshold is triggered. For test one by PuTTY, it proved that PuTTY uses TCP protocol to communicate by using TCP 22 port and the packet where it come from; for test two by

ICMP, it proved that the type of ICMP packet and the packet where they came from and how many they are.

Test one is a conceptual test; and the Test two is necessary and is a accuracy test as ICMP message that we can control the numbers to be sent.

This system was stimulated by artificial means and got a lot of difference data and graphs. It proved that it is possible to monitor our server by using this system, it graphs accurately and quickly, and it is also a real-time and effective system.

52 V. Conclusions and Future Works

In a word, this system has many advantages: open source; friendly-use; using RRDTOOL to store data; real-time alarm; quick and accurate results and so on.

5.1 Conclusions

As a whole, this thesis is divided into 5 parts.

The first part mainly talks about the background and purpose of study.

The second part mainly talks about the Cacti’s concept, work principle, the usage and the PIA; and the Sniffer’s concept, Libpcap functions, work principle, and the making method.

The third part mainly talks about how to design a system and in this session, it is also divided into 5 blocks: one is packet capture block which makes a specific Sniffer to collect data from NIC; one is counter block which uses Shell programming to count the key works from a file which the result of collected by Sniffer application was recorded in; one is inquiry block which appends and updates the log file; one is connecting to Cacti block which makes a script to connect DB to fetch the values and make Cacti graph with those values; at last one is sending alarm emails once the threshold was triggered and received an alarm email by a phone.

The fourth part mainly talks about how to make a test to stimulate this system to get different data and graphs; and how to analyze the test results.

The last part mainly talks about conclusions and future works.

5.2 Future Works

Although this system can reflect the network status by graph quickly and accurately and has many advantages. It also has two disadvantages, in my opinion, one is the inquiry block, and it is unfavorable. When we need to check the log file, we should look up the log file line by line; the other is this system only lists Ethernet type, in order to protesting our server from avoiding attack and maintain a good network environment , we should add more types and know more detail about packets.

54 References

[1] Myung-Sup Kim, Yong J.Won, and James Won-Ki Hong,

“Application-Level Traffic Monitoring and an Analysis on IP

Networks”, ETRI Journal, Volume 27, Number 1, February 2005

[2] http://www.cacti.net

[3] http://www.cacti.net/features.php

[4] http://en.wikipedia.org/wiki/Pcap

[5] http://www.mrtg.org/rrdtool

[6] http://www.scritube.com/limba/engleza/computers/INTRODUCTI

ON-OF-MRTG-AND-RRD-T14881.php

[7] http://oss.oetiker.ch/rrdtool/tut/rrdtutorial.en.html

[8] www.cacti.net/what_is_cacti.php

[9] http://www.cacti.net/downloads/docs/html/operating_principles.ht

[10] http://www.cacti.net/downloads/docs/html/install_unix.html

[11] Luis Martin Garcia, “Programming with Libpcap”

http://undergraduate.csse.uwa.edu.au/units/CITS3231/reading/lib

pcap-programming.pdf

[12] http://sourceforge.net/projects/Libpcap

[13] 노광미 “리눅스에서 pcap library 를 사용하여 패킷을 잡아보기”

V0.3 2000.09.14

[14] 강승일 “Packet Capture using Libpcap” 2006.3.10

[15] http://networksecurity.org.ua/index.html?page=0596007949/net

workst-chp-10-sect-5.html

[16] http://dog.tele.jp/winpcap/html/structbpf_program.html

[17] http://www.tizag.com/perlT/perldbiconnect.php

[18] en.wikipedia.org/wiki/PuTTY

[19] en.wikipedia.org/wiki/List_of_TCP_and_UDP_port_numbers

[20] http://www.faqs.org/rfcs/rfc792.html

[21] http://www.iana.org/assignments/icmp-parameters

56 Acknowledgement

Two years’ graduate life is tedious, tired; but it is an important. In the coming ending the graduate life, I would like to take this opportunity to thank them who have used to help me and give me a good advice.

At first, I am greatly indebted to my supervisor, Professor Jung

Hoe-Kyung who help not only give me help in learning, but also in my lift. My deepest gratitude and respect go to him. It is for his constant encouragement, critical instructions, his great care and precious advice and suggestions. This feeling is beyond the words, but still I give him my deepest wishes and wish joy, happiness, health will always be with his family.

Second, my appreciation goes to my family. Thank my father, my mother for giving a good body and thank my wife gives me continuous support, encouragement and understanding, and thank my sister, my brothers for helping me in material and spiritual. And this spiritual power gives me powerful energy.

Then, I would also like to extend my sincere thanks to all other teachers who gave me comprehensive knowledge during the past three academic years. I have benefited so much from their courses and constant encouragement. Their studies gave me some reference.

Then, I would also like to extend my thanks to my school brother

(Kim Kun-sub, Ban Tae-hak, Yu So-ra) for helping. Thank them for teaching me the basics and opening the learning door for me.

At last, I would like to thanks my aunt (Pak Sun-ai) who is Korean.

During Korea, she gives me lots of help like my mother to give me encouragement. Wish her has a good health, good luck and happiness throughout her every year.

December, 5, 2011

Xiao Huang