A Survey of Botnet Detection Techniques by Command and Control Infrastructure

Journal of Digital Forensics, Security and Law

Volume 10 Article 2

2015

Thomas S. Hyslip Norwich University

Jason M. Pittman Cal Poly Pomona University

Follow this and additional works at: https://commons.erau.edu/jdfsl

Part of the Computer Engineering Commons, Computer Law Commons, Electrical and Computer Engineering Commons, Forensic Science and Technology Commons, and the Information Security Commons

Recommended Citation Hyslip, Thomas S. and Pittman, Jason M. (2015) "A Survey of Botnet Detection Techniques by Command and Control Infrastructure," Journal of Digital Forensics, Security and Law: Vol. 10 , Article 2. DOI: https://doi.org/10.15394/jdfsl.2015.1195 Available at: https://commons.erau.edu/jdfsl/vol10/iss1/2

This Article is brought to you for free and open access by the Journals at Scholarly Commons. It has been accepted for inclusion in Journal of Digital Forensics, Security and Law by an authorized administrator of (c)ADFSL Scholarly Commons. For more information, please contact [email protected]. A Survey of Botnet Detection Techniques by Command ... JDFSL V10N1 This work is licensed under a Creative Commons Attribution 4.0 International License.

A SURVEY OF BOTNET DETECTION TECHNIQUES BY COMMAND AND CONTROL INFRASTRUCTURE

Thomas S. Hyslip, Sc.D. Jason M. Pittman, Sc.D. Norwich University Cal Poly Pomona University 919-274-4526 805-907-5313 [email protected] [email protected]

ABSTRACT Botnets have evolved to become one of the most serious threats to the Internet and there is substantial research on both botnets and botnet detection techniques. This survey reviewed the history of botnets and botnet detection techniques. The survey showed traditional botnet detection techniques rely on passive techniques, primarily honeypots, and that honeypots are not effective at detecting peer-to-peer and other decentralized botnets. Furthermore, the detection techniques aimed at decentralized and peer-to-peer botnets focus on detecting communications between the infected bots. Recent research has shown hierarchical clustering of flow data and machine learning are effective techniques for detecting botnet peer-to-peer traffic. Keywords: botnet, botnet detection, distributed denial of service, malware Cooke et al., 2005). Eventually, a network of 1. INTRODUCTION bots was developed under the direction of IRC administrators and became known as a botnet The term ‘botnet’ is now associated with (Dittrich, 2012). IRC administrators were able cybercrime and hacking (Alhomoud, Awan, to send a single command from their computer Disso, & Younas, 2013). However, botnets and the botnet would execute that command were originally developed to assist with the on all the IRC Servers. Figure 1 shows a administration of Internet Relay Chat (IRC) typical network configuration of an IRC Servers (Cooke et al., 2005). As the popularity botnet. Nefarious individuals realized the of IRC expanded, the IRC server potential of botnets for unethical purposes and administrators developed software to perform the botnets began to infect IRC users’ automated functions to assist with the computers without the users’ knowledge and administration of the IRC Servers (Cooke et use those computers without the users’ consent al., 2005). The computers that operated the (Cao & Qiu, 2013; Cooke et al, 2005). software and performed the automated A Computer Emergency Response Team, functions were referred to as robot computers Coordination Center (CERT/CC) advisory and eventually as bots (Dittrich, 2012). The published on March 11, 2003, CERT/CC Eggdrop IRC bot was the first IRC Bot, Advisory CA-2003-08, warned against the GT- developed in 1993 by Jeff Fisher to assist with bot and sdbot utilizing IRC to remotely the administration of IRC channels and which control compromised systems (Householder & is still in use today (Alhomoud et al., 2013; Danyliw, 2003). Householder and Danyliw

© 2015 ADFSL Page 7 JDFSL V10N1 A Survey of Botnet Detection Techniques by Command… This work is licensed under a Creative Commons Attribution 4.0 International License. (2003) also highlighted the growing size of Danyliw also warned of the botnets’ ability to botnets, with reports of GT-bot botnets in launch distributed denial of service attacks excess of 140,000 bots and the sdbot with over with TDP, UDP, and ICMP packets. 7000 compromised systems. Householder and

Figure 1. An IRC Botnet diagram showing the individual connections between each “bot” and the command and control server.

The size and scope of botnets continued to Santos, Bringas, & Val, 2011; Feily, rise at an alarming rate and in February 2010, Shahrestani, & Ramadass, 2009; Hasan, Spanish authorities and the FBI dismantled Awadi, & Belaton, 2013; Zeng, 2012; Zhang, the Mariposa botnet, which consisted of over 2012). This survey analyzed the history and 12 million compromised computers (Roscini, evolution of botnet detection as botnets 2014). Only 2 years after the takedown of the changed from a centralized command and Mariposa botnet, another botnet, the Metulji control structure to a decentralized peer-to- botnet, was dismantled by the FBI and peer control structure. When early research on consisted of over 20 million compromised botnet detection focused on the use of passive computers (Ventre, 2013). In 2013, Rossow honeypots and detection techniques aimed at and Dietrich considered botnets to be one of detecting botnet command and control the Internet’s most serious threats and Awan communications in centralized botnets, et al. (2013) believed botnets are a priority for Botmasters began to use peer-to-peer and many countries’ cyber defenses. decentralized communications (Feily et al., 2009; Hasan, Awadi, & Belaton, 2013; Zeng, There has been considerable research into 2012; Zhang, 2012). Botnet detection botnets and botnet detection techniques, but techniques were then developed to identify botnets are constantly evolving to stay ahead communications between infected computers of the latest detection techniques (Brezo, within the decentralized botnets and

Page 8 © 2015 ADFSL A Survey of Botnet Detection Techniques by Command ... JDFSL V10N1 This work is licensed under a Creative Commons Attribution 4.0 International License. Botmasters responded with the use of honeynet as a network of computers placed on obfuscated and encrypted communications the Internet with the intention of capturing (Brezo, Santos, Bringas, & Val, 2011; Feily et unauthorized activity directed at the al., 2009; Gu, Porras, Yegneswaran, Fong, & computers. The purpose of a honeynet is to Lee, 2007; Zeng, 2012; Zhang, 2012). monitor network activity after malicious software is installed on the honeynet’s There have been several previous surveys computers and learn how the malicious of botnet detection techniques, but most are software operates, with the goal of capturing dated prior to 2009 and do not include botnet new and unknown attacks and malicious detection techniques aimed at decentralized or software (Spitzner, 2003). In a 2009 survey of encrypted botnets (Feily et al., 2009; Bailey, botnet detection techniques, Feily et al. (2009) Cooke, Jahanian, Yunjing, & Karir, 2009; Zhu, found a vast majority of the botnet detection Lu, Chen, Fu, Roberts, & Han, 2008). Silva, techniques rely heavily on honeynets because Silva, Pinto and Salles (2013) conducted a honeynets are simple to operate and are survey of Botnets that included peer to peer, passive to the botnet, so no interaction is decentralized, and encrypted botnets. Silva et required with the botmaster or command and al. included a history of botnets and a survey control server by the researcher. The honeynet of different botnet detection techniques, as well receives the instructions or commands from the as a sample of techniques for botnet defense. botnet operator but does not itself respond or What separates this survey from previous execute the commands (Spitzner, 2003). work is the comparison of botnet detection In July 2005, Cooke, Jahanian, and techniques by command and control McPherson proposed monitoring transmission infrastructure. To the best of our knowledge, control protocol (TCP) port 6667 on live previous research has not yet clearly identified networks for IRC botnet command and control which detection techniques are effective against traffic as a possible botnet detection technique. which types of command and control TCP port 6667 is the default IRC port, but infrastructure. This survey provides a Cooke et al. recognized the default port is comprehensive review of botnet detection easily changed to non-standard ports, so the techniques and provides tables for quick review detection technique of monitoring networks for of which techniques are effective against which IRC traffic on TCP 6667 was not command and control infrastructures. recommended. Cooke et al. proposed a second botnet detection technique utilizing a honeypot 2. EARLY BOTNET and capturing traffic between the honeypot DETECTION (2005-2010) and the IRC botnet command and control server. The captured traffic was then analyzed The Honeynet project was a pioneer in botnet to develop signatures of botnet traffic (Cooke detection (Feily et al., 2009). The Honeynet et al, 2005). Cooke et al. determined there project began in 1999 as an information were no connection-based variables that would mailing list for information security be useful in detecting botnets via monitoring professionals and was established as a non- network traffic for command and control profit information security research traffic. The botnets’ ability to modify the organization with the mission to learn about mode or behavior of communications can easily computer and network attacks in 2000 defeat detection techniques based on command (Spitzner, 2003). Spitzner (2003) defined a

© 2015 ADFSL Page 9 JDFSL V10N1 A Survey of Botnet Detection Techniques by Command… This work is licensed under a Creative Commons Attribution 4.0 International License. and control traffic analysis (Cooke et al., HTTP botnets that use a centralized command 2005). and control server, but no prior knowledge of a botnet’s signature is required to detect hosts Although Cooke et al. (2005) determined within a local area network (Gu, Zhang, et al., monitoring for command and control traffic 2008). In both IRC and HTTP botnets, Gu, was not effective, Gu et al. (2007) develop Zhang, et al. recognized that the bots must BotHunter, to detect inbound command and make connections to the command and control control traffic with bots inside a local area server to obtain commands and then the bots network. Gu et al. developed two plugins and will have similar activity based on the one ruleset for the open source, intrusion commands. Based on research conducted by detection system, Snort (Cisco, 2014). For Zhuge, Holz, Han, Guo, & Zou (2007), Gu and inbound traffic detection, Gu et al. (2007) his associates developed BotSniffer to recognize developed the Snort plugin, Statistical Scan similar behavior by hosts after communicating Anomaly Detection (SCADE) which monitors with a possible command and control server 24 TCP and 4 UDP inbound ports for possible located at the same IP address. Zhuge et al. command and control traffic associated with (2007) had determined that over 28% IRC botnet malware. SCADE also monitors botnet commands are for spreading malware outbound traffic for hosts that scan a large and 25% of IRC commands are for distributed number of external IP addresses or have high denial of service attacks. Based on these number of failed external connections. statistics, Gu, Zhang et al. (2008) developed The second Snort (Cisco, 2014) plugin anomaly based algorithms to detect command developed by Gu et al. (2007) Statistical and control traffic, as well as network Payload Anomaly Detection Engine (SLADE) scanning, with the open source intrusion attempts to detect malicious payloads through detection system, Snort (Cisco, 2014). Utilizing packet inspection of all inbound traffic. previously captured network traffic with SLADE utilizes anomaly detection to known botnet infections, Gu, Zhang et al. determine if payloads are suspicious based on (2008) successfully tested BotSniffer and the payloads standard deviation from test detected 100% of IRC botnet command and payloads of normal Internet traffic (Gu et al., control traffic with a false positive rate of 2007). The problem with deep packet 0.16%. inspection is the large overhead associated with Research by Karasaridis, Rexford, and inspecting voluminous amounts of traffic in Hoeflin (2007) in anomaly-based detection large networks (Zhang, 2012). Gu et al. (2007) techniques demonstrated the ability to also developed four rulesets for Snort (Cisco, calculate the size of botnets as well as identify 2014) to monitor 1383 heuristics of known command and control servers by analyzing botnets and malware. BotHunter’s final phase flow data from the transport layer in large- of detection is a correlation matrix that weighs scale networks. However, this technique was each Snort alert and applies a coefficient based only tested against IRC based botnets utilizing on the type of alert to determine if a host is a centralized command and control server infected (Gu et al., 2007). (Karasaridis et al., 2007). Karasaridis et al. Gu, Zhang, and Lee (2008) built upon recommended additional research in the BotHunter to develop BotSniffer, a system detection of peer-to-peer and HTTP based designed to detect botnet command and botnets. control traffic through anomaly detection. BotSniffer is limited to detecting IRC and

Page 10 © 2015 ADFSL A Survey of Botnet Detection Techniques by Command ... JDFSL V10N1 This work is licensed under a Creative Commons Attribution 4.0 International License. With the introduction of botnets activities (Gu, Perdisci et al., 2008). Based on communicating via peer to peer networks, Gu, the similar traffic and activities, BotMiner Perdisci et al. (2008) developed BotMiner as a clusters similar communication traffic into C- botnet detection technique that is effective plane traffic and like malicious activities into against any botnet command and control A-plane traffic (Gu, Perdisci et al., 2008). Gu, protocol or structure, including peer to peer. Perdisci et al. then detected botnets by Figure 2 shows a typical peer to peer botnet correlating the A-plane and C-plane traffic. infrastructure without a central command and To cluster communications within the C- control server. BotMiner detects botnets by plane traffic, Gu, Perdisci, et al. (2008) clustering hosts based on similar traffic and monitored TCP and UDP network flow data malicious activities (Gu, Perdisci et al., 2008). and recorded IP addresses, network ports time Gu, Perdisci et al.’s research focused on the and duration of the traffic, and the number of botnet communications since botnets much packets and bytes transferred in each direction. communicate with a command and control Gu, Perdisci et al. used Snort (Cisco, 2014) to server of with other bots to receive commands capture A-plane traffic based on malicious such as when to scan or launch attacks. In activities, scanning, spam, and binary order for the bots to function as a botnet, the downloads. The C-plane clusters were then bots must receive the same commands; correlated with the A-plane clusters to identify therefore the researchers believed the same hosts that are part of a botnet (Gu, Perdisci et botnet would have similar traffic and malicious al., 2008).

Figure 2. Peer to peer botnet showing the decentralized infrastructure and lack of a command and control server. The Botmaster is able to communicate directly with a bot and the commands are passed between the bots.

Wang and Yu (2009) developed a botnet Experimental results showed the technique to detection technique aimed at detecting be effective for detecting command and control command and control communications of traffic of four different botnet types. However, centralized botnets, irrespective of the the technique is only effective against botnets particular botnet. Wang and Yu based their with a centralized command and control detection technique on the timing and structure (Wang & Yu, 2009). uniformity of botnet communications; Wang Using structured overlay networks for and Yu’s technique used only the packet size communication, Nagaraja, Mittal, Hong, and timing interval between arriving packets Caesar and Borisov (2010) developed BotGrep, as variables to determine if network traffic was a botnet detection technique focused on peer- botnet command and control communications. to-peer botnets. Nagaraja et al. developed an

Page 12 © 2015 ADFSL A Survey of Botnet Detection Techniques by Command ... JDFSL V10N1 This work is licensed under a Creative Commons Attribution 4.0 International License. algorithm that isolates peer-to-peer activity on host detections and utilized netflow communication based on the pairing of nodes data for network level detection but avoided that communicate with each other. BotGrep full packet inspection, which ensures privacy then utilizes graph analysis to identify botnet for network users. The researchers successfully hosts. Although BotGrep is not affected by tested the combined host and network botnets that vary ports or use encryption, detection technique. Such may very well be the BotGrep does require a seeding of botnet first combined host and network level information to be effective; therefore, the detection technique developed. Further, Zeng researchers recommend operating a honeynet et al. stated that their combined host detection to capture botnet intelligence that can be used technique was effective against IRC, peer-to- by BotGrep to identify the rest of the botnet peer, and HTTP botnets, but noted that the (Nagaraja et al., 2010). technique is limited by the scalability. Zeng et al. recognized that the host level detection Prior detection techniques relied on either technique requires installation on all hosts host level detection or network level detection. within an organization and may only be Hoever, Zeng, Hu and Shin (2010) developed a accomplished in enterprise networks. botnet detection technique that incorporates both host level detection and network level Table 1 summarizes early botnet detection detection. Zeng et al. believed that by techniques based on the techniques ability to combining the host and network level detect different types of botnet infrastructure. detections and correlating the alerts, their Table 1 also provides an indirect timeline of technique would increase the rate of detection botnet infrastructures and communications. and overcome the limitation of each technique While early botnets used IRC exclusively, the alone. Zeng et al. used registry changes, file introduction of HTTP and P2P system modifications and network stack communications is evident. changes to alert for possible botnet malware

Table 1 Early Botnet Detection Techniques Researchers IRC HTTP P2P Cooke et al. (2005) X Gu et al. (2007) X Karasaridis, Rexford, and Hoeflin (2007) X Gu, Zhang, and Lee (2008) X X Gu, Perdisci et al. (2008) X X X Wang and Yu (2009) X X Nagaraja, Mittal, Hong, Caesar and Borisov (2010) X Zeng, Hu and Shin (2010) X X X

© 2015 ADFSL Page 13 JDFSL V10N1 A Survey of Botnet Detection Techniques by Command… This work is licensed under a Creative Commons Attribution 4.0 International License. 3. MODERN BOTNET peer botnets. Zhang et al.’s technique first RESEARCH (2011-14) detects all peer-to-peer traffic and hosts and then develops signatures for different With the increase in peer-to-peer and applications. Based on the signatures, Zhang et decentralized botnets a majority of modern al. were able to differentiate legitimate peer-to- research has focused on detecting peer-to-peer peer traffic from botnet peer-to-peer traffic. To and decentralized botnets, in particular, the develop the signatures of peer-to-peer traffic, communications between bots within the Zhang et al. used the length of time a peer-to- botnet. Francois, Wang, State and Engel peer program is operating because botnets run (2011) developed BotTrack and overcome the as long as possible and whenever a computer is limitations of forensic analysis when examining turned on, while legitimate peer-to-peer large datasets of NetFlow data to detect peer- programs are often started and stopped by the to-peer botnet communications. Similar to user. Based on the length of time a peer-to- BotGrep (Nagaraja et al., 2010), Francois et peer program is active, Zhang et al. filtered out al. developed BotTrack to identify peer-to-peer peer-to-peer hosts with short active times. connections between hosts and identify botnet After filtering the peer-to-peer traffic based hosts utilizing an algorithm and graph on length of active peer-to-peer traffic Zhang analysis. Building on BotTrack, Francois, et al. (2011) further differentiated the traffic Wang, Bronzi, State and Engel (2011) used based on IP addresses contacted by peer-to- Hadoop (Hadoop, 2013), an open source form peer hosts. Since peer-to-peer botnet hosts of distributed computing based on Google’s within the same LAN/WAN will often MapReduce (Dean & Ghemawat, 2004) to communicate with the same IP addresses and develop BotCloud to efficiently analyze with other bots within the LAN/WAN, the NetFlow data. BotCloud showed improved researchers were able to filter out peer-to-peer detection rates when prior information about hosts that did not communicate with any IP botnets is developed with a honeypot (Francois addresses that were not contacted by other et al., 2011). Furthermore, BotCloud’s use of peer-to-peer hosts (Zhang et al., 2011). The Hadoop (2013) increased the efficiency and final filter Zhang et al. applied was based on speed of botnet detection (Francois et al., the connection status of the traffic. If a peer- 2011). to-peer host had completed an outgoing three Zhang, Perdisci, Lee, Sarfraz and Luo way handshake on a TCP connection or a (2011) developed a botnet detection technique UDP connection with a request and response to detect botnet peer-to-peer communications packet, the traffic is kept and all other traffic utilizing statistical fingerprints of peer-to-peer is filtered out (Zhang et al., 2011). Zhang et al. traffic. Peer-to-peer botnets have an advantage based this filter on their findings that peer-to- over IRC or HTTP protocol botnets because peer nodes function as both a server and a the former do not have a centralized command client, and must accept connections from other and control server and single point of failure hosts in the network and initiate connections (Zhang et al., 2011). The lack of a centralized with the same hosts. After this traffic filtering command and control server make peer-to-peer was complete, Zhang et al. attempted to botnets more resilient and more difficult to identify peer-to-peer botnet hosts. disable (Zhang et al., 2011). Zhang et al.’s Zhang et al.’s final action to identify peer- peer-to-peer detection technique was focused to-peer botnet hosts involved differentiating on local area networks (LANS) and enterprise between legitimate peer-to-peer traffic and wide area networks (WANS); to detect peer-to-

Page 14 © 2015 ADFSL A Survey of Botnet Detection Techniques by Command ... JDFSL V10N1 This work is licensed under a Creative Commons Attribution 4.0 International License. botnet peer-to-peer traffic. To determine this, collaborated with each other to detect patterns Zhang et al. analyzed the traffic for hosts that and alerts based on rules. Han et al. also ran the same protocol and communicated with observed that Garlic would regenerate rules a high percentage of the same IP addresses. As based on feedback from the alerts and stated earlier, bots of the same peer-to-peer redistributed updated rules to the terminal botnet will communicate with each other and nodes. During experimental testing, Han et al. share IP destinations of other bots within the were able to detect all 20 bots within 45 botnet. Furthermore, Zhang et al.’s research minutes; however, they only experimented with showed bots of the same botnet use the same IRC botnets operating on TCP ports 6660- peer-to-peer protocol. Based on these filters 6669 (including IRC port 6667), as well as and detection techniques, Zhang et al. were HTTP botnets operating on port 80. Han et al. able to detect 100% of the peer-to-peer bots did not test peer-to-peer botnet nor did they within captured network traffic with only a provide any research on peer-to-peer botnets 0.2% false positive rate. within their study. As botnets began to use encrypted Increasingly, botnets expand through drive communications, Barthakur, Dahal and Ghose by download attacks. In response, Zhang (2012) developed a procedure for detecting (2012) developed a new botnet detection encrypted peer-to-peer botnet communications. technique to identify drive by download Barthakur et al. used Support Vector attacks and detect botnets in the infection Machines to analysis network traffic and stage. Zhang recognized that many botnets use classify botnet communications based on drive by downloads to infect new bots and by patterns and statistical differences between preventing the initial infection the size and peer-to-peer botnet communications and scope of botnets could be greatly diminished. normal web traffic. Barthakur et al. recognized To identify drive by download techniques, botnet communications use many random Zhang collected HTTP traces from honeypots ports and attempt to keep packet sizes to a and whenever exploits were detected, the minimum, which is the opposite of legitimate honeypots used a dynamic WebCrawler to peer-to-peer to traffic. Based on these facts, record the URLs and IP addresses of the Support Vector Machines were able to analyze domains. Zhang then clustered groups of patterns of peer-to-peer traffic and successfully hostnames that share IP addresses. By identify botnet communications (Barhakur et clustering the hostnames based on shared IP al., 2012). addresses, Zhang was able to defeat the botnets that use fast flux network changes to Han, Chen, Xu and Liang (2012) proposed command control server domain names and IP a botnet detection and suppression system addresses. Fast flux networks use numerous IP called Garlic. Han et al. believed Botmasters addresses for one domain name and repeatedly attempted to keep botnets as small possible to update the DNS records for the domain name avoid detection and allow the Botmaster to to different IP addresses to avoid detection easily change the botnet’s command and (Caglayan, Toothaker, Drapaeau, & Burke, control server. Han et al. stated the botnet 2010). suppression system, Garlic, was capable of automatically detecting and suppressing Furthermore, Zhang (2012) also developed botnets. Han et al.’s Garlic suppression system a system to increase the scalability of botnet relied on terminal nodes distributed detection systems. Zhang’s system improved throughout a network and the nodes upon current detection systems by reducing

© 2015 ADFSL Page 15 JDFSL V10N1 A Survey of Botnet Detection Techniques by Command… This work is licensed under a Creative Commons Attribution 4.0 International License. the amount of packets requiring deep packet detect changes related malware associated with inspection; Zhang accomplished this by botnets (Ilavarasan & Muthumanickam, 2012). developing a three-step process that captures Ilavarasan and Muthumanickam analyzed network flows, correlated the network flows network traffic to identify peer-to-peer traffic and detected botnets through fine grain and cluster similar traffic based on activity and analysis. Rather than use deep packet contacted IP addresses. The final process in inspection, Zhang’s system used network flow Ilavarasan’s and Muthumanickam’s detection information and packet header information, technique was a correlation engine that which allowed for deployments in larger combined the network analysis with the host networks and the ability to inspect traffic for level detection to alert for possible botnet botnet command and control traffic. infections. Zhang (2012) also developed a flow-capture Zeng (2012) developed a three-pronged process that monitors the edge of large approach to identify and mitigate the effects of networks and gathers netflow data on possible botnets. Zeng proposed utilizing end host botnet traffic. The netflow data is then containment of infected bots, network edge assembled and passed to the flow-correlation detection of botnets, and measuring of network module. Zhang used a process developed in components at the infrastructure level for large BotMiner called C-flow (Gu et al., 2008) to botnet detection. Zeng also presented a proof build the flow-correlation module. However, of concept for future botnets utilizing mobile Zhang used a more efficient process for smart phones and SMS messages for command clustering netflows to allow for larger traffic and control of a botnet. Zeng discussed the volumes and employed correlation to identify history of botnets and botnet detection hosts that had similar persistent techniques and highlighted the limitations of communications. In Zhang’s final process, a the current strategies to detect botnets. Most fine-grained detector utilizes previous detection notably, the researcher discussed the rapidly techniques based on deep packet inspection. changing communication methods for botnets, Zhang used both BotMiner and BotSniffer to including peer-to-peer communications, and inspect the traffic identified as possible botnet the limitations of current HTTP and IRC traffic by the flow-capture and flow-correlation detection techniques (Zeng, 2012). modules and was able to achieve 100% Zeng’s (2012) research on end-host botnet detection rate when using cross correlation of detection incorporated previous techniques for flows and the B-sampling algorithm. For containment of fast spreading network worms sampling rates above 0.05%, Zhang obtained with new behavior analysis of all applications false positive rates between 0.3% and 8%, as on the computer. The behavior analysis the sampling rate increased. However, when examined the actions of applications at the Zhang used both flow-correlation and a fine- registry, file system and network stack, and grain detector, Zhang was able to detect 100% was successful at identifying suspicious actions, of botnets with no false positives for sampling while allowing legitimate applications (Zeng, rates above 0.05%. 2012). Furthermore, the rate of false positives Ilavarasan and Muthumanickam (2012) was greatly reduced when compared to existing combined host level detection and network detection techniques (Zeng, 2012) level analysis to overcome the limitations of Zeng (2012) also combined the edge each separately. The host level detection network detection technique with the host- utilized registry analysis and file monitoring to based detection to increase the effectiveness of

Page 16 © 2015 ADFSL A Survey of Botnet Detection Techniques by Command ... JDFSL V10N1 This work is licensed under a Creative Commons Attribution 4.0 International License. botnet detection. The edge network detection sends the shortest flow of data possible. utilizes NetFlow data captured from routers Similar to anomaly based IDS, Disclosure and does not access the packet payload, performed better when larger amounts of ensuring privacy for legitimate traffic (Zeng, benign flow data were analyzed. This enabled 2012). Zeng identified 17 traits of botnets that Disclosure to distinguish between benign server he used to determine if network traffic was traffic and command and control server traffic suspicious and related to botnets. The 17 traits (Bilge et al., 2012). During evaluation, Bilge identified by Zeng for botnet traffic include the et al. tried numerous settings within Disclosure following network flow features: mean, and the results showed as the detection rate variance, skewness, and kurtosis for duration; increased, so did the false positive rate. mean, variance, skewness, and kurtosis for Using the different behaviors of botnets, Li, total bytes; mean, variance, skewness, and Xie, Luo and Zhu (2013) developed Snort rules kurtosis for number of packets; and the to detect botnet activity. Specifically, Li, Xie, number of TCP flows, UDP flows, SMTP et al. determined there were six behaviors flows, unique IPs contacted, and number of unique to botnets: abnormal access to backup suspicious ports. DNS servers, large number of domain name The final portion of Zeng’s (2012) requests to a single domain, accessing fast flux technique was botnet detection at the networks, downloading malware, ingress and infrastructure level. Zeng chose to focus on egress scanning, and null TCP connections. large peer-to-peer botnets and evaluate the Based on these behaviors, Li, Xie et al. feasibility of detecting peer-to-peer botnets at developed Snort rules to detect each behavior. the Internet infrastructure level. Zeng Every Snort alert was tracked in an alert concluded that host-based techniques for matrix and correlated against the six known botnet detection are not reliable and network botnet behaviors to identify botnet activity edge detection is necessary to detect botnets. (Li, Xie et al., 2013). Li, Xie et al. were Furthermore, the behavior analysis and successful at identifying 20 known botnets with NetFlow analysis Zeng developed is detection rates between 74% and 94% with no independent of the type of botnet and false positives. Lie et al. also test the Snort command and control communication a botnet (Cisco, 2014) rules against 8 unknown botnets utilizes, thus it greatly increases the chances of and detected between 56% and 73% of botnet detection. unknown botnets with zero false positives. Lie, Xie et al. explain unknown botnets as botnets Bilge, Balzarotti, Robertson, Kirda, and that the malicious behavior of the botnet is Kruegel (2012) developed Disclosure, a botnet unknown, not the actual malware. detection system to detect command and control servers, rather than individual bots. Rossow and Dietrich (2013) recognized Using Netflow data, Disclosure distinguishes that existing intrusion detection systems are between botnet command and control server not capable of detecting all encrypted traffic, and benign server traffic through the command and control traffic based on payload flow size between a server and client. Bilge et signatures. The payload-based signatures used al. stated command and control server traffic by intrusion detection systems are easily does not fluctuate significantly due to the defeated by encrypted or obfuscated command limited number of commands used by the and control traffic because botnets employ botnet. Furthermore, the objective of the defense measures against payload signature botnet is to stay undetected, so the botnet recognition, such as dynamic encryption keys,

© 2015 ADFSL Page 17 JDFSL V10N1 A Survey of Botnet Detection Techniques by Command… This work is licensed under a Creative Commons Attribution 4.0 International License. data payloads encrypted with the XOR cipher, Weasel botnet that employs fully encrypted and varying the length of messages (Rossow & communications to test a new detection Dietrich, 2013). To counter the defenses technique that is capable of detecting employed by botnets, Rossow and Dietrich encrypted botnet communications. Garant and developed Provex, a Network Intrusion Lu and identified six features to identify the Detection system (NIDS), which detects encrypted botnet communications: length in encrypted botnet communications and was bytes, packet count, protocol, flow duration, designed to learn from previously decrypted flow direction, and TCP flags. To develop the botnet communications and identify signature of botnet communications utilizing characteristic bytes within encrypted traffic. the six features, Garant and Lu used a decision Then Provex “derives probabilistic vectorized tree classification with the C4.5 and Weka’s signatures that can be used to verify if J48 algorithms; the researchers successfully decrypted packets stem from a certain malware detected over 90% of encrypted botnet family’s C&C” (Rossow & Dietrich, 2013, p. 6). communications with a false positive rate of Although Provex must decrypt network traffic 9.9% and false negative rate of 10.5%. and match signatures to the decrypted packets, Zhang, Perdisci, Lee, Luo, & Sarfraz (2014) Rossow and Dietrich were able to operate built upon their previous work in 2011 to Provex at nearly 1Gbit/s of network traffic increase efficiency, reduce storage costs, and without packet loss and believed that Provex boost the system scalability. Zhang et al. would handle network speeds of up to eliminated the analysis of failed network 10Gbit/s. In laboratory testing, Provex connections for P2P traffic as an indicator of detected all true positive encrypted P2P botnet traffic and relied entirely on communications 100% of the time for six netflow analysis for botnet detection. Through botnet variants and 78%, 81.5%, 87%, and hierarchical clustering of P2P flows, Zhang et 97% for four botnets, with only three false al. were able to distinguish legitimate P2P positive results (Rossow & Dietrich, 2013). traffic from botnet P2P traffic with 100% true Using 1317 distinct malware samples from positive detection rate and 0.2% false positive 8 malware families that communicate via P2P, detection rate. Kheir and Wolley (2013) developed a malware Using machine learning, Haddadi, Morgan, classifier as part of their botnet detection Filho, & Zincir-Heywood (2014) developed a technique. Kheir and Wolley recognized that botnet detection technique for HTTP botnets. P2P botnet traffic can be distinguished by Haddadi et al. used C4.5 and Naïve Bayes three characteristics, time, space, and flow size. machine learning classifiers to analyze netflow Using these characteristics, Kheir and Wolley data and detect HTTP botnet traffic. Since the used machine learning to differentiate P2P detection technique only relies on netflow data, botnet traffic from benign P2P traffic. There the technique is not affected by encrypted testing showed P2P botnet traffic can be botnet traffic. The detection technique was distinguished from benign P2P traffic with low tested against the Zeus botnet and the Citadel false positive rates. botnet. Haddadi et al. tested the detection Garant and Lu (2013) reviewed existing technique with netflows containing all captured botnet detection techniques and determined traffic and with filtered netflows of only HTTP such were ineffective against unknown botnets traffic. The detection results with all traffic as well as botnets that employ encrypted ranged between 7% and 88% for true positive communications. Grant and Lu developed the detections and 1% to 16% for false positive

Page 18 © 2015 ADFSL A Survey of Botnet Detection Techniques by Command ... JDFSL V10N1 This work is licensed under a Creative Commons Attribution 4.0 International License. detections. When the HTTP filter was applied, communication and decentralized Haddadi et al. increased the true positive infrastructures. Finally, in 2012 detection detection rate to 85% and 97% for Zeus and techniques began to include the ability to Citadel traffic respectively. Furthermore, the detect encrypted communications. false positive detection rates decreased to 14% Previous botnet detection techniques have and 3%. reported varied success rates for botnet detection and rates of false positive detections. 4. COMPARISON OF In 2007 and 2008 true positive detection rates BOTNET DETECTION ranged between 95% and 96.8%, while false positive detection rates were between 0.049% TECHNIQUES BY and 0.0003%. (Gu et al., 2007; Gu, Perdisci, INFRASTRUCTURE et al., 2008). Between 2009 and 2014 true positive detection rates increased to between This section provides a comparison of botnet 99% and 100%, however false positive detection techniques. We have compared the detection rates also increased to a range of techniques based on the techniques ability to 0.0056% and 0.2% (Barthakur et al., 2012; detect IRC, HTTP, and P2P based botnets Francois et al., 2011; Haddadi et al., 2014; and whether the technique is effective against Wang and Yu, 2009; Zeng et al., 2010; Zhang encrypted botnet communications. Table 2 et al., 2011, Zhang et al., 2014). This survey provides a summary of the different techniques showed that while true positive detection rates detection ability. have increased, so have false positive detection Table 2 also shows the change in detection rates. The one exception to these results are techniques as botnets changed communication from Haddadi et al. (2014), were the true methods and infrastructures. Between 2005 positive detection rates decreased. Table 3 and 2007 researchers focused IRC and HTTP shows the true detection rates and false botnets that use a centralized command and positive detection rates for nine studies control server. Then in 2008 detection reviewed as part of this survey that provided techniques began to include P2P detection rates.

© 2015 ADFSL Page 19 JDFSL V10N1 A Survey of Botnet Detection Techniques by Command… This work is licensed under a Creative Commons Attribution 4.0 International License. Table 2 Detection Capabilities of Different Botnet Detection Techniques Researchers IRC HTTP P2P Encrypted Cooke, Jahanian and McPherson (2005) X Gu, Porras and Yegneswaran (2007) X Karasaridis, Rexford, and Hoeflin (2007) X Gu, Zhang, and Lee (2008) X X Gu, Perdisci et al. (2008) X X X Wang and Yu (2009) X X Nagaraja, Mittal, Hong, Caesar and Borisov (2010) X Zeng, Hu and Shin (2010) X X X Francois, Wang, Bronzi, State and Engel (2011) X Zhang, Perdisci, Lee, Sarfraz and Luo (2011) X Barthakur, Dahal and Ghose (2012) X X X Han, Chen, Xu and Liang (2012) X X Zhang (2012) X X X Ilavarasan and Muthumanickam (2012) X Zeng (2012) X X X X Li, Xie, Luo and Zhu (2013) X X X X Rossow and Dietrich (2013) X X X X Garant and Lu (2013) X X Zhang et al. (2014) X X Haddadi et al. (2014) X X

Table 3 Botnet Detection Rates Researchers True Positive Rate False Positive Rate Gu et al. (2007) 95.1% 0.049% Gu, Perdisci et al. (2008) 96.83% 0.0003% Wang and Yu (2009) 100% 0.0056% Zeng et al. (2010) 99.99% 0.16% Francois et al. (2011) 99% 0.1% Zhang et al. (2011) 100% 0.2% Barthakur et al. (2012) 99.01% 0.11% Zhang et al. (2014) 100% 0.2% Haddadi et al. (2014) Citadel 97% 3% Haddadi et al. (2014) Zeus 85% 14% Note: Gu, Perdisci et al. (2008) true detection rate is an average of 8 tests

Page 20 © 2015 ADFSL A Survey of Botnet Detection Techniques by Command ... JDFSL V10N1 This work is licensed under a Creative Commons Attribution 4.0 International License. An analysis of the botnet detection Library. The keywords used in the search techniques reviewed in the survey showed included botnet, distributed denial of service, that techniques which used machine learning malware, denial of service, botnet detection, and hierarchical clustering of flow data were botnet identification, and proactive botnet. more effective than techniques based on deep The review showed that botnets and botnet packet analysis or fingerprint analysis. The detection techniques are constantly evolving same was true for the efficiency and as Botmasters update and modify botnets to scalability of the techniques. Relying solely stay ahead of the latest botnet detection on Netflow data allows the techniques to techniques (Alhomoud et al., 2013; Garant & process large data sets, while maintaining Lu, 2013; Zargar, Joshi, & Tipper, 2013). high true positive detection rates and low Although IRC and HTTP botnets are still false positive rates. active, most new botnets use a decentralized infrastructure to avoid a single point of failure (Garant & Lu, 2013; Gu et al., 2009). 5. CONCLUSION Furthermore, a majority of botnets now utilize encrypted communications to avoid This survey examined the existing research detection (Garant & Lu, 2013; Gu et al., on botnet detection and distributed denial of 2009; Li, Xie et al., 2013; Rossow & Dietrich, service attacks in a chronological order. 2013). Therefore, modern botnet detection Literature was reviewed from numerous techniques attempt to detect botnet sources including scholarly journals, command and control communications conference papers, books, dissertations, and within network traffic through hierarchical government documents. The literature was clustering of flow data (Haddadi et al., 2014; obtained from numerous online databases Kheir & Wolley, 2013; Zhang et al., 2014). including, ProQuest, IEEE Computer Society Digital Library, ACM Digital Library, Google Scholar, and the IEEE Xplore Digital

Alhomoud, A., Awan, I., Disso, J., & Cao, L, & Qiu, X. (2013, July). Defense Younas, M. (2013). A next-generation against botnets: A formal definition and approach to combating botnets. a general framework. Proceedings of the Computer, 46(4), 62-66. Retrieved from 2013 IEEE Eighth International http://doi.ieeecomputersociety.org/10.11 Conference on Networking, Architecture, 09/MC.2013.67 and Storage, Xi’an, Shaanxi, China, 237- 241. Retrieved from Bailey, M., Cooke, E., Jahanian, F., Yunjing, http://doi.ieeecomputersociety.org/10.11 X., & Karir, M. (2009). A survey of 09/NAS.2013.37 botnet technology and defenses. Proceedings of the 2009 Conference for Cisco. (2014). Snort (Version 2.9.6.2) Homeland Security, Washington, DC, [Computer Software]. Retrieved from 299-304. Retrieved from http://www.snort.org/downloads http://dx.doi.org/10.1109/CATCH.2009. Cooke, E., Jahanian, F., & McPherson, D. 40 (2005, July). The zombie roundup: Bilge, L., Balzarotti, D., Robertson, W., Understanding, detecting, and disrupting Kirda, E., & Kruegel, C. (2012, botnets. Proceedings of the Steps to December). Disclosure: Detecting botnet Reducing Unwanted Traffic on the command and control servers through Internet Workshop 2005, Cambridge, large-scale Netflow analysis. Proceedings MA. Retrieved from of the 28th Annual Computer Security https://www.usenix.org/legacy/events/sr Applications Conference, New York, NY, uti05/tech/full_papers/cooke/cooke.pdf 129-138. Retrieved from Dean, J., & Ghemawat, S. (2004, December). http://dl.acm.org/citation.cfm?id=24209 MapReduce: Simplified data processing 69 th on large clusters. Proceedings of the 6 Brezo, F., Santos, I., Bringas, P., & Val, J. Symposium on Operating System Design (2011, Aug). Challenges and limitations and Implementation, San Francisco, CA, in current botnet detection. Proceedings 137-150. Retrieved from of the 22nd International Workshop on http://static.googleusercontent.com/exter Database and Expert Systems nal_content/untrusted_dlcp/research.go Applications, Toulouse, France, 95-101. ogle.com/en/us/archive/mapreduce- Retrieved from osdi04.pdf http://dx.doi.org/10.1109/DEXA.2011.19 Dittrich, D. (2012, April). So you want to Caglayan, A., Toothaker, M., Drapaeau, D., take over a botnet. Proceedings of the 5th & Burke, D. (2010, January). Behavioral USENIX Workshop on Large-Scale patterns of fast flux service networks. Exploits and Emergent Threats, LEET Proceedings of the 2010 43rd Hawaii ’12, San Jose, CA. Retrieved from International Conference on System https://www.usenix.org/system/files/con Sciences (HICSS), Honolulu, HI, 1-9. doi: ference/leet12/leet12-final23.pdf 10.1109/HICSS.2010.81 Feily, M., Shahrestani, A., & Ramadass, S. (2009, June). A survey of botnet and

Page 22 © 2015 ADFSL A Survey of Botnet Detection Techniques by Command ... JDFSL V10N1 This work is licensed under a Creative Commons Attribution 4.0 International License. botnet detection. Proceedings of the 2009 Gu, G., Porras, P., Yegneswaran, V., Fong, Third International Conference on M., & Lee, W. (2007, August). Emerging Security Information, Systems BotHunter: Detecting malware infection and Technologies, Athens, Glyfada, through IDS-driven dialog correlation. Greece, 268-273. Retrieved from Proceedings of the 16th USENEX http://doi.ieeecomputersociety.org/10.11 Security Symposium, Boston, MA. 09/SECURWARE.2009.48 Retrieved from https://www.usenix.org/legacy/events/se Francois, J., Wang, S., Bronzi, W., State, R., c07/tech/full_papers/gu/gu.pdf & Engel, T. (2011, November). BotCloud: Detecting botnets using Gu, G., Yegneswaran, V., Porras, P., Stoll, Mapreduce. Proceedings of the 2011 J., & Lee, W. (2009, December). Active IEEE International Workshop on botnet probing to identify obscure Information Forensics and Security, command and control channels. Iguazu Falls, Parana, Brazil, 1-6. Proceedings of the 2009 Annual Retrieved from Computer Security Applications http://dx.doi.org/10.1109/WIFS.2011.61 Conference, Honolulu, HI, 241-253. doi: 23125 10.1109/ACSAC.2009.30 Francois, J., Wang, S., State, R., & Engel, Gu, G., Zhang, J., & Lee, W. (2008, T. (2011). BotTrack: tracking botnets February). BotSinffer: Detecting botnet using NetFlow and PageRank. th command and control channels in Proceedings of the 10 International th network traffic. Proceedings of the 15 IFIP TC 6 Conference on Networking, Annual Network and Distributed System Heidelbert, Germany, 1-14. Retrieved Security Symposium, San Diego, CA. from Retrieved from http://dl.acm.org/citation.cfm?id=20087 http://www.isoc.org/isoc/conferences/nd 82 ss/08/papers/17_botsniffer_ Garant, D., & Lu, Wei. (2013). Mining detecting_botnet.pdf botnet behaviors on the large-sale web Haddadi, F., Morgan, J., Filho, E., & Zincir- application community. Proceedings of Heywood, A. (2014). Botnet behaviour the 2013 27th International Conference analysis using IP flows: With HTTP on Advanced Information Networking filters using classifiers. Proceedings of the and Applications Workshops, Barcelona, 28th International Conference on Spain, 185-190. Retrieved from Advanced Information Networking and http://doi.ieeecomputersociety.org/10.11 Applications Workshops, Victoria, British 09/WAINA.2013.235 Columbia, 7-12. Retrieved from Gu, G., Perdisci, R., Zhang, J., & Lee, W. http://dx.doi.org/10.1109/WAINA.2014. (2008, July). BotMiner: Clustering 19 analysis of network traffic for protocol Hadoop. (2013). The Apache Hadoop project. and structure independent botnet Retrieved from detection. Proceedings of the 17th http://hadoop.apache.org/ USENEX Security Symposium, San Jose, CA. Retrieved from Han, F., Chen, Z., Xu, H., & Liang, Y. https://www.usenix.org/legacy/event/sec (2012, June). Garlic: A distributed 08/tech/full_papers/gu/gu.pdf botnets suppression system. Proceedings

© 2015 ADFSL Page 23 JDFSL V10N1 A Survey of Botnet Detection Techniques by Command… This work is licensed under a Creative Commons Attribution 4.0 International License. of the 2012 32nd International Conference Roscini, M. (2014). Cyber operations and the on Distributed Computing Systems use of force in international law. New Workshops, Macau, China, 634-639. York, NY: Oxford University Press Retrieved from Rossow, C., & Dietrich, C. (2013, July). http://doi.ieeecomputersociety.org/10.11 PROVEX: Detecting botnets with 09/ICDCSW.2012.30 encrypted command and control Hasan, A., Awadi, R., & Belaton, B. (2013). channels. Proceedings of the 10th Multi-phase IRC botnet and botnet International Conference on Detection of behavior detection model. International Intrusions and Malware, and Journal of Computer Applications, Vulnerability Assessment, Berlin, 66(15), 41-51. doi: 10.5120/11164-6289 Heidelberg, 21-40. Retrieved from http://dx.doi.org/10.1007/978-3-642- Householder, A., & Danyliw, R. (2003, 39235-1_2 March). Increased activity targeting windows shares (CERT advisory CA- Spitzner, L. (2003). The honeynet project: 2003-08). Retrieved from Trapping the hackers. IEEE Security & http://www.cert.org/advisories/CA-2003- Privacy, 1(2), 15-23. 08.html doi: 10.1109/MSECP.2003.1193207 Karasaridis, A., Rexford, B., & Hoeflin, D. Ventre, D. (2013). Cyber Conflict: (2007, April). Wide-scale botnet Competing National Perspectives. detection and characterization. Indianapolis, IN: Wiley. Wang, T., & Yu, Proceedings of the First Workshop on S. (2009). Centralized botnet detection Hot Topics in Understanding Botnets, by traffic aggregation. Proceedings of the Cambridge, MA. Retrieved from 2009 IEEE International Symposium on https://www.usenix.org/legacy/event/ho Parallel and Distributed Processing with tbots07/tech/full_papers/karasaridis/kar Applications, Chengdu, China, 86-93. asaridis.pdf Retrieved from http://dx.doi.org/10.1109/ISPA.2009.74 Kheir, N., & Wolley, C. (2013). BotSuer: Suing stealthy P2P bots in network Zargar, S., Joshi, J., & Tipper, D. (2013). A traffic through Netflow analysis. In M. survey of defense mechanisms against Abdalla, C. Nita-Rotaru, & R. Dahab distributed denial of service (distributed (Eds.), Cryptology and Network Security denial of service) flooding attacks. IEEE (pp. 162-178). doi: 10.1007/978-3-319- Communications Surveys and Tutorials, 02937-5_9 PP(99), 1-24. Li, W., Xie, S., Luo, J., & Zhu, X. (2013, doi: 10.1109/SURV.2013.031413.00127 April). A detection method for botnet Zeng, Y. (2012). On detection of current and based on behavior features. Proceedings nd next-generation botnets (Doctoral of the 2 International Conference on dissertation). University of Michigan. Systems Engineering and Modeling Retrieved from (ICSEM-13), Beijing, China, 512-517. http://deepblue.lib.umich.edu/handle/20 Retrieved from http://www.atlantis- 27.42/91382 press.com/php/download_paper.php?id= 5594 Zeng, Y., Hu, X., & Shin, K. (2010, June). Detection of botnets using combined host

Page 24 © 2015 ADFSL A Survey of Botnet Detection Techniques by Command ... JDFSL V10N1 This work is licensed under a Creative Commons Attribution 4.0 International License. and network level information. University of Mannheim Technical Proceedings of the 2010 IEEE/IFIP Report. Retrieved from International Conference on Dependable https://ub-madoc.bib.uni- Systems and Networks, Chicago, IL, 291- mannheim.de/1710/1/botnet_china_TR. 300. Retrieved from pdf http://doi.ieeecomputersociety.org/10.11 09/DSN.2010.5544306 Zhang, J. (2012). Effective and scalable botnet detection in network traffic. (Doctoral Dissertation). Retrieved from ProQuest Dissertations and Theses database. (AAT 1115317916) Zhang, J., Perdisci, R., Lee, W., Sarfraz, U., & Luo, X. (2011, June). Detecting stealthy P2P botnets using statistical traffic fingerprints. Proceedings of the 2011 IEEE/IFIP 41st International Conference on Dependable Systems and Networks, Hong Kong, China, 121-132. Retrieved from http://doi.ieeecomputersociety.org/10.11 09/DSN.2011.5958212 Zhang, J., Perdisci, R., Lee, W., Luo, X., & Sarfraz, U. (2014). Building a scalable system for stealthy P2P-Botnet detection. IEEE Transactions on Information Forensics and Security, 9(1), 27-38. Retrieved from http://dx.doi.org/10.1109/TIFS.2013.229 0197 Zhu, Z., Lu, G., Chen, Y., Fu, Z., Roberts, P., & Han, K. (2008) Botnet research survey. Proceedings of the 32nd Annual IEEE, International Computer Software and Applications, Turku, Finland, 967– 972. Retrieved from http://dx.doi.org/10.1109/COMPSAC.20 08.205 Zhuge, J., Holz, T., Han, X., Guo, J., & Zou, W. (2007, December). Characterizing the IRC-Based Botnet Phenomenon. Peking University and