Journal of JAITC, Vol. 9, No. 1, pp.1-10, Jul. 31, 2019 1 http://dx.doi.org/10.14801/JAITC.2019.9.1.1 Effective Detecting Method of Nmap Idle Scan Jungsik Hwang1, Minsoo Kim2* 1Interdisciplinary Program of Information & Protection Graduate School, Mokpo National University 2Dept. of Information Security, Mokpo National University [email protected], https://orcid.org/0000-0003-2782-1514, [email protected], https://orcid.org/0000-0001-7056-1952, *(Corresponding Author) Abstract In recent years, information collection of attacks through stealth port scanning technology has become more sophisticated. The most commonly used Nmap port scanner supports a variety of stealth scanning technologies along with the existing scanning techniques. Nmap also supports Idle scan that is different from conventional stealth scans. This is a more sophisticated stealth scan technique by applying the SYN scan and ACK scan techniques. In previous studies, the detection of Idle scanning was on zombie system, but was not on victim system. In this paper, we propose an effective detection method of Idle scan on victim system. The Idle scanning is composed of two stages; they are probing the zombie and victim system and scanning the victim system. We analyzed the characteristics of the two stages. The characteristics, we captured, are that SYN and RST packets are different from normal packet. We applied them to detection method, then Idle scanning is detected effectively. Keywords: Nmap, Idle scan, port scanner, probe, zombie PC 1. Introduction A cyberattack usually starts with vulnerability scanning. Accordingly, vulnerability scanning methods have been developed. Scanned objects are diversified into OS, Network and service. After an attacker gathered various information through the scanning, he attacks using the information. According to the InfoSec Report provided by SANS [1] from 2016 to 2018, there is an increase in port scanning attacks on services. The graph in Figure 1 represents the number of scans for the service port numbers provided on the network. Port scan techniques have been developed in various ways. Many tools for such attacks have also been developed. The typical attack tools are Nmap port scanner and McAfee- developed SuperScan scanner. These tools not only provide a common connection scan to detect open ports, but also provide the stealth scan techniques to avoid logging on victim systems. Many countermeasures against such stealth scan techniques have been proposed. Received: Dec. 10, 2018 Revised: Jan. 24, 2019 Accepted: Jan. 27, 2019 pISSN 2234-3072/eISSN 2234-0963 Copyright ⓒ KIIT J. Hwang et al.; Effective Detecting Method of Nmap Idle Scan 2 Figure 1. SANS InfoSec report (PortScan) FIN scan, NULL scan, XMAS scan and ACK scan among the stealth scans use wrong TCP flags. The detection method of them is checking the TCP flags in TCP connection. In addition, the Idle scan [2] provided by Nmap performs a SYN port scan attack using a zombie PC. The victim system recognizes the zombie PC as an attacker and responds to it. The attacker can create new attack opportunities using the other zombie PC, even if the victim detects and blocks the zombie PC. Patel [3] proposed a detection method for Idle scan. But there is a high possibility of false positives due to network environment between zombie PC and victim. In this paper, we propose the effective detection method of Idle scan. To do this, we analyze the scanning process and packets of Nmap Idle scan. Nmap Idle scan was composed of two stages; the probing and scanning. We found the characteristics of the SYS/ACK packets in the probing stage and the RST packets of zombie PC against the scanning. Therefore, we propose a detection method for Nmap Idle scan by comparing SYN/ACK and RST packets with the characteristics that we found. 2. Related Work A port scan is an attack that detects an open port on a particular system using specific program or technique. Port scan attacks in Nmap have two basic operating principles: Positive scan and Negative scan [4]. Positive scan uses packets that are answered by the open ports of the system, which are SYN scan, ACK scan, Idle scan, Ping scan and Connect scan. Negative scan uses packets that are answered by the closed ports of the system, which are FIN scan, NULL scan and XMAS scan. 2.1 Port scan tools There are a variety of tools that support port scans. Nmap and SuperScan support stealth scans and normal port scans. 2.1.1 Nmap pISSN 2234-3072/eISSN 2234-0963 Copyright ⓒ KIIT Journal of JAITC, Vol. 9, No. 1, pp.1-10, Jul. 31, 2019 3 http://dx.doi.org/10.14801/JAITC.2019.9.1.1 Nmap is a scanning tool that has been under study since September 1997 [5]. The functions of Nmap are host detection, TCP/UDP port scanning, and OS(Operating System) detection. Nmap uses ping responses to detect hosts connected on the network. The port scan of Nmap lists open ports and services of target computers. Nmap detects the OS version using a different response for each OS and network devices. 2.1.2 SuperScan Superscan is offered by McAfee. SuperScan 3.0 and 4.0 can scan TCP and UDP ports such as Nmap. In addition, they can check the services using the ports and execute user queries. SuperScan4.0 additionally supports SYN scan. 2.2 Stealth Scan Detection The stealth scan [6] includes SYN, ACK, FIN, NULL, XMAS, and Idle scans. These technique is characterized by not leaving a log information on the victim system during the scanning process. So hackers often use the stealth scan to avoid leaving traces. Efficient Port Scan Detection Rules (EPSDR) [7] shows the method to detect SYN, FIN, NULL, and XMAS scans through the flag value of TCP packet header. EPSDR uses Snort to realize the method. For detecting these scans, EPSDR proposed Snort rules to check TCP flags of TCP packet header. EPSDR detects a scanning attack as SYN scan when Reserved bit 1, 2, and SYN flag are set. Likewise, EPSDR detects it as FIN scan when FIN flag is set and session is not established, NULL scan when all TCP flags are not set, and XMAS scan when FIN, PSH, and URG flags are set. Patel [3] proposed a method to detect Idle scan using Snort. The method is to check increasing IPid value, which is a characteristic of the idle scan. He explained that IPid value of the RST packet, which is the response packet from the zombie PC, increased by 2 during Idle scan. He collected RST packets from the zombie PC and compared IPid value of each other to detect Idle scan. However, if there are several sessions in the network, the IPid value of the RST packet can be changed by another session. This means that there is possibility of false alarms. 3. Analysis of Nmap Idle Scan Principle Nmap manually creates the packet to perform stealth scan, So Nmap packets have different characteristics from normal packets. Nmap Idle scan includes features of SYN and ACK scans. 3.1 Characteristics of SYN Scan Figure 2 shows the SYN scanning process. SYN scan uses half-open connection to target system [8]. In normal connection, the SYN packet is transmitted using connect system call. In step 3 in Figure 2, the packet is transmitted with raw socket made by Nmap instead of the system call. When raw sockets are used, the attacker does not leave any information in the own local TCP/IP stack [9]. Therefore, the attacker system judges that the SYN/ACK packet returned from the victim system is an unauthorized TCP handshake as in step 5 to 6. Because of this process, the system, instead of Nmap, sends RST packet back to the victim system. pISSN 2234-3072/eISSN 2234-0963 Copyright ⓒ KIIT J. Hwang et al.; Effective Detecting Method of Nmap Idle Scan 4 Figure 2. SYN scanning process Table 1 shows the packet characteristics of SYN scan supported by Nmap. The characteristics of the SYN scan packet are as follows: The length of the TCP header is 24 bytes, and the window size is set 1024 to 4096 in most cases. The TTL has a random value in the range of 37 to 59, and the TCP option value is fixed to \x02\x04\x05\xb4. The source port of SYN scan packets is a random number, and the destination port is an import service ports of the target. Table 1. Nmap SYN scan packet features SYN packet Packet option Explanation TCP length 24 Bytes Window size 1024 ~ 4096 Bytes TTL 37 ~ 59 Source port Random number Option \x02\x04\x05\xb4 TimeiInterval One packet in 0.01 seconds 3.2 Characteristics of ACK Scan Figure 3 shows the ACK scanning process. The attacker sends ACK packet to the victim system, then the victim regards the ACK packet as an inappropriate response. This is because there is no connection information with the attacker system in the victim system's local TCP/IP stack. pISSN 2234-3072/eISSN 2234-0963 Copyright ⓒ KIIT Journal of JAITC, Vol. 9, No. 1, pp.1-10, Jul. 31, 2019 5 http://dx.doi.org/10.14801/JAITC.2019.9.1.1 Figure 3. ACK scanning process 3.3 Characteristics of Idle Scan Idle scan is a stealth port scan using zombie PC. The zombie PC used for Idle scan is a system that is rarely used or is already occupied by an attacker. Idle scan can hide the position of the attacker by keeping the zombie PC in the middle. Therefore, the information of attacker does not remain in the victim system after scanning [2].