Validation of NMAP’s Network Behavior using

Daniel Cruz Ramírez Master of Engineering in Computer Engineering Dr. Jeffrey Duffany Electrical and Computer Engineering and Computer Science Department Polytechnic University of Puerto Rico

Abstract  NMAP is used to actively scan networks, but works fine against single hosts. networks using different ping techniques. There is NMAP runs on all major computer operating not much information available on how NMAP systems, and official binary packages are available works besides its website. Although the program for , Windows, and Mac OS X. In addition to states how it works, there is little validation of its the classic command-line NMAP executable, the functionality. Wireshark, a network protocol NMAP suite includes an advanced GUI and results analyzer, was used to validate these features in a viewer (ZeNMAP), a flexible data transfer, test system environment: ping scans, OS detection, redirection, and debugging tool (Ncat), a utility for including port scanning and version detection. comparing scan results (Ndiff), and a packet Among NMAP’s weaknesses, we find it relies on an generation and response analysis tool (Nping) [1]. OS Database that should be updated regularly to be NMAP possesses the following ping scanning able to detect new operating systems and that its techniques: scans produce a large number of packets, which  TCP SYN might cause detection of the scan in a properly  TCP ACK protected network environment. NMAP’s OS  UDP Database can also be used to simulate operating  ICMP systems for network scans, such as in a honeypot,  Ping sweep using a program called honeyd. Any scan in a  ARP foreign network environment should be  Broadcast corroborated with other tools, passively if possible. Key Terms  NMAP, Ping Scan, Remote OS WIRESHARK Detection, Wireshark. Wireshark is the world’s foremost and widely- NMAP used network protocol analyzer. It lets you see what’s happening on your network at a microscopic NMAP ("Network Mapper") is a free and open level and is the de facto (and often de jure) standard source (license) utility for network discovery and across many commercial and non-profit enterprises, security auditing. Many systems and network government agencies, and educational institutions. administrators also find it useful for tasks such as Wireshark development thrives thanks to the network inventory, managing service upgrade volunteer contributions of networking experts schedules, and monitoring host or service uptime. around the globe and is the continuation of a project NMAP uses raw IP packets in novel ways to started by Gerald Combs in 1998 [2]. determine what hosts are available on the network, Test Setup for Ping Scanning Techniques what services (application name and version) those hosts are offering, what operating systems (and OS There is little information regarding how versions) they are running, what type of packet NMAP works besides the documentation on filters/firewalls are in use, and dozens of other NMAP’s website. Although the program is open characteristics. It was designed to rapidly scan large source, no invasive study exists about how the program performs and what information is sends o Realtek RTL8101/2/6E PCI Express with each scan attempt. Fast/Gigabit Ethernet controller For this purpose, we setup a test network for . MAC Address – 5C:F9:DD:4F:0C:56 this project which includes a router, a Dell Inspiron . IP Address – 10.0.0.11 5720 laptop running Ubuntu Mate 16.10 x64 with  RAM: 12GB DDR3 10600 Linux kernel 4.8.0.30-generic, an Internet Router, All ping scanning techniques are done from the and a Raspberry Pi 3 Model B, running Raspbian Dell Laptop to the Raspberry Pi 3 Model B. To Jessie Lite and Linux kernel 4.4.34-v7+. Both minimize communication, the Dell Laptop has no operating systems updated as of December 12, gateway or DNS information and, whenever 2016. There are no other network objects. possible, there are no other programs running. A Raspberry Pi is a credit card-sized computer Both NMAP and Wireshark are run from the Dell originally designed for education, inspired by the laptop. Although NMAP has a GUI (ZENMAP), 1981 BBC Micro. The Raspberry Pi 3 is the third we have used the command line version only. generation Raspberry Pi. It has: Wireshark is used to analyze NMAP’s output.  A 1.2GHz 64-bit quad-core ARMv8 CPU  802.11n Wireless LAN (not used in the project)  Bluetooth 4.1  Bluetooth Low Energy (BLE)  1GB RAM  4 USB ports  40 GPIO pins  Full HDMI port  Ethernet port o MAC Address - B8:27:EB:CF:6C:77 o IP Address – 10.0.0.100  Combined 3.5mm audio jack and composite video  Camera interface (CSI) Figure 1 Test Network Setup  Display interface (DSI)  Micro SD card slot This project was started with NMAP version  VideoCore IV 3D graphics core [3] 6.40 and Wireshark version 2.2.3, distributed with Ubuntu Mate 16.10. As it progressed, newer The Dell Laptop’s specifications are: NMAP versions came out and were tested. Finally,  CPU: Intel Core i7-3632QM we settled on NMAP version 7.31, compiled from  Graphics: source. o Intel 3rd Gen Core Processor Graphics For the OS detection scans, although we have Controller primarily used the TEST SETUP, we have also o NVIDIA GF117M [GeForce tested several other systems, without any 610M/710M/810M/820M / GT modification to their LAN structure, to determine 620M/625M/630M/720M] NMAP’s efficiency.  Audio: Intel 7 Series/C210 Series Family High TCP SYN Ping Scan: map -sP -PS Definition Audio Controller  Network: For this type of ping scan, NMAP sends a TCP o Intel Centrino Wireless-N 2230 (disabled) SYN packet to port 80. If the port is closed, the host responds with an RST packet. If the port is open, response to errors in IP operations (as specified in the host responds with a TCP SYN/ACK packet RFC 1122). ICMP errors are directed to the source indicating that a connection can be established. IP address of the originating packet. ICMP ping Afterwards, an RST packet is sent to reset this scans use these types of packets to determine if a connection [1]. host is active or not [1]. For this scan, we expect to find a SYN/ACK For this type of scan, we expect to find ICMP packet from the source ip to the destination ip. ping responses from the target to the source of the scan, along with the time for the ping to return. TCP ACK Ping Scan: NMAP -sP -PA Ping Sweeps: NMAP -sP -PO This type of scan can be used to detect hosts that block SYN packets or ICMP echo requests, but This technique tries sending different packets it will most likely be blocked by modern firewalls using different IP protocols, hoping to get a that track connection states. NMAP sends an empty response indicating that a host is online. By TCP packet with the ACK flag set to port 80, if the default, this ping scan will use the protocols IGMP, host is offline, it should not respond to this request, IP-in-IP, and ICMP to try to obtain a response that if the host is online, it returns an RST packet, since will indicate that the host is online. Using --packet- the connection does not exist. TCP ACK ping trace will show more details of what happened scans need to run as a privileged user, otherwise a behind the curtains [1]. system call connect() is used to send an empty TCP For this type of scan, we expect to find a ping SYN packet [1]. responses from the target to the source of the scan, For this scan, we expect to find an ACK packet along with the time for the ping to return. to port 80 from the source IP to the destination IP. ARP Ping Scans: NMAP -sP -PR UDP Ping Scans: NMAP -sP -PU IPv4 devices must respond to ARP packets UDP is a minimal message-oriented transport even if the targeted device uses firewalls or other layer protocol that is documented in RFC 768. stealthy methods to hide from ICMP or UDP packet UDP provides no guarantees to the upper layer based ping tools. ARP, a non-routable protocol, protocol for message delivery and the UDP layer operates at OSI Layer 2. The IPv4 address of the retains no state of UDP messages once sent [4]. device must be known. In this scan, ARP requests UDP ping scans have the advantage of being are sent to the target, if the host responds with an capable of detecting systems behind firewalls with ARP reply, it’s online [1]. strict TCP filtering leaving the UDP traffic For this type of scan, we expect to find an arp forgotten. NMAP is one of a handful of programs responses from the target to the source of the scan. capable of doing a UDP ping scan. NMAP sends Broadcast Pings: NMAP -script broadcast-ping an empty UDP packet to ports 31 and 338. If the host is responding, it should return an ICMP port unreachable error. If the host is offline, various Broadcast pings send ICMP echo requests to ICMP error messages could be returned [1]. the local broadcast address, and then waiting for For this type of scan, we expect to find a UDP hosts to reply with an ICMP echo reply. This a nice broadcast packet from the source machine, way of discovering hosts in a network without expecting a response from the target machine. sending probes to the other hosts [1]. For this type of scan, we expect to find ICMP ICMP Ping Scans: NMAP -sP -PE echo requests throughout the local broadcast ICMP messages are typically used for address, along with ICMP echo replies. diagnostic or control purposes or generated in

Figure 2 Wireshark TCP SYN Ping Scan Results

Figure 3 Wireshark TCP ACK Ping Scan Results

Figure 4 Wireshark UDP Ping Scan Results

Figure 5 Wireshark ICMP Ping Scan Results

Figure 6 Wireshark Ping Sweep Scan Results

Figure 7 Wireshark ARP Ping Scan Results

Figure 8 Broadcast Ping Scan Results

tracked and resent at least once if there is no OS Detection using NMAP: NMAP -O -v response. All of the packets are IPv4 with a random IP ID value. Probes to an open TCP port are NMAP OS fingerprinting works by sending up skipped if no such port has been found. For closed to 16 TCP, UDP, and ICMP probes to known open TCP or UDP ports, NMAP will first check if such a and closed ports of the target machine. These port has been found. If not, NMAP will just pick a probes are specially designed to exploit various port at random and hope for the best [5]. ambiguities in the standard protocol RFCs. Then NMAP then compares the results to its OS NMAP listens for responses. Dozens of attributes in database and prints out the OS details if there is a those responses are analyzed and combined to match. Each fingerprint includes a freeform textual generate a fingerprint. Every probe packet is description of the OS, and a classification which provides the vendor name, underlying OS, OS same, that makes the results more credible. If they generation, and device type [6]. come out wildly different, further investigation OS detection enables some other tests which must determine what is going on before relying on make use of information that is gathered anyway either. Since OS and version detection go together during the process: so well, the -A option enables them both [6].  Device type – if several device types are When NMAP performs OS detection against a shown, they will be separated with the pipe target and fails to find a perfect match, it usually symbol. repeats the attempt: by default, it tries five times if  Running - shows the OS Family and OS conditions are favorable for OS fingerprint generation if available. submission, and twice when conditions aren't so  OS CPE - shows a Common Platform good [6]. Enumeration (CPE) representation of the Besides the OS Database, NMAP also has a when available. MAC Address database which maps MAC address  OS details - gives the detailed description for prefixes to vendor names. Ethernet devices are each each fingerprint that matches. programmed with a unique 48-bit identifier known as a MAC address. This address is placed in Uptime guess. Ethernet headers to identify which machine on a  Network Distance – how many routers are local network sent a packet, and which machine the between it and a target host? The distance is packet is destined for. To assure that MAC zero when you are scanning localhost, and one addresses are unique in a world with thousands of for a machine on the same network segment. vendors, the IEEE assigns an Organizationally Each additional router on the path adds one to Unique Identifier (OUI) to each company the hop count. manufacturing Ethernet devices. The company must  TCP Sequence Prediction - Systems with poor use its own OUI for the first three bytes of MAC TCP initial sequence number generation are addresses for equipment it produces. It can choose vulnerable to blind TCP spoofing attacks. In the remaining three bytes however it wishes, as other words, you can make a full connection to long as they are unique. NMAP can determine the those systems and send (but not receive) data MAC address of hosts on a local Ethernet LAN by while spoofing a different IP address. reading the headers off the wire. It uses this  IP ID sequence generation - This field database to look up and report the manufacturer describes the ID generation algorithm that name based on the OUI. This can be useful for NMAP was able to discern [6]. roughly identifying the type of machine being dealt Interrogating open ports for clues is another with [1]. effective approach for OS fingerprinting. Some Version Detection using NMAP: NMAP -O -sV applications, such as Microsoft IIS, only run on a single platform, while many other apps divulge Version detection is one of the most popular their platform in overly verbose banner messages. features of NMAP. Knowing the exact version of a Adding the -sV option enables NMAP version service is highly valuable for penetration testers detection, which is trained to look for these clues, who use this service to look for security among others [6]. vulnerabilities, and for system administrators who TCP/IP fingerprinting will identify the proxy, wish to monitor their networks for any while version scanning will generally detect the unauthorized changes. Fingerprinting a service may server running the proxied application. Even when also reveal additional information about a target, no proxying or port forwarding is involved, using both techniques is beneficial. If they come out the such as available modules and specific protocol One reason for why the TTL and window size information [1]. values varies between different OS's is because the NMAP has a special flag to activate aggressive RFC's for TCP and IP do not require detection, -A. Aggressive mode enables OS implementations to use any particular default value detection (-O), version detection (-sV), script for these fields. There is, however, a scanning (-sC), and traceroute (--traceroute). This recommendation in RFC 1700 saying: “The current mode sends a lot more probes and it is more likely recommended default time to live (TTL) for the to be detected, but provides a lot of valuable host Internet Protocol (IP) is 64.” This recommendation information [1]. is obviously not followed in many IP Results ultimately come from the target implementations [7]. machine itself. Numerous reconnaissance methods Basically, from a simple ping, if ttl=64, the to explore a network should be used to confirm the system pinged is Linux/Unix-based; else if ttl=128, information needed, you should not trust only one the system pinged is Microsoft based; else if of them [1]. ttl=255, the system pinged is CISCO based. Current Apple Operating Systems are based on OS Detection using a Simple Ping Unix. [9] Below are some typical initial TTL values and OS Detection on Project System window sizes of common operating systems: OS Detection of the Raspberry Pi 3 failed with Table 1 TTL and TCP Window Size for Different Operating Systems NMAP version 6.40. I believe this to be because [7] the shipping version of NMAP in Ubuntu Mate Operating System (OS) IP Initial TCP window 16.10 was released on July 30, 2013 [1], before the TTL size introduction of the Raspberry Pi 3 (February 29, 2016) [3]. Linux (kernel 2.4 and 64 5840 With version 7.30 (released on September 29, 2.6) 2016) [1], however, OS Detection of the Raspberry Google's Android and 64 5720 Pi 3 takes about 15 seconds. It correctly identifies Chrome OS the operating system as a Linux 3.2 – 4.4. It does this with a combination of several methods: port FreeBSD 64 65535 scanning, arp ping response time, and MAC Address identification. In this case, since Windows XP 128 65535 Raspberry Pis only run either Linux or Windows 10 IoT Core [3], the program uses the open ports and Windows 7, Vista and 128 8192 Server 2008 the ping response to correctly identify the operating system. Cisco Router (IOS 12.4) 255 4128 NMAP Output: Starting NMAP 7.31 (https://NMAP.org) at MacOS/MacTCP X 64 2017-01-16 13:43 AST (10.5.6) NMAP scan report for 10.0.0.100 Host is up (0.0044s latency). In Windows Vista and 2008 server, Microsoft Not shown: 998 closed ports introduced a new TCP/IP stack with a number of PORT STATE SERVICE VERSION improvements. It also includes a concept called 53/tcp open domain TCP Window "Auto-Tuning" that's been used in MAC Address: B8:27:EB:CF:6C:77 Linux for years [8]. (Raspberry Pi Foundation) Device type: general purpose Aggressive OS guesses: Running: Linux 3.X|4.X 8.1 R1 (92%), Microsoft Windows Phone 7.5 or 8.0 OS CPE: cpe:/o:linux:linux_kernel:3 (92%), Microsoft Windows Server 2008 or 2008 cpe:/o:linux:linux_kernel:4 Beta 3 (92%), Microsoft Windows Server 2008 R2 OS details: Linux 3.2 - 4.4 or Windows 8.1 (92%), Microsoft Windows 7 Network Distance: 1 hop Professional or Windows 8 (92%), Microsoft Windows Vista SP0 or SP1, Windows Server 2008 OS Detection on Other Systems SP1, or Windows 7 (92%), Microsoft Windows NMAP version 7.31 was tested on several Vista SP2, Windows 7 SP1, or Windows Server systems, identifying them correctly: Windows 2008 (92%), Microsoft Windows Embedded 2003 R2, Windows 2008 R2, Windows 10 Standard 7 (92%), Microsoft Windows 7 (90%), Professional, Windows 7 Professional, Linux kernel Microsoft Windows Server 2008 SP1 (89%) XXX, among others. The only time NMAP failed No exact OS matches for host (test conditions was against a Windows 2016 Data Center Edition non-ideal).” Virtual Machine, in which it guesses as being: NMAP’s OS Database must be continually Microsoft Windows Server 2012 or Windows updated to be able to detect new systems released. Server 2012 R2 (93%), Microsoft Windows Server This can be done by downloading the database 2012 R2 (88%), Microsoft Windows 10 build directly or recompiling from the sources from the 10586 - 14393 (87%), Microsoft Windows 7 project’s Github Page [10]. Professional (87%), Microsoft Windows Phone 7.5 or 8.0 (86%), Microsoft Windows 10 build 10586 Honeyd & Honeypots (86%), Microsoft Windows Server 2008 R2 or NMAP’s OS database can be used with other Windows 8.1 (86%), Microsoft Windows 7 purposes, such as creating virtual hosts on a Professional or Windows 8 (86%), Microsoft network with other programs (a honeypot), such as Windows Vista SP0 or SP1, Windows Server 2008 HONEYD. SP1, or Windows 7 (86%), Microsoft Windows Honeyd is a small daemon that creates virtual Vista SP2, Windows 7 SP1, or Windows Server hosts on a network. The hosts can be configured to 2008 (86%). Since it detects it as both Windows 10 run arbitrary services, and their personality can be and Windows Server 2012, you could extrapolate adapted so that they appear to be running certain that the system is the Server version of Windows operating systems. Honeyd enables a single host to 10. Windows Server 2016 was released on claim multiple addresses. Honeyd improves cyber September 26, 2016. security by providing mechanisms for threat If the conditions are not ideal, NMAP will not detection and assessment. It also deters adversaries detect correctly the operating system. We added a by hiding real systems in the middle of virtual Windows 10 system into out test setup. If we setup systems. Honeyd can mimic several operating the LAN connection as private, NMAP version 7.31 systems within a LAN from reading a NMAP detected the correct operating system. If the LAN fingerprint file. The configured personality is the connection was setup as Public (affecting the operating system that NMAP will return. ), NMAP version 7.31 was unable to detect Personalities can be annotated to determine if they the operating system correctly and gave us the allow FIN-scans for open ports or to select the following message and the end of the scan: preference in which they reassemble fragmented IP “Warning: OSScan results may be unreliable packets [11]. because we could not find at least 1 open and 1 closed port.

Figure 9 Test System Detection Start

Figure 10 Test System Detection Stop A honeypot is a closely monitored computing Its OS detection database is extremely useful resource intended to be probed, attacked, or for scans and also for other applications such as compromised. The value of a honeypot is honeyd, which might be used to confuse would-be determined by the information that we can obtain infiltrators by using it to create the illusion of other from it. Monitoring the data that enters and leaves a computer systems in an internal LAN. However, honeypot allows us to gather information that is not newer systems are not identified until NMAP available to Network Intrusion Detection Systems updates its database. You might be able to [11]. extrapolate the result from a new system if you We downloaded the honeyd version 1.6d have some knowledge it is being used and from source from its github repository [12] and compiled discrepancies in the NMAP results. it in the Raspberry Pi 3 of our test system setup. Also, conditions must be ideal for OS detection We were able to successfully create virtual hosts to work correctly, or else the program might guess with services on our test network as follows: with whatever information it received from the  An HP Microsoft Windows 10 with IP address target. 10.0.0.14 and ports 139, 445 (File sharing), Either way, if running NMAP within an 137 (netbios), and 112 open unknown environment, it is always good practice to  An Intel Apple macOS 10.12 (Sierra) (Darwin run similar tools to corroborate the information 16.0.0) with IP address 10.0.0.15 and ports 21 NMAP presented. (FTP) and 23 (TELNET) open NMAP has to send packets at the potential  A Dell Linux Computer with IP address target to get a read on their attributes, risking 10.0.0.16 and ports 22 (SSH), 23 (TELNET), detection. Whenever packets are sent to a target, 56 (DNS), and 80 (HTTP) open your IP address is attached (unless you spoof your IP address, but then you wouldn't get a response  A CISCO OpenWRT Router with IP address from the target). In the interest of remaining as 10.0.0.1 and ports 22 (SSH) and 23 (TELNET) stealthy as possible, you want to be able to open. determine the operating system of a potential target These hosts were created by adding the without touching it. This can be accomplished with corresponding signatures from the NMAP database other programs like p0f, which uses attributes of the into the database used by HONEYD. We were able packets on the wire to determine the operating to fool NMAP into identifying these virtual hosts system that sent the packet. Packets are captured systems as real systems on a network, since it’s with a tool like Wireshark and then analyzed with a using the same information that NMAP uses to passive tool, such as P0F. P0F passively listens to identify them. the network traffic without creating any extra packets. It determines the operating system of the CONCLUSIONS remote host by analyzing certain fields in the NMAP does its job as stated. It produces a lot captured packets [13]. Due to this passive analysis, of packets on some scans, so it may be detectable in the remote system will not be able to detect the a network setting if the looking for it. Some of its packet capture, such as p0f for example [14]. scan methods are unique and, if it is not being run However, the reliability of passive recon is lower by an authorized party, may be attributed to a than active recon [15]. hacking attempt. Its various ping methods go Passive fingerprinting could also be used to beyond the standard ping in an attempt to get a sniff TCP/IP ports, rather than generating network traffic by sending packets to them. Hence, it’s a response from the target independent of network more effective way of avoiding detection or being conditions. stopped by a firewall [16]. REFERENCES [15] OccupytheWeb. (2014, February 27). How to conduct Passive OS Fingerprinting with Pof [Online]. Available: [1] NMAP Website [Online]. Available: http://www.nmap.org. http://null-byte.wonderhowto.com/how-to/hack-like-pro- [Accessed: Feb. 1, 2017]. conduct-passive-os-fingerprinting-with-p0f-0151191/. [Accessed: Feb. 1, 2016]. [2] Wireshark Website. Available: http://www.wireshark.org. [Online, accessed: Feb. 1, 2017]. [16] The InfoSec Institute. (2014, June 19). What You Must Know About OS Fingerptinting [Online]. Available: [3] Raspberry Pi Website. [Online]. Available: http://resources.infosecinstitute.com/must-know-os- https://www.raspberrypi.org. [Accessed: Feb. 1, 2016]. fingerprinting/. [Accessed: Feb. 1, 2016]. [4] Wikipedia. 2017, January 13, User Datagram Protocol. [Online]. Available: https://en.wikipedia.org/wiki/User_ Datagram_Protocol. [Accessed: Feb. 1, 2016].

[5] Gordon “Fyodor” Lyon. (2011). “NMAP OS Fingerprinting” in The Official Nmap Project Guide to Network Discovery and Security Scanning, Free Edition [Online]. Available: https://nmap.org/book/osdetect- methods.html. [Accessed: Feb. 1, 2016].

[6] Gordon “Fyodor” Lyon. (2011). “NMAP OS Detect” in The Official Nmap Project Guide to Network Discovery and Security Scanning, Free Edition [Online]. Available: https://nmap.org/book/osdetect.html. [Accessed: Feb. 1, 2016].

[7] Erick Hjelmvik. (2011, November 5). Passive OS Fingerprinting. [Online]. Available: http://www.netresec.com/?page=Blog&month=2011- 11&post=Passive-OS-Fingerprinting. [Accessed: Feb. 1, 2016].

[8] Philip. (2009, December 13). The TCP Windows, Latency, and the Bandwidth Delay product. [Online]. Available: http://www.speedguide.net/articles/the-tcp-window- latency-and-the-bandwidth-delay-2678. [Accessed: Feb. 1, 2016].

[9] Wikipedia. (2017, February 1). Macinstosh Operating Systems. [Online]. Available: https://en.wikipedia.org/ wiki/Macintosh_operating_systems. [Accessed: Feb. 1, 2016].

[10] NMAP Github Repository. [Online]. Available: https://github.com/nmap/nmap. [Accessed: Feb. 1, 2016].

[11] Honeyd.org Webpage. [Online]. Available: http://www.honeyd.org. [Accessed: Feb. 1, 2016].

[12] Honeyd Github Page. [Online]. Available: https://github.com/DataSoft/Honeyd/. [Accessed: Feb. 1, 2016].

[13] M. Zalewski. (2012-2014). Pof Website, V3 [Online]. Available: http://lcamtuf.coredump.cx/p0f3/. [Accessed: Feb. 1, 2016].

[14] S. Pillai. (2012, December 29). Fingerprinting-Detect Remote Operating Systems [Online]. Available: http://www.slashroot.in/fingerprinting-detect-remote- operating-system. [Accessed: Feb. 1, 2016].