The 10th Conference for Informatics and Information Technology (CIIT 2013) The 10 th Conference for Informatics and Information Technology (CIIT 2013) EVALUATION AND TESTING OF SEVERAL FREE/OPEN SOURCE WEB VULNERABILITY SCANNERS Nataša Šuteva Dragi Zlatkovski, Aleksandra Mileva Faculty of Computer Science, UGD Faculty of Computer Science, UGD Štip, Macedonia Štip, Macedonia ABSTRACT significant number of vulnerabilities in test applications [1, 4, 12, 14, 15, 22]. Bau et al [1], testing eight WVSs, showed that Web Vulnerability Scanners (WVSs) are software tools for WVSs need to be improved in detection of the “stored” and identifying vulnerabilities in web applications. There are second-order forms of XSS and SQLI, and in understanding commercial WVSs, free/open source WVSs, and some of active content and scripting languages. Khoury [7, 8] companies offer them as a Software-as-a-Service. In this analyzed three state-of –art black box WVSs against stored paper, we test and evaluate six free/open source WVSs using SQLI, and their results showed that stored (persistent) SQLI the web application WackoPicko with many known are not detected even when these automated scanners are vulnerabilities, primary for false negative rates. taught to exploit the vulnerability. They propose also a set of recommendations for increasing a detection rate in WVSs for I. INTRODUCTION this type of vulnerability. Doupé et al [4] tested eleven WVSs, Our everyday live heavily depends on using different web and found that eight out of sixteen vulnerabilities were not applications, as web e-mail clients, web instant messaging detected by any of the used scanners. They discuss also a clients, Voice over IP services, e-learning portals, social critical limitations of current WVSs, lack of better support for networks, electronic banking, e-commerce platforms, etc. well known, pervasive technologies as JavaScript and Flash, Because of this, the web applications became the most and the need for more sophisticated algorithms to perform interest target for attackers to gain an unauthorized account “deep” crawling and track the state of the application under access, steal sensitive data and identity, etc. test. The OWASP (Open Web Application Security Project) Kals et al [6] implement an automated black box scanner Top Ten 2013 [16] offers a list of the most critical Web SecuBat which targets XSS and SQLI vulnerabilities. application vulnerabilities, including different types of McAllister et al [10] also implement an automated black box injection, broken authentication and session management, scanner which targets reflected and stored XSS utilizing user cross-site scripting, cross-site request forgery, etc. This list is interactions. Maggi et al [9] discuss techniques applicable to often used also as a minimum standard for website black box testing, for reducing the number of false positives. vulnerability assessment and PCI compliance according to Fonseca et al [5] evaluated the XSS and SQLI detection Payment Card Industry Data Security Standard (PCI DSS). performance of three WVSs via automated software fault- Classification of web application vulnerabilities can be found injection methods. also in Common Vulnerabilities and Exposures database [3] For evaluating and testing WVSs, vulnerable test and Web Application Security Consortium (WASC) Threat applications are needed. These applications need to have Classification v2.0 [20]. exactly listed known vulnerabilities, so one can obtain the Web Application Security Scanners (WASSs) or Web false positive and false negative rates also. Unfortunately, no Vulnerability Scanners (WVSs) are software tools for standard test suite is currently available. There are several identifying potential vulnerabilities in the web applications, well-known, publicly-available, vulnerable web applications independently of the particular technology used for their like DVWA (Dam Vulnerable Web Application) [13] and implementation. They access the web applications in the same WebGoat [17], but their design is focused more on teaching manner as user do, through the web front-end. Usually they web application security rather than testing WVSs. The are black-box testers, because they do not have access to the exception is the realistic and fully functional web application source code, so they detect vulnerabilities by actually WackoPicko [21] with 16 known vulnerabilities, created by performing attacks or by looking for known vulnerabilities and A. Doupé, and used in [4] for their testing. We use this web report potential exposures. application for our experiments. Additionally, WASC [19] The beauty of WVSs hides in automatically and cost- has published evaluation criteria for web application scanners. effective conduction of security checks and production of the Because most of the research papers are concentrated on final report. Almost every report includes a remedy for found commercial WVSs, we decided to test and evaluate only vulnerability, which is necessary for PCI compliance. Today free/open source WVSs. After Introduction Section, Section there are more than 130 scanning vendors approved for PCI II is devoted to basic architecture of the black box WVSs. In compliance [11]. Vulnerability scanning is essential part of Section III we give brief explanation of used testbed maintaining security in a given organization and should be used application, used six WVSs with their general characteristics continuously, especially when new version of web application or and input vector support, followed by used methodology and new equipment or technology is planning to use. But WVSs are obtained results on the false negative rates at the first place. not all-in-one oracles, they are not capable of detecting all of At the end, we give short concluding remarks. the possible vulnerabilities and attack vectors that exist. There are several reports showing that today WVSs fail to detect a ©2013 Faculty of Computer Science and Engineering 221 The 10th Conference for Informatics and Information Technology (CIIT 2013) The 10 th Conference for Informatics and Information Technology (CIIT 2013) II. BLACK BOX WEB VULNERABILITY SCANNERS 2012 run only on Windows, and other four can be installed on Linux and OS X also. W3Af additionally is available on Conceptually, almost all WVSs consist of three main FreeBSD and Open BCD. Their input vector support are components: a crawling component, an attacker component, given on Table 2. Many different characteristic comparisons and an analysis component. on older versions of these WVSs can b e found on Chen’s web At the beginning of the scanning process, the user enters site SecToolMarket [2]. at least one URL, with or without user credentials for the given web application. Using these data, the crawling Table 1: General characteristics of the evaluated scanners component identifies all the reachable pages in the application, and all the input points to the application, such as the parameters of GET requests, the input fields of HTML Iron Free ZAP 2012 Stalker Vega W3Af WASP Edition OWASP forms, etc. After user sets the scanning profile, scanners can N- NetSparker proceed automatically or with user interaction. We used only Community Company/ Mavituna W3Af L. N-Stalker OWASP Subgraph automated mode for our experiments. Creator Security Devel. Kuppan The attacker component analyzes discovered data and for 1.0 Version 2.5 7.1.1.126 2.0.0 1.2 -r6654 0.9.5.0 each web form, for each input and for each vulnerability type (beta) Freeware ASF2 GPL2 GPL3 Li cence/ Freeware EPL1 for which the WVS has test vectors, the attacker module Unknown Java Python .Net 2.0 Technology .Net 3.5 Java 1.6.x generates values that are likely to trigger a vulnerability. (Win32) 1.6.x 2.6.x SP2 Windows Then, the form content is send to the web server using either a Windows Linux Windows Windows Operating GET or POST request, and appropriate response is obtained Windows Windows Linux OS X Linux Linux System from the server via HTTP. OS X FreeBSD OS X OS X OpenBSD Next, the analysis module has to parse and interpret the Authent. Yes Yes Yes server response. Decision if a given attack was successful is Report Yes Yes Yes made by calculation of confidence value, by using attack- Scan Log Yes Yes Yes Yes Yes specific response criteria and keywords. NetSparker Community Edition have many features disabled, compared to its commercial version, but still you can scan and exploit SQL injection vulnerabilities without III. EXPERIMENTS AND RESULTS any false-positives. N-Stalker Free 2012 provides a restricted set of features, A. Vulnerable web application compared to its commercial version, and will inspect up to Vulnerable WackoPicko application is a photo sharing and 500 pages within target application. photo-purchasing site. Users of WackoPicko can upload OWASP Zed Attack Proxy (ZAP) is an easy to use photos, browse other user’s photos, comment on photos, and integrated scanning and penetration testing tool, and it is purchase the rights to a high-quality version of a photo. It has designed to be used by people with a wide range of security 10 vulnerabilities accessible without authentication (reflected experience . and stored XSS, reflected XSS behind JavaScript, predictable Session ID for admin, weak admin password, reflected SQLI, Table 2: Supporting input vectors by the evaluated scanners command line injection, file inclusion, unauthorized file exposure, and parameter manipulation), and 6 vulnerabilities accessible after logging into the