A Survey on Web Penetration Test

ACSIJ Advances in Computer Science: an International Journal, Vol. 3, Issue 6, No.12 , November 2014 ISSN : 2322-5157 www.ACSIJ.org

A survey on web penetration test

Mahin Mirjalili1, Alireza Nowroozi2, Mitra Alidoosti3

1Department of information security, Malek-Ashtar university of technology, Tehran, Iran [email protected]

2Department of information security, Malek-Ashtar university of technology, Tehran, Iran [email protected]

3Department of information security, Malek-Ashtar university of technology, Tehran, Iran [email protected]

Abstract - How the penetration test is performed? This paper reviews the penetration test specifically in the - What are types of penetration test? field of web. For this purpose, it first reviews articles generally on penetration test and its associated methods. - How the penetration test is done automatically? Then articles in the field of web penetration test are - What tools can we use to perform an automatic examined in three aspects: comparing automatic penetration test tools, introduction of new methods or tools penetration test? for manual penetration test, and articles that presented a test - Comparison of tools and their effectiveness environment for training or checking various instruments and methods. This article studied 4 different methodologies - What are the new tools and methods and what for web penetration test, 13 articles for comparing web are their features? vulnerability scanners, 10 articles that proposed a new method or tool for penetration test and 4 test environments. - How can we examine various tools and techniques? Keywords: Penetration test, web scanner, web This paper attempts to answer these questions by application, web vulnerabilities. examining 4 different methodologies for penetration test, 13 articles for comparing web vulnerability 1. Introduction scanners, 10 articles that proposed a new method or Penetration test is a security evaluation process for tool for penetration test and 4 test environments. network or computer systems that simulates an attack by an ethical hacker. The most important distinction Due to the large volume of papers in the studied area, between a hacker and a penetration tester is that some criteria were used for paper selection such as penetration test is done with a license and a signed covering a wide time period from 2006 to 2014 and contract with an organization or company, and the the number of citations per paper. Most selected output is provided as a report. The goal of penetration articles were considered by many authors in previous test is to increase data security. Security information years. and weaknesses that are specified in penetration test We will initially review articles that provided a are considered confidential and shall not be disclosed method for the penetration test, then papers in the until complete resolution of defects. field of web penetration test in three views: Given the importance of web application security,  Articles on the comparison of existing methods this article reviews studies in the field of penetration test, particularly web penetration test. and tools for web penetration test.  Articles that proposed a new tool or method for Questions that engaged the mind of researchers in the field of penetration test can be expressed as follows: web penetration test.

107

Copyright (c) 2014 Advances in Computer Science: an International Journal. All Rights Reserved. ACSIJ Advances in Computer Science: an International Journal, Vol. 3, Issue 6, No.12 , November 2014 ISSN : 2322-5157 www.ACSIJ.org

 Studies that proposed a test environment for Table 1- manual vs. automated penetration testing MANUAL AUTOMATED testing tools and ideas. Testing Labor-intensive, Fast, easy and safe. Process inconsistent and error Eliminates errors and 2. Penetration testing -prone, with no tedious manual tasks. specific quality Centralized and standards. standardized to 2.1. History and Importance Requires many produce consistent and Penetration testing is one of the oldest methods for disparate tools. repeatable results. assessing the security of a computer system. In the Results can vary Easy to use and early 1970's, the Department of Defense used this significantly from provides clear, method to demonstrate the security weaknesses in test to test. actionable reports. computer systems and to initiate the development of Generally requires programs to create more secure systems. Penetration highly-paid, testing is increasingly used by organizations to assure experienced security the security of Information systems and services, so personnel to run and interpret tests. that security weaknesses can be fixed before they get Network Often results in Systems remain exposed [1]. Modification numerous systems unchanged. modifications. Large companies have important data and one of Exploit Developing and Product vendor Development their concerns is to protect the data. The penetration maintaining an develops and and test tests security mechanisms of companies by Management exploit database is maintains all exploits. simulating multiple attacks. In the situations like new time-consuming and Exploits are infrastructure is added, Software is installed, System requires significant continually updated updates are applied, Security patches are applied and expertise. for maximum User policies are modified it is necessary to do Public exploits are effectiveness. penetration testing. Some of the principal reasons for suspect and can be Exploits are adopting penetration testing are Security Issues, unsafe to run. professionally Protect Information, Prioritize security risks and Re-writing and developed, thoroughly Financial Loss [2]. porting code is tested, and safe to run. necessary for cross- Exploits are written The penetration test can be done either manually or platform and optimized for a automatically. Manual penetration test requires a functionality. variety of platforms

skilled and experienced tester team to control all and attack vectors. Cleanup Tester must Leading products offer things. They must be physically present at the test remember and undo comprehensive duration. Thus this is not an affordable option. all changes. cleanup with one click Automatic penetration test is a simple and safe way Backdoors can be left and never install to perform all tasks related to the penetration test. behind backdoors. Moreover, since most work is done automatically, it Reporting Requires significant Comprehensive effort, recording and history and findings is more economical in terms of time. Another collating of all results reports are advantage of this test is the ability to reuse the set manually. All reports automatically parameters for the test. In [3], a comparison is made must be generated by produced. Reports are between the two tests, as follows: hand. customizable. Logging/ Slow, cumbersome, Automatically records Auditing often inaccurate a detailed record of all process. activity. Training Testers need to learn Users can learn and non-standardized, ad- install in as little as hoc testing methods. one day.

108

Penetration testing can be segregated into the stage, the more successful the next stages will be. following classes [4]: Attack visibility: Blue-teaming The second phase of this methodology is divided into or Red-teaming, and system access: Internal testing two categories: port scanning which obtains a list of or External testing. open ports and services running on each of them, and vulnerability scanning which is a process to Blue-teaming is done with the consent of an entire recognize weaknesses of desired services and organization. The information security team is fully applications. According to the results obtained in step aware of the testing requirements as well as resources 2 and knowing what ports are open, what services are needed. Blue-teaming is a more efficient way to running on this port and what vulnerabilities they perform testing as the system availability is not an have, one can attack the target. Maintaining access is issue and hence there is a considerable reduction in the last phase. Most accesses obtained in the attack the overall time for testing. The shorter test times phase are temporary, and are removed after mean lesser system idle time and reduced testing disconnection. In this phase, it is tried to maintain costs. Red-teaming refers to testing that is performed access. Although this reference ignored "reporting" in a stealth manner without the knowledge of IT staff as a step in the penetration test, it stated the last [4]. Upper-level management authorizes such an activity of a penetration test as reporting. According exercise. The objectives of the test are to judge the to [5], reporting should include details on how to strength of the network security, the awareness of IT perform the test, a summary of the found security organization, and its ability to follow the standard threats, cases the test does not cover, etc. protocols. The entire test is done without the support

of the organization’s resources. Reconnaissance Scanning

2.2. Available methodologies and

techniques Maintaining Access Exploitation

A methodology is a scheme that is used to reach the Figure 1 - proposed methodology in [5] destination. Lack of use of a methodology for penetration test may lead to an incomplete test, high In the NIST penetration test methodology, the time-consumption, failure and test ineffectiveness. penetration test consists of four phases: planning, Despite the large number of methodologies, there is discovery, attack, reporting. In the planning phase, nothing called "true methodology" and each rules are defined and objectives of the test are set. penetration test can have a different methodology but The discovery phase is performed in two stages. The the use of a methodology leads to a professional and first includes test initiation and information collection efficient penetration test at lower cost. and the second stage, which takes place after the attack phase, includes vulnerability analysis. In the The methodology considered for the penetration test attack phase, which is known as the heart of usually has 4 or 7 phases. Although the name or penetration test, various vulnerabilities in the target number of phases is different in different are examined. The report is prepared in conjunction methodologies, they all show an overview of with the other phases. In the planning phase, the penetration test. For example, some methodologies evaluation plan is developed. In the discovery and use the term "information gathering" while some call attack phases, events are usually recorded and this process "reconnaissance." periodically reported to the director. At the end of the test, a report is provided to describe recognized The methodology proposed in [5] consists of four vulnerabilities, ranking of risks, and tips for how to phases: reconnaissance, scanning (port scanning, improve the known weaknesses [6]. vulnerability scanning), exploitation and maintaining access. The first phase of the penetration test is reconnaissance which focuses on information gathering. The more information gathered at this

109

In the induction phase, the time period and type of the test should be specified. The interaction phase planning Discovery attack indicates the objectives of the penetration test. In the inquest phase, the maximum possible data is achieved on the target system. In the final phase, the Reporting security performance is measured. After completion of the penetration test, results are processed and a Figure 2- proposed methodology in [6] report is prepared. The OSSTMM uses a set of tools called Security Test Audit and Reporting for The methodology presented in [7] consists of three processing the results [8]. main parts: information, team, and tools. In the information part, information is gathered on the In [41], a penetration test scheme is presented for target using different methods. In this paper, the web application based on the RUP test scheme. Each information phase is defined in four steps: studying of the described methodologies may be appropriate the network, identifying the OS, scanning ports and for different purposes and penetration tests. As identifying services. The second part of the mentioned earlier, it cannot be said that a methodology is team formation. If teams are formed methodology is better than another. This scheme with different roles and responsibilities, the provides a systematic, consistent and affordable penetration test will be carried out more effectively. method fully integrated with the security-based Another important parameter in the penetration test is software development lifecycle for the penetration to use tools. To do an effective penetration test, it is test and improves the accuracy, quality and better to dominate a smaller number of tools, instead performance of such tests. This study also presents a of many tools. Another point discussed in this article database of techniques and tools required for the is to set policies that must be followed by the tester penetration test of web applications which was and the client. In the proposed policy, there are issues compiled using various sources including valid such as preservation of information obtained by the guidelines and standards and test techniques on the tester and reporting them completely at the end of the Internet. penetration test, scheduling agreement, confidentiality of all information such as contract, In [42], an penetration test methodology is presented use of information obtained just for the test, lack of based on the agile method which uses the benefits of responsibility of the penetration tester in the event of the agile method in the process of penetration test, a real attack, and so on. and a model is design based on the Scrum and XP methodologies which show information flows methodology between activities. Thus, this method improves the penetration test cycle and can be a framework for Information Team Tools improving the accuracy, efficiency, job satisfaction and quality of testers. In this process, change Shared Closed Roles Responsibiluties Toolkit Reason management can be easily done and information technology goals are aligned with business goals,

Information interaction with customer increases, so prioritizing gathering the depth and range of penetration test is easier.

Figure 3- proposed methodology in [7] 3. Web penetration test

One of the proposed methodologies is Open Source Today, with the Internet expansion and use of web Security Testing Methodology Manual designed by applications in various fields such as military, ISECOM. The OSSTMM phases include induction, medical, finance, etc. web security is an important interaction, inquest, and intervention. concern, and the penetration test is used to ensure it. The penetration test can be performed manually or

110

automatically. The two options are compared in is successful, the vulnerability is reported. Grey-box Table 1. tools can find vulnerabilities in all the application paths with low false positive but, like white-box Tools used to recognize vulnerabilities can be divided tools, they depend on a specific language or into three categories based on information of the framework [9]. target application that they use: white-box, black-box and grey-box. 3.1. Web vulnerabilities

A white-box tool uses the target application code to Given that applications are highly vulnerable, assess vulnerabilities. By analyzing the code invaders use different methods and paths to damage application, a white-box tool can find all the hidden various organizations. These vulnerabilities can be application paths which lead to finding vulnerabilities very simple or complex. In complex ones, the in the application path. In this set of tools, due to discovery and exploitation become very difficult for access to the application code, vulnerabilities may be invaders. According to the characteristics of its reported that are not available. In other words, it is business environment and associated risks, especially likely that there is no possibility to use a known threatening factors, each organization identifies vulnerability. The disadvantage of white-box test is implemented security controls and their impact on its reliance on a specific language and framework. business and financial matters of the organization. Owasp annually publishes a list of ten common Unlike white-box tools, black-box tools assume that vulnerabilities. The last list published in 2013 there is no knowledge of the application code. Instead includes the following vulnerabilities [36]: of using the application code, the tester, like a regular user, uses the application with a browser. A black- 1- Injection: Injection flaws, such as SQL, OS, and LDAP injection occur when untrusted data is sent box tool first examines different parts of the to an interpreter as part of a command or query. application to find all possible injection vectors. The attacker’s hostile data can trick the Whatever way through which the attacker can enter interpreter into executing unintended commands the application is called attack vector like URL or accessing data without proper authorization. parameters, HTML form parameters, cookies, HTTP 2- Broken Authentication and Session Management: headers etc. After identifying injection vectors to the Application functions related to authentication application, inputs are given to recognize and session management are often not implemented correctly, allowing attackers to vulnerabilities. This process is called fuzzing. In compromise passwords, keys, or session tokens, fuzzing, the type of injection vector and its use or to exploit other implementation flaws to duration can be different in black-box tools. Finally, assume other users’ identities. these tools examine http and html responses related 3- Cross-Site Scripting (XSS): XSS flaws occur to fuzzing, and if successful, report it as vulnerability. whenever an application takes untrusted data and sends it to a web browser without proper Among advantages of white-box over black-box validation or escaping. XSS allows attackers to tools, one can point to independence of the execute scripts in the victim’s browser which can hijack user sessions, deface web sites, or redirect application code and less false positive. Given that a the user to malicious sites. black-box tool can only identify vulnerabilities that 4- Insecure Direct Object References: A direct run their related attack, one of its disadvantages is object reference occurs when a developer exposes that it cannot guarantee to identify all the a reference to an internal implementation object, vulnerabilities of the application. such as a file, directory, or database key. Without an access control check or other protection, Grey-box tools, as their name implies, are a attackers can manipulate these references to access unauthorized data. combination of black-box and white-box. These tools 5- Security Misconfiguration: Good security use static white-box analysis techniques to identify requires having a secure configuration defined vulnerabilities. Then they try to really attack and deployed for the application, frameworks, identified vulnerabilities to confirm them. If this step application server, web server, database server,

111

and platform. Secure settings should be defined, open source scanners and then review research on the implemented, and maintained, as defaults are web penetration test. Articles in this category have often insecure. Additionally, software should be considered three issues: comparing scanner available kept up to date. tools, developing a new scanner and designing a 6- Sensitive Data Exposure: Many web applications do not properly protect sensitive data, such as vulnerable application. credit cards, tax IDs, and authentication credentials. Attackers may steal or modify such 3.2.1. Open-source vulnerability scanners weakly protected data to conduct credit card Table 2 presents some open source scanners and their fraud, identity theft, or other crimes. Sensitive data deserves extra protection such as encryption developer companies, used technology and platform. at rest or in transit, as well as special precautions Then Table 3 compares them in terms of use scale, when exchanged with the browser. initial scan method and outputs of each one. 7- Missing Function Level Access Control: Most web applications verify function level access Table 2-Open-source scanners overview [39] rights before making that functionality visible in scanner Company Technology Platform the UI. However, applications need to perform 1 Websecurify GNU JavaScript Windows, the same access control checks on the server CITIZEN (General) Mac, Linux when each function is accessed. If requests are 2 Skipfish Micheal C (General) Linux, Zalewski FreeBSD, not verified, attackers will be able to forge Mac OS X, requests in order to access functionality without Windows proper authorization. (Cygwin) 8- Cross-Site Request Forgery (CSRF): A CSRF 3 Wapiti Informatica Python Unix/Linux, attack forces a logged-on victim’s browser to Gesfor (2.6.x) FreeBSD, send a forged HTTP request, including the Mac OS X, victim’s session cookie and any other Windows automatically included authentication 4 Parosproxy MileSCAN Java 1.4x Linux, Mac OS X, information, to a vulnerable web application. This Windows allows the attacker to force the victim’s browser 5 Arachni Tasos Ruby Windows to generate requests the vulnerable application Laskos (1.9.x) thinks are legitimate requests from the victim. 6 Opent John Java 1.6 Windows 9- Using Components with Known Vulnerabilities: Acunetix Martinelli Components, such as libraries, frameworks, and 7 Grendel- David Java1.6 Linux, Mac other software modules, almost always run with Scan Byrne OS X, full privileges. If a vulnerable component is Windows 8 W3af W3af Python Linux, Mac exploited, such an attack can facilitate serious developers (2.5.x) OS X, data loss or server takeover. Applications using Windows components with known vulnerabilities may 9 WebScarab OWASP Java (1.5.x) Linux, Mac undermine application defenses and enable a OS X, range of possible attacks and impacts. Windows 10- Invalidated Redirects and Forwards: Web 10 SQLmap SQLmap Python 2.6 Linux, Mac applications frequently redirect and forward users developers OS X, to other pages and websites, and use untrusted Windows data to determine the destination pages. Without 11 ZAP OWASP Java 1.6x Linux, Mac OS X, proper validation, attackers can redirect victims Windows to phishing or malware sites, or use forwards to 12 Andiparos Compass Java 1.5x Linux, Mac access unauthorized pages. Security OS X AG Windows 3.2. Black-box web vulnerability scanners 13 Watabo Andreas Ruby 1.8x Linux, Mac Schmidt OS X, In recent years, automated or semi-automated Backtrack, Windows application scanning has been addressed to find

vulnerabilities. To perform an automatic penetration test, there are many commercial and open source scanners. Next, we introduce some commercial and

112

Table 3-Open-source scanners compare [39] Table 4- Commercial scanners overview [40] A B C D E F G H I J Scanner Features 1 Acunetix Low false positive rate, possible reform 1  V.S V.S V.ST F      application security 1 2  S C ST F      2 N-Stalker Set of security programs to evaluate the 3  C C FG F      application security, possibility of define 4  V.S V.S UST3 SL2      evaluation policies 5  S C ST F      3 Netsparker Focus on reducing the false positive, first 6  S S ST F      scanner without false positive 7  V.S S ST SL      4 Burp An integrated platform for attacking and 8  C C FG4 SL      Scanner testing web applications, operate in 9  V.S V.S V.ST F      passive/active mode, manual/live scan mode 10  C S ST SL      5 Rational advanced web application security scanning, 11  V.S7 V.S V.ST6 F5      AppScan results with the Results Expert wizard 12  V.S S8 ST F      6 HP fast scanning capabilities, broad security WebInspect assessment coverage, accurate web 13  V.S V.S UST F      application security scanning result

7 NTOSpider identiﬁes application vulnerabilities, ranks A: GUI F: Spider threat priorities, graphical HTML reports B: Configuration G: Manual crawl C: Usability H: File analysis D: Stability I: Logging Table 5- Commercial scanners compare [38] E: Performance J: Report Scanner A B C D E I J 1  V.S V.S V.ST F   3.2.2. Commercial vulnerability scanners 2  V.S V.S V.ST V.F   3  V.S V.S V.ST F   Commercial vulnerability scanners are developed by 4  V.S V.S ST V.F   various organizations. Although they cost too much, 5  S S V.ST F   they have fewer security bugs compared to open 6  V.S V.S S F   source scanners. Because of competition between 7  V.S V.S S F   different organizations for developing scanners, their limitations are not disclosed. Table 4 introduces some 3.3. Academic research about web scanners popular commercial scanners. Articles in the field of penetration testing are analyzed in three groups. A group of articles compared commercial and open source scanners and tested them against some vulnerability. A second group is articles that provide a new method or tool for automatic penetration tests. Moreover, for assessment of security methods and tools, we require vulnerable applications whose vulnerabilities are

clear. The third group of studies is devoted to these applications.

3.3.1. Comparison of existing tools and methods

Numerous articles reviewed and compared web scanners. In some of these papers, the comparison of tools is the main goal, and some others compared tools with the goal of providing a new tool or 1 Complex 2 Slow technique. Most vulnerabilities considered in these 3 UnStable comparisons are SQL and XSS injection. Many of 4 Fragile these articles suggest that available tools cannot 5 Fast 6 Very Stable identify all vulnerabilities and some others concluded 7 Very Simple that false positive is high. 8 Simple

113

Table 2 and implementation from 4 web services specified By Table 3 present the scanners tested in each article, TPC-APP benchmark. It concluded that the vulnerabilities and the test environment. implemented tool performance in terms of coverage and false positive is better than commercial tools. In [10], the name of tools has not been mentioned for commercial reasons and the neutrality of the article. In [18], it is concluded that crawling in an advanced web application is a serious challenge for penetration In [11], results indicate that different tools produce test tools and they require supporting technologies different results, cannot identify many vulnerabilities such as Flash and Java, more advanced algorithms for and have about 20% to 70% false positive. performing deep crawling, and tracking the state of the application under test and more studies on The comparison by McAllister et al. (2008) shows automating the identification of vulnerabilities in the that tools have a low ability to detect vulnerability, application logic. however, using the proposed technique, more vulnerabilities were found [12]. In [19], the authors concluded that even when scanners are taught to exploit vulnerabilities, they In [13], the authors, due to the detection rate of cannot detect stored SQL injection. They also state vulnerabilities, claimed that their test suit is effective that improving some of the functionality of black-box for different tools. They also stated that none of the scanners like state full scanning, input selection by tools are capable of identifying level 2 or higher field name and tag, the novelty of attack vector, vulnerabilities. server response analysis and post scanning can improve the discovery rate. Shelly (2010) concluded that the use of a test environment with secure and insecure versions is a Reference [20], which is a generalization of [15, 18], good way to find the reasons of producing false studied three scanners and concluded that scanners positive and false negative by tools. The conclusion cannot detect vulnerabilities because of weaknesses declared on the quality of tools indicates that tools in the third phase. can detect simple XSS and SQL injection. But to identify non-simple XSS and SQL injection, session In [21], it is concluded that Iron WASP, NetSparker management flows, running malicious files and community edition, OWASP ZAP, Vega, N-Stalker buffer overflow, more work is needed to improve and W3AF identified highest to lowest number of techniques and tools [14]. vulnerabilities and had the lowest to highest false negative, respectively. In [15], it is noted that tools require a greater understanding of active contents and scripting Based on the results of [22], W3AF, Archani and languages Like SilverLight, Flash, Java Applet and Skipfish were chosen as the best. It was also shown JavaScript. that the most difference in scanners is in identifying injection vulnerabilities, cross-site scripting, session 9 In [16], a new tool called CIVS-WS is developed management and broken authentication. with a new method to identify SQL/XPath injection. It came to the conclusion that the implemented tool Table 2-Scanners evaluation summery1 has the 100% coverage power and 0% false positive. [] Scanners Used test suits [10] HP WebInspect, IBM Rational The paper [17] is based on the results obtained in AppScan, Acunetix Web Vulnerability Scanner [10]. The author proposed a method to identify SQL [11] 3 commercial scanners. MyReferences, injection and developed a tool called VS.WS. To test BooksStore this method, test [10] was repeated. All tools were Online executed against 262 public web services and Java [12] Brup Spider, w3af, Acunetix Web Django-basic- Vulnerability scanner free edition blog, Django- forum 9 Command Injection Vulnerability Scanner for Web Services

114

[13] 4 commercial and open-source scanner An online [17] SQL injection banking [18] Different type of XSS, SQL injection, command-line application injection, file inclusion , file exposure [14] Grendel-Scan, Wapiti, w3af, Hailstorm, BuggyBank , [19] Different type of SQL injection( specially stored SQL N-stalker, Netsparker, Acunetix Web Hokie injection) Vulnerability Scanner, Brup Scanner Exchange [20] Stored XSS [15] Acunetix Web Vulnerability Scanner, Drupal, phpBB, [21] reflected SQLI, stored SQLI, reflected XSS, stored XSS, Cenzic Hailstorm Pro,HP WebInspect, WordPress, reflected XSS behind JavaScript, reflected XSS behind flash IBM Rational AppScan, Mcafee customized test , predictable session ID, command line injection, file SECURE, N-Stalker QA Edition, bed inclusion, file exposure, parameter manipulation, directory QualysGuard PCI, Rapid7 NeXpose traversal, logic flow, forceful browsing , weak passwords 10 [16] CIVS-WS , IBM Rational AppScan, A prototype tool [22] OWASP TOP 10 Acunetix Web Vulnerability Scanner, with 9 services VS.BB with multiple operations [17] VS.WS Repeat [16] In this section, we reviewed several papers that evaluation test compared different scanners. As seen in Table 7, [18] Acunetix Web Vulnerability Scanner , WackoPicko vulnerabilities addressed in Owasp Top 10 were IBM Rational AppScan, Burp scanner, considered in these papers. To enhance the value of Grendel-Scan, Hailstorm, Milescan, N- Stalker , NTOSpider, Paros, w3af , HP assessments, most articles used popular scanners Like WebInspect Acunetix Web Vulnerability Scanner, IBM Rational [19] Acunetix WVS, IBM Rational AppScan PCI, AppScan, and Burp scanner. Table 8 summarizes the Enterprise , QualysGuard Express Suite WackoPicko, number of using each scanner in articles. MatchIt [20] OWASP ZAP, N-Stalker WVS, PCI, Table 4- Frequency of used scanners in papers Acunetix WVS,IBM Rational AppScan WackoPicko, Scanners Used in SimplifiedTB papers [21] Iron WASP ,W3AF ,N-Stalker , WackoPicko (1) Acunetix Web Vulnerability Scanner 8 NetSparker Community Edition ,Vega and OWASP ZAP (2) IBM Rational AppScan 6 [22] IronWASP, ZedAttackProxy(ZAP), (3) w3af, (4) N-stalker 5 SQLmap, W3AF,arachni,Skipfish, Watobo, VEGA, Andiparos, HP WebInspect , Brup Spider, Grendel- 3 ProxyStrike, Wapiti , ParosProxy, Scan, Hailstorm, Wapiti, OWASP ZAP GrendelScan, PowerFuzzer,Oedipus, Netsparker, VS.BB, Paros, Iron WASP, Vega 2 UWSS(UberWebSecurityScanner), Grabber, WebScarab, MiniMySQLat0r, other 1 WSTool, Crawlfish, Gamja, iScan, DSSS(DamnSimpleSQLiScanner), Secubat, SQID(SQLInjectionDigger), Table 5 lists attacks to different vulnerabilities and SQLiX, Xcobra Table 8 lists scanners that were investigated in the compared articles more than others and shows that Table 3- Scanners evaluation summery2 [] Checked vulnerabilities they cover what percentage of cases in Table 6(those [10] SQL injection, Xpath injection, Code execution, Possible not listed in the table are covered by all scanners) parameter based buffer overflow, possible username or [37]. password disclosure, possible server path disclosure [11] SQL injection, XSS Table 5- Attacks [12] stored XSS, reflected XSS # Attack # Attack [13] XSS, SQL injection, blind SQL injection, file inclusion 1 Error based SQL injection 18 Format String Attack [14] SQL injection, XSS, Buffer overflow, Session management 2 Blind SQL injection 19 Code Injection flaws, Malicious file execution [15] Different type of XSS, Different type of SQL injection, 3 Server Side Java Script 20 XML Injection Cross-Channel Scripting, session management flaws, injection CSRF, SSL/Server Conﬁguration, Information Leakage 4 Reflected Cross Site 21 Expression Language [16] SQL injection, XPath injection Scripting Injection 5 Persistent Cross Site 22 Buffer Overflow Scripting 10 Command Injection Vulnerability Scanner for Web Services 6 DOM Cross Site Scripting 23 Integer Overflow

115

7 JSON Hijacking 24 Source Code Disclosure Yapig, 15 unknown vulnerabilities were identified 8 Path Traversal & Local 25 Old, backup and with 16 false positives. A restriction of Pixy is that it File Inclusion Unreferenced Files does not support 'OO' (object orientation). 9 Remote File Inclusion 26 Padding Oracle 10 Command Injection 27 Forceful Browsing/ Authentication Bypass Saner is a tool developed by combining static and 11 Unrestricted File Upload 28 Privilege Escalation dynamic techniques to identify errors in applications 12 Open Redirect 29 Xml External Entity in PHP language [24]. It uses static techniques to 13 CLRF injection 30 Weak Session Identifier identify inputs in each path with the help of string 14 LDAP Injection 31 Session Fixation modeling methods. The second technique is dynamic 15 Xpath injection 32 Cross Site Request analysis which operates bottom-up. It first recreates Forgery 16 SMTP/IMAP/EMAIL 33 Application Denial of codes to identify inputs and then uses a large Injection service collection of malicious inputs to identify exploitable 17 Server-Side Includes flows. The dynamic phase aims to test all the Injection application paths that were identified suspicious in the static phase. Since the static analysis phase is prudent, it is possible to produce false positive which

Table 6- Compare scanners must be manually evaluated which is a boring 3 5 6 7 11 14 15 16 17 18 19 20 process. The purpose of automatic dynamic analysis (1) × × × × is to automate this process or at least automate (2) × detection of what vulnerability input should be used. (3) × × × × For assessment, this tool was evaluated on five (4) × × × × × × × × × × × × applications of Jetbox, MyEasyMarket, 21 22 23 26 27 28 29 30 31 32 33 % (1) × × × × 75.7 PBLGuestbook, PHP-Fusion and Sendcard which are (2) × × 90.9 developed in PHP. Although this tool has not a very (3) × × × × × × 69.6 good execution performance, the time required to (4) × × × × × × × × × × × 30.3 analyze most applications is good.

In [25], the most common input validation Since XSS and SQL injection vulnerabilities are vulnerability model called Tainted Mode Model is among common vulnerabilities, they were reviewed used to identify internal vulnerabilities. This paper in the majority of articles. improves the classical tainted mode model to investigate internal data flows. It also introduces a 3.3.2. New methods and tools new method using information obtained from dynamic analysis for doing the automated penetration This section reviews a number of papers that test. Given a larger view it gives from the application, designed a new scanner. its accuracy will be more, and the accuracy of input In [23], a tool was design with static analysis validation procedures can be tested. Using improved capabilities called Pixy to identify vulnerabilities in tainted mode model, applications were modeled as web applications. Pixy is an open source tool and its follows: main purpose is to identify cross-site scripting W: ( Scheme, Req × State DDG × Resp × {query } vulnerability in PHP scripts. PHP was selected × State) because it is widely used in designing web applications and by a large number of security Where scheme is a set of relational data schemes consultants in PHP applications. Vulnerability which shows the application database. Req is the http detection was based on data flow analysis. To request sent to the application. State, on the left, evaluate this tool, six open source PHP applications refers to the application state which includes contents were used. In PhpNuke, PhpMyAdmin and Gallery, of the application environment (such as a database, 36 known vulnerabilities were recreated with 27 false system files and LDAP). DDG = (V, E) is a data positives. In Simple PHP Blog, Serendipity and

116

dependency graph which indicates the execution path variables. BLOCK has two key phases to identify and data flow obtained by the application to process state violation attacks: in the training phase, the the received request. Resp is the response returned by desired behavior model is obtained by observing web the application. {query} is a set of queries from the request/response sequence, and the variable values database generated by the application when corresponding to the session during its running processing a request. State, on the right, is the state to without attacks. In the identification phase, the which the application transfers. obtained model is used to evaluate each incoming web request and outgoing response, and any violation The implemented approach has three main is identified. To assess this tool, the applications of components: dynamic analysis module that gathers Scarf, Simplecms, BloggIt, WackoPicko and the effects of application execution. Analyst that OsCommerce were used. Results show that this produces DDGs for gathered effects and the method is effective in identifying state violation penetration test module that enters normal or attacks and is incurred little overhead. Since this malicious inputs into the application. For assessment, method is independent of the application code and three applications in Python (Test application, Spyce can be used for a large number of web applications and Trac) were used. In the presented results, false with different frameworks, it is of particular positive rate is zero. importance.

In [26], a scanner is proposed to identify injection In [28], a mealy state machine is used to model web vulnerabilities. This system analyzes websites, applications for management of requests that change aiming at automatically finding SQL and XSS the application mode. To detect a state change in this injection vulnerabilities. The proposed system model, difference in the returned response in case of consists of two components: spider and scanner. The identical sent requests was considered. To implement spider is used to navigate the site and find input this method, htmlUnit and the fuzzer of w3af tool points. The scanner initiates injection test and were used. In the state graph the tool generates, nodes response analysis and consists of two parts: response represent states and edges represent the requests sent analyst and author of rules. The system was run in to the application. Using the graph coloring problem, VMware work station ACE with two hosts, one for this article combined similar states. For evaluation, the defense server and the other for the web server. this product was studied with wget, w3af and skipfish The system was designed with PHP5 and MySQL scanners on the Gallery application, two versions of and used the cURL module to execute attacks. Seven PhpBB, Scraf, vanilla forums, WackoPicko and three applications from National Vulnerability Database versions of wordPress. The evaluation results indicate (NVD)11 were selected for assessment. Finally, the that the designed scanner not only can run more designed scanner was compared with some other codes than web applications, but also it can identify scanners. It concluded that this system is effective, vulnerabilities not identified by other scanners. Due and vulnerability detection based on input points to the use of HtmlUnit, despite using the w3af fuzzer definitely can find vulnerabilities. tool, this tool has a less false positive than other tools. Among the limitations of this tool, one can BLOCK is a black-box approach designed in [27] to point to the lack of support of AJAX and applications identify state violation attacks based on the that can be used publicly because users may WebScrab tool. In this paper, the application is influence the state-change detection algorithm. considered as a stateless system, and the application behavior model is obtained from the interaction In [29], an automatic black-box tool is presented to between client and application, i.e. the relationships identify reflected XSS and stored XSS vulnerabilities between web requests, responses and session in web applications. The tool uses user interactions to do the test more effectively. First, user interactions 11 A set of standards based on vulnerability management data are recorded, then changes are made on these owned by the US government which uses the Security Content interactions for attacks and finally, this transaction is Automation Protocol (SCAP). re-executed on the system. To evaluate the

117

performance, the tool was compared with Spider, inputs to identify the application paths which violate Brup Spider, w3af and Acunetix on three applications these conditions. This paper focuses on logical of the Django framework from different aspects. vulnerabilities. The provided tool is Waler which is Results show that the proposed method can identify used for servlet-based application developed in Java. more bugs than the listed commercial and open To assess the tool, 12 applications were used. source tools. According to this article, Waler is the first tool that can detect logical processes in applications In [30], a tool called MiMoSA is presented based on automatically without human intervention. static analysis for PHP applications. The paper divides multi-module into two groups (data flow and There are many different tools for implementing each workflow) and input states into two categories (server of the penetration test steps. In [32], for saving time, side and client side). The analysis intended to some of these tools were combined including MiMoSA has two phases: intra-module which theHarvester, Metagoofile, ZAP, NMAP, Nessus and reviews each module of the application on its own, Metasploit. The language used in this article is and inter-module which considers the whole Python. The function of proposed method can be application. The intra-module analysis aims to explained in three phases: information gathering, summarize each module of the application by analysis of the information obtained and the use of defining preconditions, post-conditions and sinks. All this information to find possible vulnerabilities. First, links in each module are also extracted. This phase is the tools theHarvester, Metagoofile and NMAP are dependent on the programming language. This run as information gathering tools and Nessus and information is then used for inter-module analysis to ZAP as two scanners, and information obtained by obtain the future state of the application workflow. In each tool is stored in a separate file. Then the results the inter-module analysis phase, results from intra- of ZAP and Nessus are analyzed and the resulting module analysis are put together in a single graph output file is used for implementing the attack phase which models the future work flow of the whole on Metasploit. application. Then a model survey technique is used to identify data flow vulnerabilities and future workflow The section addresses 10 studies that proposed a new defects. This phase is independent of the method or tool for finding vulnerabilities in web programming language used in the application applications. Each study is based on static or dynamic development. To assess the tool, three applications of analysis, or a combination of the two. A summary of Aphpkb, BloggIt, MyEasyMarket, Scarf and these characteristics is given in Table . SimpleCms, developed in the PHP language, were used. Evaluation results show that MiMoSA can Table 11- papers proposed tools and methods summery [] Year Static Dynamic Description identify all known vulnerabilities and also discover analysis analysis some new vulnerabilities. In evaluating the tool, only [23] 2006  cross-site scripting one false positive was seen. The number of states the detection tool considers is more than the number of actual [24] 2008   identify errors in states in the application code. This problem occurs applications by path input detection for two main reasons: first, in MiMoSA, states may [25] 2008  Use TDM to identify be produced correspond to paths that cannot be run in internal vulnerabilities the application and second, the possibility of [26] 2010  XSS and injection repetitive states in the tool with aligned but different vulnerabilities detection conditions. [27] 2011   identify state violation attacks based on the WebScrab tool In [31], first the dynamic analysis and observing the [28] 2012   Use state machine to application performance are used to understand the model web application for application's behavioral characteristics. Then the detect more found characteristics are filtered to reduce false vulnerabilities stored و  reflected XSS 2008 [29] positive. The symbolic evaluation model is used on

118

XSS detection Manipulation, Reflected XSS Behind JavaScript, [30] 2007  Code analysis to models Logic Flaw, Reflected XSS Behind a Flash Form and the future work flow of Weak username/password. the application [31] 2010  logical vulnerabilities detection BodgeIt is a web based vulnerable application [32] 2013  Combine different tools to designed by Simon Bennett with learning goals for save time pentesters. This web application consist of cross site scripting, sql injection , hidden(but unprotected) content, cross site request forgery, debug code, application logic و Design of test environments insecure object references .3.3.3 Vulnerable web applications are used to web vulnerabilities[35]. vulnerability scanners evaluation. In this part we will Table 7- Test bed vulnerabilities have a review on some of this applications like Damn DVWA WebGoat WackoPicko BodgeIt Vulnerable Web App(DVWA)[33], OWASP Brute Force   WebGoat[34], WackoPicko[18] and BodgeIt[35] that CSRF   are used to training courses and numerous articles. File Inclusion   SQL Injection     Damn Vulnerable Web App (DVWA) is a Access Control  Flaws PHP/MySQL web application that is damn XSS     vulnerable. Its main goals are to be an aid for security Buffer Overflows  professionals to test their skills and tools in a legal Logic   environment, help web developers better understand vulnerabilities the processes of securing web applications and aid AJAX Security  teachers/students to teach/learn web application security in a class room environment. Brute Force Login, Command Execution, CSRF, File Inclusion, 4. Conclusion SQL Injection, Upload Vulnerability and XSS are the The present paper reviewed studies in the field of vulnerabilities exist in DVWA[33]. penetration test, especially web penetration test. Manual penetration test is not effective in terms of Webgoat is a J2EE based web application designed time and money, so its automatic version is by OWASP to introduce common security flaws in considered. For performing the automatic web the web applications. The following categories are penetration test, web scanners are used. They first available in Webgoat : Access Control Flaws, AJAX crawl the target, then attack to the results of the Security, Authentication Flaws, Buffer Overflows, previous phase and finally report vulnerabilities in Code Quality, Concurrency, Cross-Site Scripting the target. In this paper, we examined research in the (XSS), Improper Error Handling, Injection Flaws, field of web penetration test in three categories: Denial of Service, Insecure Communication, Insecure articles that compared and analyzed available Configuration, Insecure Storage, Malicious scanners, articles that proposed a new method or tool Execution, Parameter Tampering, Session for penetration test and articles that proposed a test Management Flaws, Web Services, and Admin environment to test different tools. According to Functions [34]. papers that analyzed various scanners, the Acunetix WackoPicko is a vulnerable website that designed by Web Vulnerability Scanner and IBM Rational Adam Doupé and used in [18] for the first time. AppScan scanners and the SQL injection and XSS Vulnerabilities of this application are reflected XSS, vulnerabilities were considered more than others. We stored XSS, SessionID vulnerability, stored SQL also reviewed 10 studies that proposed a new tool or injection, reflected SQL injection , Directory method for penetration test, some of which were Traversal, multi-step stored XSS, forceful browsing, based on the dynamic analysis, some on the static Command-line Injection, File Inclusion, Parameter analysis and some on a combination of the two. To

119

evaluate any method or tool in the field of penetration Dependable Computing, 2007. PRDC 2007. 13th Pacific test, we require test environments. Four test Rim International Symposium on (pp. 365-372). IEEE. environments were introduced in the final section. [12] McAllister, S., Kirda, E., & Kruegel, C. (2008, The problems in existing scanners include the lack of January). Leveraging user interactions for in-depth testing of web applications. In Recent Advances in Intrusion support of attacks like stored sql and stored XSS that Detection (pp. 191-210). Springer Berlin Heidelberg. need to several steps to complete the attack, the lack of support of new technologies and vulnerabilities [13] Fong, E., Gaucher, R., Okun, V., & Black, P. E. (2008, related to application logic flows. It is hoped that January). Building a test suite for web application scanners. future work will consider these items. In Hawaii International Conference on System Sciences, Proceedings of the 41st Annual (pp. 478-478). IEEE. References [14] Shelly, D. A. (2010). Using a Web Server Test Bed to [1]http://searchnetworking.techtarget.com/tutorial/Network Analyze the Limitations of Web Application Vulnerability -penetration-testing-guide Scanners (Doctoral dissertation, Virginia Polytechnic Institute and State University). [2] Samant, N. (2011). Automated penetration testing (Doctoral dissertation, San Jose State University). [15] Bau, J., Bursztein, E., Gupta, D., & Mitchell, J. (2010, May). State of the art: Automated black-box web [3]http://www.coresecurity.com/comparing-security- application vulnerability testing. In Security and Privacy testing-options (SP), 2010 IEEE Symposium on (pp. 332-345). IEEE.

[4]http://www.c2networksecurity.com/ [16] Antunes, N., Laranjeiro, N., Vieira, M., & Madeira, H. (2009, September). Effective detection of SQL/XPath [5] Engebretson, P. (2013). The Basics of Hacking and injection vulnerabilities in web services. InServices Penetration Testing: Ethical Hacking and Penetration Computing, 2009. SCC'09. IEEE International Conference Testing Made Easy. Elsevier. on (pp. 260-267). IEEE.

[6] Scarfone, K., Souppaya, M., Cody, A., & Orebaugh, A. [17] Antunes, N., & Vieira, M. (2009, September). (2008). Technical guide to information security testing and Detecting SQL injection vulnerabilities in web services. In assessment. NIST Special Publication, 800, 115. Dependable Computing, 2009. LADC'09. Fourth Latin- American Symposium on (pp. 17-24). IEEE. [7] Alisherov, F., & Sattarova, F. (2009). Methodology for penetration testing. InInternational Journal of Grid and [18] Doupé, A., Cova, M., & Vigna, G. (2010). Why Distributed Computing. Johnny can’t pentest: An analysis of black-box web vulnerability scanners. In Detection of Intrusions and [8] Herzog, P. (3). The Open Source Security Testing Malware, and Vulnerability Assessment (pp. 111-131). Methodology Manual. 2010.ISECOM.[Links]. Springer Berlin Heidelberg.

[9] Doupé, A. L. (2014). Advanced Automated Web [19] Khoury, N., Zavarsky, P., Lindskog, D., & Ruhl, R. Application Vulnerability Analysis (Doctoral dissertation, (2011, October). An analysis of black-box web application UNIVERSITY OF CALIFORNIA Santa Barbara). security scanners against stored SQL injection. InPrivacy, security, risk and trust (passat), 2011 ieee third [10] Vieira, M., Antunes, N., & Madeira, H. (2009, June). international conference on and 2011 ieee third Using web security scanners to detect vulnerabilities in web international conference on social computing (socialcom) services. In Dependable Systems & Networks, 2009. (pp. 1095-1101). IEEE. DSN'09. IEEE/IFIP International Conference on (pp. 566- 571). IEEE. [20] Alassmi, S., Zavarsky, P., Lindskog, D., Ruhl, R., Alasiri, A., & Alzaidi, M. An Analysis of the Effectiveness [11] Fonseca, J., Vieira, M., & Madeira, H. (2007, of Black-Box Web Application Scanners in Detection of December). Testing and comparing Web vulnerability Stored XSSI Vulnerabilities. scanning tools for SQL injection and XSS attacks. In

120

[21] Suteva, N., Zlatkovski, D., & Mileva, A. (2013). [31] Felmetsger, V., Cavedon, L., Kruegel, C., & Vigna, G. Evaluation and Testing of Several Free/Open Source Web (2010, August). Toward automated detection of logic Vulnerability Scanners. vulnerabilities in web applications. In USENIX Security Symposium (pp. 143-160). [22] SAEED, F. A., & ELGABAR, E. A. (2014). ASSESSMENT OF OPEN SOURCE WEB [32] Haubris, K. P., & Pauli, J. J. (2013, April). Improving APPLICATION SECURITY SCANNERS. Journal of the Efficiency and Effectiveness of Penetration Test Theoretical and Applied Information Technology, 61(2). Automation. In Information Technology: New Generations (ITNG), 2013 Tenth International Conference on (pp. 387- [23] Jovanovic, N., Kruegel, C., & Kirda, E. (2006, May). 391). IEEE. Pixy: A static analysis tool for detecting web application vulnerabilities. In Security and Privacy, 2006 IEEE [33] http://www.dvwa.co.uk/ Symposium on (pp. 6-pp). IEEE. [34]https://www.owasp.org/index.php/Category:OWASP_ [24] Balzarotti, D., Cova, M., Felmetsger, V., Jovanovic, WebGoat_Project N., Kirda, E., Kruegel, C., & Vigna, G. (2008, May). Saner: Composing static and dynamic analysis to validate [35] https://code.google.com/p/bodgeit/ sanitization in web applications. In Security and Privacy, 2008. SP 2008. IEEE Symposium on (pp. 387-401). IEEE. [36]https://www.owasp.org/index.php/Top_10_2013- Top_10 [25] Petukhov, A., & Kozlov, D. (2008). Detecting security vulnerabilities in web applications using dynamic analysis [37] http://www.sectoolmarket.com/ with penetration testing. Computing Systems Lab, Department of Computer Science, Moscow State [38]http://www.sectoolmarket.com/general-features- University. comparison-unified-list.html#Glossary

[26] Chen, J. M., & Wu, C. L. (2010, December). An [39] Khari, M., Singh, N. (2014, May). An Overview of automated vulnerability scanner for injection attack based Black Box Web Vulnerability Scanners. In Computer on injection point. In Computer Symposium (ICS), 2010 Science and Software Engineering, 2014 International International (pp. 113-118). IEEE. Journal of Advanced Research.

[27] Li, X., & Xue, Y. (2011, December). BLOCK: a [40] Shelly, D. A. (2010). Using a Web Server Test Bed to black-box approach for detection of state violation attacks Analyze the Limitations of Web Application Vulnerability towards web applications. In Proceedings of the 27th Scanners (Doctoral dissertation, Virginia Polytechnic Annual Computer Security Applications Conference (pp. Institute and State University). 247-256). ACM. [41] Arefzadeh, A.(2012). Representing a penetration test [28] Doupé, A., Cavedon, L., Kruegel, C., & Vigna, G. plan for web-applications , based on RUP test plan(Master (2012, August). Enemy of the State: A State-Aware Black- thesis, Department of Information Technology Box Web Vulnerability Scanner. In USENIX Security Engineering, Maleke-ashtar University of Technology) Symposium (pp. 523-538). [42] Marvari, S. (2012). Presentation An Improved [29] McAllister, S., Kirda, E., & Kruegel, C. (2008, Structure of Methodology of Penetration Test. (Master January). Leveraging user interactions for in-depth testing thesis, Department of Information Technology of web applications. In Recent Advances in Intrusion Engineering, Maleke-ashtar University of Technology Detection (pp. 191-210). Springer Berlin Heidelberg.

[30] Balzarotti, D., Cova, M., Felmetsger, V. V., & Vigna, G. (2007, October). Multi-module vulnerability analysis of web-based applications. In Proceedings of the 14th ACM conference on Computer and communications security (pp. 25-35). ACM.

121