IJCSNS International Journal of Computer Science and Network Security, VOL.10 No.8, August 2010 239 AutoFuzz: Automated Network Protocol Fuzzing Framework Serge Gorbunov and Arnold Rosenbloom [email protected], [email protected] Department of Mathematical and Computational Sciences, University of Toronto Mississauga, Mississauga, Ontario, Canada L5L 1C6 Abstract clients or servers. However, 'dumb' fuzzing is measured to Assessing software security involves steps such as code review, be 50% less effective than 'smart' fuzzing [11]. One risk analysis, penetration testing and fuzzing. During the fuzzing example of a 'dumb' fuzzer is ProxyFuzz [17]. ProxyFuzz phase, the tester‟s goal is to find flaws in software by sending is a man-in-the-middle non-deterministic network fuzzer. unexpected input to the target application and monitoring its It randomly changes the network traffic [17] between a behavior. In this paper we introduce the AutoFuzz [1] - connected client and server. Fuzzers of the second type, extendable, open source framework used for testing network protocol implementations. AutoFuzz is a „smart‟, man-in-the- 'smart' fuzzers, have a pre-programmed understanding of middle, semi-deterministic network protocol fuzzing framework. the protocol implemented by the targets they fuzz. They AutoFuzz learns a protocol implementation by constructing a typically understand the protocol‟s state machine, Finite State Automaton (FSA) which captures the observed messages syntax and field types and use this to efficiently communications between a client and a server [5]. In addition, fuzz deep into target implementation code. Peach is an AutoFuzz learns individual message syntax, including fields and example of a „smart‟ fuzzer [16]. Disadvantages of „smart‟ probable types, by applying the bioinformatics techniques of [2]. fuzzers include their reliance on the availability of a Finally, AutoFuzz can fuzz client or server protocol protocol‟s specification documents and the degree to implementations by intelligently modifying the communication which a target implementation conforms to the published sessions between them using the FSA as a guide. AutoFuzz was applied to a variety of File Transfer Protocol (FTP) server specification. In addition, „smart‟ fuzzers require manual implementations, confirming old and discovering new adaptation to customize them for each new protocol they vulnerabilities. are to apply to. Therefore, its application to new protocols Key words: is labour intensive and tedious. Automated Fuzzing, Software Security, Vulnerability Detection 1.2 Previous Work 1. Introduction A number of attempts have been made to automatically 1.1 Background extract protocol specifications for „smart‟ fuzzers [2][4][5]. In [5] the automatic extraction of the protocol‟s Flaws in the implementations of network protocols are specification is based on synthesizing an abstract some of the most serious security problems. One such flaw behavioral model of a protocol implementation. The could allow a malicious user to attack vulnerable systems behavioral model is realized as a Finite State Automaton remotely over the Internet. Approximately 85% of all (FSA) constructed from the recorded conversations vulnerabilities reported by the National Vulnerability between a client and a server. The FSA represents, in a Database [15] in the last 3 years can be exploited succinct way, the key states and transitions of a protocol remotely. implementation and can be used to systematically guide A fuzzer is a tool used to discover implementation flaws the flaw detection process. The main algorithm proposed by sending the target implementation unusual inputs in in [5] for synthesizing an abstract behavioral model of a hopes of producing unexpected behavior. A protocol protocol implementation is based on passive synthesis fuzzer can be classified as 'smart' or 'dumb' depending on with partial FSA reduction. Given a large collection of its knowledge of the network protocol implemented by its network traces the algorithm constructs and minimizes a targets. A 'dumb' fuzzer sends random inputs to its target. FSA. The construction of a FSA relies on an abstraction It has no knowledge of the communication protocol function. An abstraction function is a simple function used implemented by the target. „Dumb‟ fuzzers are easy to to map similar messages to a unique abstract develop and are immediately applicable to any protocols representation. For example, SMTP client requests can be 240 IJCSNS International Journal of Computer Science and Network Security, VOL.10 No.8, August 2010 abstracted to their first four characters. That is, messages Sequence (GMS) that is used to capture the syntax „mail from: [email protected]‟ and „mail from: information of the message. A GMS is a representation of [email protected]‟ are abstracted to „mail‟. Also, SMTP a message that separates static from variable data fields server replies can be abstracted to their first three and associates variable data fields with type and length characters. For example, messages “550 Permission information. By using GMSs, AutoFuzz eliminates the denied”, “221 Bye!” and “230 User anonymous logged need for protocol specific fuzzing functions as required by in” are abstracted to “550”, “221” and “230” [5]. Fuzzing functions can now be performed on GMS respectively. The tester must supply two abstraction representations instead of individual messages and be functions, one for the input messages to the target being based on the derived type or length information of the fuzzed, the other for the output messages. In [4], the static or variable data fields. AutoFuzz can also be authors focus on automated protocol specification extended with new fuzzing functions. Finally, AutoFuzz extractions by constructing the protocol‟s FSA and intelligently fuzzes server or client network protocol determining message types. However, their technique of implementations acting as a man-in-the-middle and using FSA construction is substantially different from the the constructed FSA as a guide during the vulnerability technique presented in [5]. Their final system can be used detection process. AutoFuzz was successfully applied to to extract protocol specifications. However, to the best of several File Transfer Protocol (FTP) implementations our knowledge, neither of the systems [4] nor [5] is where it found both existing and new vulnerabilities. available publically for future development or research. In [2], the authors try to determine fields of individual 2. Framework Overview protocol messages by using bioinformatics algorithms. In order to determine message fields, similar message samples are aligned using multiple string alignment 2.1 Main Components algorithms and their consensus sequences are analyzed to understand the beginning and the end of fields in the The main components of AutoFuzz are (1) AutoFuzz message [2]. Their open-source tool can be used to Graphical User Interface (GUI), (2) Proxy Server, (3) determine message fields for a collection of protocol Protocol Specifications Extractor and (4) Fuzzing messages. Engine. We elaborate on each below. (1) AutoFuzz GUI allows testers to easily interact with 1.3 The New Fuzzing Framework the fuzzer and control its actions. It is constructed using the JAVA Swing library [13]. To visualize a protocol‟s FSA AutoFuzz uses JUNG graphing library [14]. This paper introduces the AutoFuzz. This open source (2) Proxy Server. AutoFuzz works as a proxy server fuzzing framework is a „smart‟, man-in-the-middle fuzzer. between a client and a server. It records and modifies the For simplicity in the discussion that follows we assume application level traffic to extract protocol specifications that AutoFuzz is used to fuzz the server side of a network and perform fuzzing operations. The proxy server is based protocol implementation. More specifically, the messages on the JAVA Socks server [6], but has been modified to coming from the client to the server are denoted as input allow direct manipulation of the application level traffic. messages, and the messages coming from the server to the client are denoted as output messages. However, AutoFuzz Modified input Original input can be applied with equal effectiveness to fuzz the client side. First, AutoFuzz extracts specifications of a network Server AutoFuzz Client protocol implementation from conversations recorded by Output Output acting as a man-in-the-middle between server/client Figure 1. AutoFuzz Proxy Model sessions. As in [5] AutoFuzz constructs a FSA which captures the sampled conversations, and so, understands (3) Protocol Specifications Extractor. The specifications the protocol at a high level. AutoFuzz can be extended to extractor extracts the FSA of a network protocol understand any protocol by importing appropriate implementation from a sample of communication sessions abstraction functions. Then, using the techniques between a client and a server. AutoFuzz can understand presented in [2], AutoFuzz finds the fields of individual any application level protocol implementation after messages. In addition, it derives the type information of appropriate input/output abstraction functions are imported the variable data fields of individual messages, and so, in it. It also extracts GMSs using the algorithm outlined in understands the protocol at a lower level. More the Generic Message Sequence Construction section to specifically, for each message of the sampled understand to the syntax of individual messages.