Buffer Overflow Attacks: Detection and Prevention Techniques
Total Page:16
File Type:pdf, Size:1020Kb
Buffer Overflow Attacks: Detection and Prevention Techniques A report submitted in partial fulfilment of the requirements for the award of IASc-INSA-NASI SUMMER RESEARCH FELLOWSHIP Sponsored by Indian Academy of Sciences, Bangalore By ANAMIKA GHOSH Reg No.: ENGS990 Under the guidance of Dr. B.M. Mehtre, Professor Centre for Excellence in Cyber Security Institute for Development and Research in Banking Technology Established by Reserve Bank of India Castle Hills, Road No. 1, Masab Tank, Hyderabad-57, Telengana State July, 2018 1 DECLARATION I hereby declare that the project entitled Buffer Overflow Attacks: Detection and Prevention Techniques, submitted to Indian Academy of Sciences, Bangalore, and at IDRBT, Hyderabad, is prepared by me and was not submitted to any other institution for any fellowship to the best of my knowledge. Anamika Ghosh ENGS990 2 CERTIFICATE This is to certify that the project entitled Buffer Overflow Attacks: Detection and Prevention Techniques submitted to the Indian Academy of Sciences, Bangalore and at IDRBT by ANAMIKA GHOSH, bearing Registration No: ENGS990, in the partial fulfilment for the requirement for the award IASc-INSA-NASI SUMMER RESEARCH FELLOWSHIP is a bonafide work carried out by her under my supervision and guidance. Dr. B.M. Mehtre Professor Project Guide, Centre of Excellence for Cyber Security IDRBT, Hyderabad 3 ACKNOWLEDGEMENT I wish to express my deep sense of gratitude and indebtedness to Prof. B.M. Mehtre, Centre of Excellence in Cyber Security, Institute for Development and Research in Banking Technology (IDRBT) for introducing the topic and for his inspiring guidance, constructive criticism and valuable suggestion throughout the project work. I am thankful to all the staff members at the Centre of Excellence in Cyber Security, IDRBT for their insight and help throughout the project period. I would also like to express my deep sense of gratitude for my family for their unwavering support and encouragement and for being there when I needed them the most. Date: Anamika Ghosh ENGS990 CONTENTS 4 Abstract . 6 Abbreviations . 7 1 Background 1.1 Introduction . 8 1.2 Objectives . 10 2 Literature Review 2.1 The Memory Layout . 11 2.2 Stack . 11 2.3 Types of Buffer Overflows . 13 2.4 Stack Overflow . 13 2.5 Shellcode . 14 3 Buffer Overflow Attack Testing . 15 4 Return-to-LibC Attack . 18 4.1 Experimental Setup . 18 4.2 Testing . 19 5 Prevention and Detection Techniques of Buffer Overflows 5.1 Safe Coding Practices . 24 5.2 Kernel-enforced Protection . 25 i. Non-executable Stack ii. ASLR 5.3 Complier based protection . 26 i. StackGuard ii. ProPolice iii. StackShield 6 Conclusions . 28 References . 29 ABSTRACT 5 With the rapid development of technology all around the globe, more and more people are connected to the internet for more than just connecting with people. It is also used for electronic record keeping, mail, financial transactions through online banking etc. It is essential for this reason that the data involved in these transactions be saved from the hands of malicious attackers who may exploit them for their own interests. Due to various vulnerabilities in a multitude of implemented code, attackers may find ways to exploit user data or cause loss of resources. One of the oldest and common vulnerability, Buffer Overflow has been exploited since the 1980’s and in spite of undertaking various mitigation techniques, it remains as one of the top concerns for security. A large number of web servers, application servers and web application environments may be susceptible to buffer overflows, notable exceptions being environments written in interpreted languages like Java or Python. Buffer Overflows can be used to execute arbitrary code or may lead to crashes which may be used for Denial of Service attacks. Defence techniques such as using safe-string libraries, non-executable memory, ASLR, canary protection, avoiding logic errors in code etc. have been implemented to prevent attackers from exploiting the weaknesses. However, these mitigation efforts are often bypassed by creative techniques used by the attackers when new vulnerabilities are identified in the code of vulnerable programs. This report deals with Buffer Overflow - the most common vulnerability exploited by attackers and some of the common prevention and detection techniques that have been implemented to protect the users from such attacks. The objective of this is to analyze buffer overflow attack techniques such as Stack smashing (Kali Linux) and Return-to libc (on Protostar), the damage that can be incurred when a successful buffer overflow attack is carried out by an intruder and the steps that have been taken to mitigate the repercussions. ABBREVIATIONS 6 ASLR Address Space Layer Randomization BOF Buffer Overflow CPU Central Processing Unit CVE Common Vulnerabilities and Exposures DoS Denial of Service EBP Extended Base Pointer EIP Extended Instruction Pointer ESP Extended Stack Pointer GCC GNU Compiler Collection gdb GNU Debugger msf Metasploit Framework NIST National Institute of Standards Technology NOP No OPeration instruction OS Operating System SSL Secure Socket Layer TLS Transport Layer Security 1. BACKGROUND 1.1 Introduction 7 Cyber Security comprises of technologies, processes and controls that are designed to reduce the risk of cyber attacks and protect systems, networks and data of organizations or individuals from unauthorized exploitation by attackers. Cyber Security is aimed at protecting systems, networks from malicious attacks and provides tools for Authentication, authorization and nonrepudiation that can be used to maintain security with respect to confidentiality, integrity, and availability.[1] With more and more devices connected to the internet, privacy and data security is indispensable but due to various system flaws and vulnerabilities in the existing codes, a system might be open to attackers for exploitation. They may exploit certain vulnerabilities to bypass authentication and gain access to sensitive information on the victim’s system. Buffer Overflow is one of the oldest and common vulnerabilities exploited by attackers which can be used for data corruption, data theft or to gain remote access. Buffer Overflow occurs when the input to a program is larger than the buffer size allocated for it and there is no bounds checking. This allows attackers to overwrite the buffer with malicious code by overrunning the buffer’s boundary and writing into the adjacent memory locations. This is particularly noticeable in C and C++ programming languages which allow direct access to memory and provide no built-in protection against accessing or overwriting data in any part of memory. They do not automatically check whether the data written to an array is within the boundaries of that array[2]. Nevertheless, a lot of security-critical programs are written in C. Of the various forms of BOF Attacks, the most vulnerable and widely exploited memory region is the Stack which is little-endian and the very nature of it makes it an easy target for intruders to exploit. According to a study of top vulnerabilities with respect to NIST over a span of 7 years (as of 2014), it shows that Buffer errors is one of the most common vulnerabilities constituting almost 12% of the total vulnerabilities reported and accounted for 22-23% of the total severe vulnerabilities (Homae and Shahriari, 2017) [35]. The first known Buffer Overflow Attack was carried out by the Morris Worm in 1988 which was one of the first computer worms to be propagated on a large scale on the internet. It exploited Buffer Overflow vulnerabilities in the finger daemon [using vulnerable gets() call] of UNIX and caused several computers to crash and rendered them unusable unless reformatted. The large scale impact of the worm may be attributed to its spreading mechanism which often allowed it to replicate itself multiple times thereby causing the systems to crash. However, after the news of the existence of such a fatal vulnerability in the common software codes became known, it initiated multiple attempts by adversaries to use it for gaining access to other systems. BOF Attacks accounted for 14 percent of the total attacks over the past 25 years (as of 2013) [15]. Among other major internet worms, Code Red worm exploited BOF vulnerability in Microsoft’s IIS 5.0(Internet Information Services) in 2001 which caused almost 2.5 billion USD in damages [22]. Another well known vulnerability is the Heartbleed bug of April, 2014(CVE-2014-0160)[36], which was found in the open source software, OpenSSL that affected a vast majority of the websites, VPN concentrators, client application and mobile devices. TLS (Transport Layer Software, successor to SSL) is used to secure connection between two end points to prevent intruders from gaining access to sensitive information. However, after the initial handshake, when data transfer is completed, the connection is terminated. Heartbeat extension was added to keep the connection open by default even when data transfer had ceased as long as the client continues to receive Heartbeat response to its corresponding request. The attacker could exploit the BOF vulnerability by constructing a request with a payload length as high as 65,535 bytes and as a result, the OpenSSL implementation placed the tainted input into memory(without checking the size of the payload) and copied the section of memory up to the attacker’s specified payload length into the heartbeat response. This could lead to leak of information ranging from usernames, passwords, to possibly encryption keys. 8 Even though Buffer Overflow vulnerabilities can be minimized by using library functions with bounds checking and reducing programming errors but in spite of taking proper precautions, programmers tend to make mistakes and hence other forms of detection techniques have also been applied to the modern OS. Unlike C and C++, buffer overflow is not a problem in Java programming language. Fuzzing, an automated software testing technique is often used to detect bugs in a program by providing random data as the input to detect unpredictable behaviour due to vulnerabilities [13].