Data collection on security flaws caused by design errors Harald Terkelsen Master’s Thesis Master of Science in Information Security 30 ECTS Department of Computer Science and Media Technology Gjøvik University College, 2006 Institutt for informatikk og medieteknikk Høgskolen i Gjøvik Postboks 191 2802 Gjøvik Department of Computer Science and Media Technology Gjøvik University College Box 191 N-2802 Gjøvik Norway Data collection on security flaws caused by design errors Preface After three years of studying in my spare time while working a full time day job, it has not always been easy to find the energy or the time to work on the thesis in the evening after a long day at work first. But the motivation of finishing what I started and getting the master’s degree has kept me going. The time has unfortunately run out, and the thesis is not as complete as I hoped it should be at this time. I would like to thank my employer, Oslo University College (Høgskolen i Oslo), for letting me have days off work to follow the lectures at Gjøvik two days a week, six weeks each semester to pursue the masters degree. I also thank my supervisor Hanno Langweg, who originally proposed the topic, for his feedback on the drafts he reviewed, and also for being patience with my slow progression being a part time student. iii Data collection on security flaws caused by design errors Contents Preface .......................................... iii Contents ......................................... v 1 Introduction ..................................... 1 2 Previous and related work ............................. 5 2.1 Software design ................................. 5 2.1.1 Design principles ............................ 5 2.1.2 Security design patterns ........................ 7 2.2 Taxonomy and classification theory ...................... 7 2.3 Software flaw taxonomies ........................... 8 2.3.1 Security taxonomies .......................... 8 2.3.2 Defect classifications .......................... 16 2.4 The Common Vulnerabilities and Exposure list (CVE) ............ 17 2.5 Common Criteria ................................ 17 3 Software Design and Software Flaws ....................... 19 3.1 Software design ................................. 19 3.2 What is a design flaw? ............................. 22 3.3 Design versus Implementation ......................... 22 3.4 Some common implementation flaws and why they are not design flaws . 23 3.5 Design versus configuration .......................... 25 4 Vulnerability databases ............................... 27 4.1 Databases, Advisories and alerts ........................ 27 4.2 Evaluation and selection of databases ..................... 27 4.2.1 National Vulnerability Database .................... 29 4.2.2 Open Source Vulnerability Database ................. 29 4.3 Method for data collection ........................... 30 5 Developing a design flaw classification scheme ................. 33 5.1 Design elements from existing classifications and related work ....... 33 5.2 Design properties found in vulnerabilities from the databases ....... 34 5.3 The proposed classification .......................... 35 5.3.1 Description ............................... 35 5.3.2 Selection criteria when using the classification ............ 39 5.4 Other possible approaches ........................... 40 6 Applying the classification ............................. 43 6.1 Data source and method ............................ 43 6.2 Experience gained from using the classification ............... 44 6.3 Results from applying the classification .................... 44 7 Discussions ...................................... 47 7.0.1 The need for more details in the vulnerability databases ...... 48 8 Conclusions and future work ........................... 49 Bibliography ....................................... 51 v Data collection on security flaws caused by design errors A Vulnerability databases ............................... 55 B Design flaw vulnerabilities by category ..................... 61 C Vulnerabilities identified as design flaws .................... 65 D Vulnerabilities identified as maybe design flaws ................ 81 E Vulnerabilities identified as not design flaws .................. 91 F Vulnerabilities identified as unknown type ................... 115 G Programs to work with National Vulnerability Database ............ 119 G.1 nvd-parse.py .................................. 119 G.2 nvd-manage.py ................................. 123 vi Data collection on security flaws caused by design errors 1 Introduction Producing secure software is extremely hard to do right. The number of security flaws and vulnerabilities discovered in software each day is increasing at high speed. According to the National Vulnerability Database the number of vulnerabilities discovered in 2005 was 4859, more than twice the number of vulnerabilities discovered the year before. One way to classify vulnerabilities is to classify them after when in the development phase they are introduced, for example[28]: • Design • Implementation • Configuration Other phases like analysis (requirements), testing, or maintenance phase are also sometimes used [40, 34]. This thesis is a study of security related flaws with origin in the design phase. Such flaws are rooted in the design of the software, and exist even if the programmer im- plements the design perfectly making no mistakes in the programming. Security related design flaws are a lot about how security mechanisms like authentication, authorization and encryption are used and implemented, or how error handling is performed. Typical examples of design flaws include weak encryption, missing or insufficient access control. What make design flaws different from other classes of flaws? • They often cost more to fix. • They are not easily detected by testing or static analysis tools as implementation flaws. • Two different implementations of a flawed design will both be vulnerable Implementation flaws can often easily be fixed with small changes to the code, adding an extra check or using a more secure API call. Design flaws, on the other hand, are rooted in the design. Fixing them means making changes to the design. If the design is already implemented, changes to the design often means recoding parts of the code, sometimes changing larger portions of the code. The later in the development phase a design flaw is found, the more it cost to fix it. It can be 6.5 times more expensive to fix a flaw during the implementation phase than during design. And if the flaw is not found before the maintenance phase, it can cost as much as 60-100 times more. [18] Static analysis tools are programs built to detect flaws by automatically examine the source code of other programs. Design flaws are not as easily detected by static analysis methods as implementation flaws are. This means it is harder to create tools to auto- mate the search for design flaws. Several studies conclude that design reviews are more efficient for detecting design flaws than testing[44]. 1 Data collection on security flaws caused by design errors Security functionality are often added to software applications at the end of the de- velopment process when the rest of the functionality is finished, with weak security solu- tions as a result. There have been launched several books, reports and projects lately to increase the awareness and the importance of including security considerations in soft- ware projects from the start and throughout the entire software development process. Most notably is the Build security in1 project by the Strategic Initiatives Branch of the Na- tional Cyber Security Division (NCSD) of the Department of Homeland Security (DHS) in USA. The mission statement on the project’s web page describe the goal as “seeks to reduce software vulnerabilities, minimize exploitation, and address ways to improve the routine development and deployment of trustworthy software products“. The Internet Engineering Task Force (IETF) has also seen the need to focus more on security in their protocols. In RFC 3631 from 2003 they state that all protocols should provide appropri- ate security mechanisms, even if it is believed that the protocol’s domain of application is limited. Taxonomies and classification schemes are tools which can be used to classify and analyze the different vulnerabilities and flaws that can be found in computer software. Over the years many classification schemes for software vulnerabilities have been pro- posed. They take different approaches to classifying vulnerabilities, depending on the researchers goal. Many of them have a special focus on operating system flaws. From a designers view point, an operating system has a more standardized set of functionality needed to be implemented than a software application which can be one of a kind. But many of the classifications that consider application flaws, are based on the same cate- gories as those designed for classifying operating system flaws. This leads us to believe they might not be optimal in classifying design flaws. Collections of discovered vulnerabilities exist in different vulnerability databases. Sev- eral of these databases have a classification scheme they use to indicate the type of vul- nerability when data on the vulnerabilities are presented. Categories for design flaws exist, but only the top-level class ”design flaw“. No sub-categories for different types of design flaws seems to be used. The classification of the vulnerabilities does not seems to be consistent over the different databases. A vulnerability