Building Security In Editor: Gary McGraw, [email protected]
Software Security Testing
ecurity testing has recently moved beyond the be easier to exploit. Because attacks are now becoming more sophisti- realm of network port scanning to include probing cated, the notion of which vulnera- bilities actually matter is changing. software behavior as a critical aspect of system be- Although timing attacks, including the well-known race condition, havior (see the sidebar). Unfortunately, testing were considered exotic just a few S years ago, they’re common now. software security is a commonly misunderstood task. Security Similarly, two-stage buffer overflow attacks using trampolines were once BRUCE POTTER testing done properly goes deeper Security is always relative to the the domain of software scientists, but Booz Allen than simple black-box probing on the information and services being pro- now appear in 0day exploits.4 Hamilton presentation layer (the sort performed tected, the skills and resources of ad- Design-level vulnerabilities are by so-called application security versaries, and the costs of potential the hardest defect category to han- GARY tools)—and even beyond the func- assurance remedies; security is an ex- dle, but they’re also the most preva- MCGRAW tional testing of security apparatus. ercise in risk management. Risk lent and critical. Unfortunately, as- Cigital Testers must use a risk-based ap- analysis, especially at the design level, certaining whether a program has proach, grounded in both the sys- can help us identify potential secu- design-level vulnerabilities requires tem’s architectural reality and the at- rity problems and their impact.1 great expertise, which makes finding tacker’s mindset, to gauge software Once identified and ranked, soft- such flaws not only difficult, but par- security adequately. By identifying ware risks can then help guide soft- ticularly hard to automate. risks in the system and creating tests ware security testing. Examples of design-level prob- driven by those risks, a software se- A vulnerability is an error that an lems include error handling in ob- curity tester can properly focus on attacker can exploit. Many types of ject-oriented systems, object sharing areas of code in which an attack is vulnerabilities exist, and computer and trust issues, unprotected data likely to succeed. This approach pro- security researchers have created tax- channels (both internal and exter- vides a higher level of software secu- onomies of them.2 Security vulnera- nal), incorrect or missing access con- rity assurance than possible with bilities in software systems range trol mechanisms, lack of auditing/ classical black-box testing. from local implementation errors logging or incorrect logging, and or- (such as use of the gets() function dering and timing errors (especially What’s so different call in C/C++), through interpro- in multithreaded systems). These about security? cedural interface errors (such as a sorts of flaws almost always lead to Software security is about making race condition between an access security risk. software behave in the presence of a control check and a file operation), malicious attack, even though in the to much higher design-level mis- Risk management real world, software failures usually takes (such as error handling and re- and security testing happen spontaneously—that is, covery systems that fail in an insecure Software security practitioners per- without intentional mischief. Not fashion or object-sharing systems form many different tasks to manage surprisingly, standard software test- that mistakenly include transitive software security risks, including ing literature is only concerned with trust issues). Vulnerabilities typically what happens when software fails, fall into two categories—bugs at the • creating security abuse/misuse regardless of intent. The difference implementation level and flaws at cases; between software safety and software the design level.3 • listing normative security re- security is therefore the presence of Attackers generally don’t care quirements; an intelligent adversary bent on whether a vulnerability is due to a • performing architectural risk breaking the system. flaw or a bug, although bugs tend to analysis;
32 PUBLISHED BY THE IEEE COMPUTER SOCIETY ■ 1540-7993/04/$20.00 © 2004 IEEE ■ IEEE SECURITY & PRIVACY Building Security In
• building risk-based security test Security External Static Penetration plans; requirements review analysis testing • wielding static analysis tools; (tools) • performing security tests; Abuse Risk Risk-based Risk cases analysis security tests analysis Security • performing penetration testing in breaks the final environment; and • cleaning up after security breaches.
Three of these are particularly Requirements Design Test Code Test Field closely linked—architectural risk and use cases plans results feedback analysis, risk-based security test plan- Figure 1. The software development life cycle. Throughout this series, we’ll focus on ning, and security testing—because a specific parts of the cycle; here, we’re examining risk-based security testing. critical aspect of security testing relies on probing security risks. Last issue’s installment1 explained how to ap- problem. Software testers charged rity and Exploiting Software help edu- proach a software security risk analy- with security testing often fall prey to cate testing professionals on how to sis, the end product being a set of se- the same thinking. think like an attacker.4,5 Nevertheless, curity-related risks ranked by business software exploits are surprisingly so- or mission impact. (Figure 1 shows How to approach phisticated these days, and the level of where we are in our series of articles security testing discourse found in books and articles about software security’s place in the Like any other form of testing, secu- is only now coming into alignment. software development life cycle.) rity testing involves determining White- and black-box testing and The pithy aphorism, “software se- who should do it and what activities analysis methods both attempt to un- curity is not security software” pro- they should undertake. derstand software, but they use differ- vides an important motivator for secu- ent approaches depending on rity testing. Although security features Who whether the analyst or tester has ac- such as cryptography, strong authenti- Because security testing involves two cess to source code. White-box cation, and access control play a criti- approaches, the question of who analysis involves analyzing and under- cal role in software security, security it- should do it has two answers. Stan- standing source code and the design. self is an emergent property of the dard testing organizations using a It’s typically very effective in finding entire system, not just the security traditional approach can perform programming errors (bugs when au- mechanisms and features. A buffer functional security testing. For ex- tomatically scanning code and flaws overflow is a security problem regard- ample, ensuring that access control when doing risk analysis); in some less of whether it exists in a security mechanisms work as advertised is a cases, this approach amounts to pat- feature or in the noncritical GUI. classic functional testing exercise. tern matching and can even be auto- Thus, security testing must necessarily On the other hand, traditional mated with a static analyzer (the sub- involve two diverse approaches: QA staff will have more difficulty ject of a future installment of this performing risk-based security test- department). One drawback to this 1. testing security mechanisms to ing. The problem is one of expertise. kind of testing is that it might report a ensure that their functionality is First, security tests (especially those potential vulnerability where none properly implemented, and resulting in complete exploit) are actually exists (a false positive). Nev- 2. performing risk-based security difficult to craft because the designer ertheless, using static analysis methods testing motivated by under- must think like an attacker. Second, on source code is a good technique standing and simulating the at- security tests don’t often cause direct for analyzing certain kinds of soft- tacker’s approach. security exploit and thus present an ware. Similarly, risk analysis is a white- observability problem. A security box approach based on a deep under- Many developers erroneously test could result in an unanticipated standing of software architecture. believe that security involves only outcome that requires the tester to Black-box analysis refers to ana- the addition and use of various secu- perform further sophisticated analy- lyzing a running program by probing rity features, which leads to the in- sis. Bottom line: risk-based security it with various inputs. This kind of correct belief that “adding SSL” is testing relies more on expertise and testing requires only a running pro- tantamount to securing an applica- experience than we would like. gram and doesn’t use source-code tion. Software security practitioners analysis of any kind. In the security bemoan the over-reliance on “magic How paradigm, malicious input can be crypto fairy dust” as a reaction to this Books like How to Break Software Secu- supplied to the program in an effort
www.computer.org/security/ ■ IEEE SECURITY & PRIVACY 33 Building Security In
cards with new functionality—such From outside→in to inside→out as the ability to provide secure card- holder identification or remember raditional approaches to computer and network security testing focus on network infra- personal preferences—many credit- Tstructure, firewalls, and port scanning. The notion is to protect vulnerable systems (and card companies are turning to multi- software) from attack by identifying and defending a perimeter. In this paradigm, testing application smart cards. These cards focuses on an outside→in approach. One classic example is the use of port scanning with tools use resident software applications to such as nmap
34 IEEE SECURITY & PRIVACY ■ SEPTEMBER/OCTOBER 2004 Building Security In
critical implementation errors on functional requirements is an ab- uncovered under normal functional any given card. Testing for this sort of solute necessity for customers to ac- security testing. Fielding cards with risk involves creating and manipulat- cept the cards. After all, they must these vulnerabilities would let an at- ing stored objects where sharing is perform properly worldwide. tacker execute successful attacks on involved. Given a technical descrip- However, every card submitted to live cards issued to the public. Be- tion of this risk, building specific the risk-based testing paradigm ex- cause of proper risk-based security probing tests is possible. hibited some manner of failure when testing, the vendors were notified of tested with the hostile applet suite. the problems and corrected the code Automating Some failures pointed directly to crit- responsible before release. security testing ical security vulnerabilities on the Over the years, we (the authors) have card; others were less specific and re- been involved in several projects that quired further exploration to deter- here is no silver bullet for soft- have identified architectural risks in mine the card’s true security posture. T ware security; even a reasonable the GP/Java Card platform, sug- As an example, consider that risk security testing regimen is just a start. gested several design improvements, analysis of Java Card’s design docu- Unfortunately security continues to and designed and built automated ments indicates that proper imple- be sold as a product, and most defen- security tests for final products (each mentation of atomic transaction sive mechanisms on the market do of which had multiple vendors). processing is critical for maintaining little to address the heart of the prob- Several years ago, Cigital began a secure card. Java Card has the capa- lem, which is bad software. Instead, developing an automated security bility of defining transaction bound- they operate in a reactive mode: test framework for GP cards built on aries to ensure that if a transaction don’t allow packets to this or that Java Card 2.1.1 and based on exten- fails, data roll back to a pre-transac- port, watch out for files that include sive risk analysis results. The end re- tion state. In the event that transac- this pattern in them, throw partial sult is a sophisticated test framework tion processing fails, transactions can packets and oversized packets away that runs with minimal human inter- go into any number of possible states, without looking at them. Network vention and results in a qualitative se- depending on what the applet was traffic is not the best way to approach curity testing analysis of a sample attempting. In the case of a stored- this predicament, because the soft- smart card. value card, bad transaction process- ware that processes the packets is the The first test set, the functional ing could allow an attacker to “print problem. By using a risk-based ap- security test suite, directly probes money” by forcing the card to roll proach to software security testing, low-level card security functionality. back value counters while actually testing professionals can help solve It includes automated testing of class purchasing goods or services. security problems while software is codes, available commands, and When creating risk-based tests to still in production. crypto functionality. This test suite probe transaction processing, we di- also actively probes for inappropriate rectly exercised transaction-process- References card behavior of the sort that can lead ing error handling by simulating an 1. D. Verndon and G. McGraw, “Risk to security compromise. attacker attempting to violate a Analysis in Software Design,” IEEE The second test set, the hostile transaction—specifically, transac- Security & Privacy, vol. 2, no. 4, applet test suite, is a sophisticated tions were aborted or never commit- 2004, pp. 79–84. set of intentionally hostile Java ted, transaction buffers were com- 2. C.E. Landwehr et al., A Taxonomy Card applets designed to probe pletely filled, and transactions were of Computer Program Security Flaws, high-risk aspects of the GP on a nested (a no-no according to the Java with Examples, tech. report NRL/ Java Card implementation. Card specification). These tests were FR/5542—93/9591, Naval not based strictly on the card’s func- Research Laboratory, Nov. 1993. Results: tionality—instead, security test en- 3. G. McGraw, “Software Security,” Nonfunctional security gineers intentionally created them, IEEE Security & Privacy, vol. 2, no. testing is essential thinking like an attacker given the 2, 2004, pp. 80–83. Most cards tested with the automated results of a risk analysis. 4. G. Hoglund and G. McGraw, test framework pass all functional se- Several real-world cards failed Exploiting Software, Addison-Wes- curity tests, which we expect because subsets of the transaction tests. The ley, 2004. smart-card vendors are diligent with vulnerabilities discovered as a result 5. J. Whittaker and H. Thompson, functional testing (including security of these tests would let an attacker How to Break Software Security, functionality). Because smart cards terminate a transaction in a poten- Addison-Wesley, 2003. are complex embedded devices, ven- tially advantageous manner, a critical dors realize that exactly meeting test failure that wouldn’t have been Bruce Potter is a senior associate with
www.computer.org/security/ ■ IEEE SECURITY & PRIVACY 35 Building Security In
Booz Allen Hamilton. He is also the founder of the Shmoo Group of security professionals. His areas of expertise include wireless security, large-scale net- work architectures, smartcards, and pro- motion of secure software engineering practices. Potter coauthored the books 802.11 Security (O’Reilly and Associates, 2003) and Mac OS X Security (New Rid- ers, 2003); he’s currently coauthoring Master FreeBSD and OpenBSD Security (O’Reilly and Associates, summer 2004). He was trained in computer science at the University of Alaska, Fairbanks.
Gary McGraw is chief technology officer of Cigital. His real-world experience is grounded in years of consulting with major corporations and software pro- ducers. He serves on the technical advi- sory boards of Authentica, Counterpane, Fortify, and Indigo. He also is coauthor of Exploiting Software (Addison-Wesley, 2004), Building Secure Software (Addi- son-Wesley, 2001), Java Security (John Wiley & Sons, 1996), and four other books. He has a BA in philosophy from the University of Virginia and a dual PhD in computer science and cognitive science from Indiana University. Contact him at [email protected].
36 IEEE SECURITY & PRIVACY ■ SEPTEMBER/OCTOBER 2004