JULY 2005 Vol.8. No.2 Secure Software

The Challenge of Low Defect, Secure Software ...... 8 Enhancing Customer Security ...... 11 Security ...... 15 Comment ...... 18 Unclassified and Unlimited Distribution 2 STN 8-2: Software Cost, Quality and Productivity Benchmarks Developing Secure Software

Noopur Davis, Institute

Abstract This article begins with a discussion , invalid , Most security vulnerabilities result of why defective software is seldom incorrect use of , failure from defects that are unintentionally secure, why defective software is not to protect data, and failure to carefully introduced in the software during design inevitable, and why reducing defects is partition applications. But most are and development. Therefore, to signifi- less costly than responding to released caused by simple oversight that leads cantly reduce software vulnerabilities, vulnerabilities. Next, security through- to defect types such as declaration er- the overall defect content of software out the software development life cycle rors, logic errors, loop control errors, must be reduced. Defect reduction is a will be discussed. The paper closes conditional expressions errors, failure pre-requisite for secure software devel- with a brief description of the Software to validate input, interface specification opment, but it is not enough. Security Engineering Institute’s (SEI’s) Team errors, configuration errors, and failure must also be deeply integrated into the Software ProcessSM for Secure Software to understand basic security issues. In full software development life cycle Development (TSP-Secure). a recent interview, Alan Paller, direc- (SDLC). tor of research at the SANS Institute, “expressed frustration with the fact that Defective Software Is Seldom Secure everything on the [SANS Institute Top Introduction SEI analysis of thousands of 20 ] vulnerability list Most security vulnerabilities result programs produced by thousands of is a result of poor coding, testing and from defects that are unintentionally developers show that even experienced sloppy software engineering. These are introduced in the software during design developers inject numerous defects as not ‘bleeding edge’ problems, as an in- and development. Therefore, to signifi- they perform activities for understand- nocent bystander might easily assume. cantly reduce software vulnerabilities, ing requirements, developing designs, Technical solutions exist to them all, but the overall defect content of software coding, and testing software. One defect they are simply not implemented.” is injected for every 7 to 10 lines of new must be reduced. Today’s common It is clear that software development and changed code produced. Even if software engineering practices lead to practices in common use today lead to 99% of these defects are removed before a large number of defects in released defective software, that software defects the software is released, this leaves 1 to software. However, data from dozens are a principal cause of software secu- 1.5 defects in every thousand lines of of real-world software projects that rity vulnerabilities; therefore, to reduce new and changed code produced. Soft- have systematically applied improved vulnerabilities the overall defect content ware benchmark studies conducted on software development practices show of software must be reduced. one to two orders of magnitude reduc- hundreds of software projects show that tion in the number of defects in released the average defect content of released software. Applying these improved software varies from about 1 to 7 defects Defective Software Is Not Inevitable practices should lead to a similar reduc- per thousand lines of new and changed When presented with the security tion in the defects that lead to vulner- code [Jones]. problems caused by defective software, abilities. Furthermore, by focusing on According to preliminary analysis a common response is that software the specific types of defects that lead to done by the SEI’s CERT® group, over development is inherently prone to vulnerabilities, even greater reduction 90% of software security vulnerabilities defects, and that defective software in vulnerabilities could be achieved. are caused by known software defect is somehow inevitable. Many people Organizations that have applied these types. The analysis also showed that believe that trying to figure out how practices have realized additional ben- most software vulnerabilities arise from to build better software is “a no-win efits of reduced cycle times and reduced common causes: the top ten causes situation and just beating a dead horse” software development costs. account for about 75% of all vulner- [ World]. However, data Along with defect reduction, Secu- abilities. Another analysis of forty-five from dozens of real-world projects have rity must be deeply integrated into the e-business applications showed that 70% shown that when developers follow de- full software development life cycle of the security defects were software de- fined, measured, and quality controlled (SDLC). Security must be “built-in” sign defects [Jacquith]. Some problems practices, they produce products with while the product is being developed, are caused by sophisticated architectural very few overall defects. A recent study and not just “bolted-on” after the fact. and design issues such as inadequate continues on page 4

Data & Analysis Center for Software (DACS) 3 Developing Secure Software Continued from page 3.

found that the defect content of such Secure Software Development each removal step? products can be reduced to an average What can be done to reduce defects Suppose an organization has of .06 defects per thousand lines of in software, and thus reduce vulnerabili- determined that it wants to produce new and changed code [Davis]. This ties in software? Two things must be software with less than 1 vulnerability represents 10 to 100 times fewer defects done: defects must be managed through- per million lines of code. Also suppose when compared to industry averages of out the software development life cycle, that 25% of all software defects can 1 to 7 defects per thousand lines of new and security must be addressed through- lead to software vulnerabilities. Thus and changed code. out the software development life cycle. the quality goal for the organization is to release software with less than 4 defects per million lines of code. How The Cost of Reducing Defects Managing Defects throughout the will the organization know it can deliver The next question usually asked Software Development Life Cycle an acceptably small number of defects is “doesn’t it cost too much to reduce Defects delivered in released soft- defects in software”? The simple to meet its quality goals? Like most ware are a percentage of the total defects organizations, suppose the first time this answer is that software projects that introduced during the software devel- produce near defect-free software also organization measures defects in the opment life cycle. To reduce defects software development life cycle is dur- consistently meet their schedules (thus in released software, defects must be avoiding costs associated with delayed ing test. If testing exposes 100 defects managed throughout the software devel- per million lines of code, and like most releases), and spend less time on soft- opment life cycle. Defect management ware repair (thus improving overall organizations, testing in this organiza- includes both defect removal and defect tion is 50% effective, 100 defects per productivity). For example, the average measurement. schedule error for projects using best million line of code would remain in practices was just 6%, the average time There should be multiple defect the software after testing, and would be spent on software repair was just 4%, removal points in the software devel- released with the software (200 defects and the average increase in productivity opment life cycle. The more defect per million lines of code existed before was 78% [Davis]. Another large scale removal points there are, the closer one the software entered test, 50% of these study showed a near perfect corelation is to finding problems right after they defects were found and fixed during between schedule and quality: the fewer are introduced. So the problems can be test, while another 50% remained un- the defects in the software, the lesser the more easily fixed, and the root cause found and unfixed). Not only will the schedule error [Jones]. more easily determined and addressed. organization not meet its quality goal, but few options will be available for When discussing costs, it is also Each time defects are removed, corrective action at this late stage in the fair to discuss the costs of releasing they should be measured. Every defect development life cycle. software with vulnerabilities. Producers removal point becomes a measurement of vulnerable software face the tangible point. Defect measurement leads to On the other hand, if this organiza- costs of fixing and releasing patches for something even more important than de- tion had several defect removal points vulnerabilities, as well as the intangible fect removal and prevention: it tells you in the software life cycle, each 50% ef- costs of bad press, customer dissatisfac- where you stand against your goals now, fective, the defects in the released soft- tion, and threat of legal action. For helps you decide whether to move to the ware would be much fewer. Each de- consumers, the costs are even higher. next step or to stop and take corrective fect removal activity can be thought of A recent analysis conducted at a major action, and indicates where to fix your as a filter that removes some percentage corporation determined that the cost to process to meet your goals. of defects that can lead to vulnerabilities deploy a single patch was close to half The following questions must be from the software product, while others a million dollars. This cost was incurred considered when managing defects: defects that can lead to vulnerabilities just by the corporate infrastructure team: where are the places in the software escape the filter and remain in the soft- it did not include costs incurred by other development life cycle where defects ware (see Figure 1). The more defect teams such as the development teams. should be measured? What work prod- removal filters there are in the software When these costs are multiplied by hun- ucts should be examined for defects? development life cycle, the fewer de- dreds of patches that need to be applied What tools and methods should be used fects that can lead to vulnerabilities will by thousands of corporations, the overall to measure the defects? How many de- remain in the software product when costs to the consumers are enormous. fects can be removed at each step? How it is released. More importantly, be- many estimated defects remain after continues on page 5

4 STN 8-2: Software Cost, Quality and Productivity Benchmarks Developing Secure Software Continued from page 4.

problem. But what are some best prac- Requirements tices that would address not only buffer Activities overflows, but other potential defects Some % of vuls injected during as well? A specific design practice may % requirements activities are removed during , threat be input validation via custom typed modeling, or developing abuse cases classes. A specific verification practice may be state machine verification for session management. A specific cod- Design ing practice may be language specific, Activities checklist-based security code reviews. Some % of vuls injected during A specific tool may be a static code ana- % requirements and design activities are lyzer that scans the code for potentially removed during design reviews and verification unsafe calls. A specific testing method may be Fuzz testing. Just as important as defining best practices is Implementation deciding when in the secure software Activities development process these practices

Some % of vuls injected during should be used (process scripts), how % requirements, design, and coding they should be measured (in-process as are removed during code reviews, well as predictive measures), and how dynamic analysis, static analysis, their use can be ensured. and testing Some % of vuls escape all removal Once the best practices have been filters and are released with the defined, they must be applied through- software. out the software development life cycle. Figure 1: Vulnerability Removal Filters Figure 2 shows some best practices cause the defects were being measured examples: some organizations have that address security through different earlier, the organization would have 700-page documents to teach developers phases of a software development life time to take corrective action early in about common causes of vulnerabilities cycle. No life cycle model is implied. the software development life cycle. and how to avoid them. No one should For spiral, incremental, or iterative de- velopment, best practices will be cycled Some examples of defect removal expect developers to use such a volume of information as they perform their through more than once as the software and measurement points in the software product evolves. development life cycle are architectural day to day software development activi- analysis, threat modeling, design verifi- ties. Although an overall knowledge of Examples of SDLC best practices cation, design review, code review, static security issues is important, eliminating include security risk analysis, secure code analysis, unit test, penetration test, common causes of vulnerabilities re- design principles (such as defense in and system test. quires defining a set of operational best depth, application partitioning, and least practices that development teams can privilege), threat modeling, static code

use in their day to day work: scripts, analysis, checklist based inspections Addressing Security throughout the tools, checklists, and methods that focus and reviews, and testing methods such Software Development Life Cycle on the particular job the developer is do- as Fuzz testing, Ballista, or penetration Although defect reduction is the ing at a particular time. testing. key to vulnerability reduction, more is For example, consider buffer over- Since schedule pressures and lack needed to produce secure software. flows, the most common and arguably of senior management sponsorship First, common causes of security the best understood cause of software can get in the way of implementing vulnerabilities must be understood. vulnerabilities. Teaching develop- best practices, organizational support Some common causes include buf- ers about buffer overflows, showing is needed for setting security policies, fer overflows, SQL injection, race them examples of code that leads to providing management oversight for conditions, and cross-site scripting. overflows, and cataloging library calls security activities, and for providing Understanding involves much more that are prone to buffer overflows are all security training and resources. Project than reading a laundry list of causes and good ways to sensitize developers to this continues on page 6

Data & Analysis Center for Software (DACS) 5 Developing Secure Software Continued from page 5.

Figure 2: Addressing Security Throughout the Software Development Lifecycle

Organizational policies, management oversight, resources and training, project planning, project tracking, risk management, measurement and feedback

Requirements Design Implementation Testing Threat modeling Security design Security specifications principles “Secure ” language (defense in depth, Asset identification subsets Security test plans Use cases least privilege …) Coding standards Black-box testing Abuse cases Security feature design (authentication, Code reviews White-box testing authorization, Static analysis tools Test-defect review secure communication, Dynamic analysis tools secure data storage, configuration...) Design review

management is needed to ensure that developed life cycle The Team Software Process for security activities are planned and 3) control the process through mea- Secure Software Development (TSP- tracked. Risk management is needed surement Secure) augments the TSP with secu- to ensure security risks are identified, rity practices throughout the software 4) monitor the process assessed, and managed. development life cycle. The research 5) address defect prevention as well as Finally, the secure software devel- objectives of TSP-Secure are to reduce removal opment process should be measured or eliminate software vulnerabilities to determine its effectiveness, and to 6) use predictive measures for remain- that result from and determine which measures are predictive ing defects implementation defects, and to provide measures for latent vulnerabilities in Since schedule pressures and people the capability to predict the likelihood released software. issues get in the way of implementing of latent vulnerabilities in delivered best practices, the TSP helps build self- software. Areas of exploration include directed development teams, and then vulnerability analysis by defect type, The Team Software Process for Secure puts these teams in charge of their own operational process for secure software Software Development work. TSP teams: production, predictive process metrics The Software Engineering Institute and checkpoints, quality management 1) develop their own plans developed the Team Software Process practices for secure programming, (TSP)SM as a set of defined and mea- 2) make their own commitments design patterns for common vulnerabili- sured best practices for use by individual 3) track and manage their own work ties, verification techniques, and remov- software developers and software de- 4) take corrective action when needed ing vulnerabilities in legacy software. velopment teams [Humphrey]. Teams The TSP includes a systematic Teams using TSP-Secure are first using the TSP: way to train software developers and trained in fundamental software engi- 1) use common sense software engi- managers, to introduce the methods into neering practices. They then attend a neering practices an organization, and to involve manage- workshop where they are introduced to 2) manage defects throughout the ment at all levels. continues on page 7

6 STN 8-2: Software Cost, Quality and Productivity Benchmarks Developing Secure Software Continued from page 6.

common causes of vulnerabilities and she works with Watts Humphrey on his ogy, October 12, 2004. http://www. practices they should use to address Team Software Process initiative. She globetechnology.com/servlet/story/RT- the common causes of vulnerabilities. is also Principal of Davis Systems, a GAM.20041001.gtkirwanoct1/BNStory/ Next, the teams plan their product company that has been providing Soft- Technology/ development work. Along with busi- ware Engineering Process Improvement [Humphrey] Humphrey, Watts S. ness and feature goals, teams define the consulting and training services for over Winning with Software: An Executive security goals for their product, and then twelve years. Noopur has been involved Strategy. Reading, MA: Addison-Wes- measure and track the security goals in the software field for over twenty ley, 2002. throughout the product development years as a developer, a manager, and a [Jacquith] Jacquith, Andrew. “The life cycle. At least one team member consultant. Her experience ranges from Security of Applications: Not All Are assumes the role of Security Manager. real-time embedded systems software Created Equal.” At Stake Research. This role is responsible for ensuring that to commercial desktop products. She the team is addressing security through has launched and coached dozens of http://www.atstake.com/research/re- all their product development activities. teams at major industry and government ports/acrobat/atstake_app_unequal.pdf To date, the TSP has been used by organizations. [Jones] Jones, Capers. Software many organizations. A recent study She has authored several reports Assessments, Benchmarks, and Best showed that teams using the TSP and articles on software process and Practices. Reading, MA: Addison-Wes- produced software with an average software security. ley, 2000. delivered defect level of 0.06 defects Noopur has a Masters in Computer [Viega] Viega, Jones and McGraw, per thousand lines of new and changed Science and a Bachelors in Electrical Gary Building Secure Software Build- code, with an average schedule error Engineering. She is an SEI-Authorized ing Secure Software: How to Avoid of just 6%. The average productivity Team Software Process coach, an SEI- Security Problems the Right Way, improvement was 78%. TSP-Secure is Authorized Personal Software Process Reading, MA: Addison Wesley, 2001. still under development, but an initial instructor, program committee member proof-of-concept pilot produced near for the 2003 XP/Agile Universe confer- SM Team Software Process and TSP defect free software with no security ence, program committee member for are service marks of Carnegie Mellon defects found during security audits and International Symposium on Secure University. in several months of use. Software Engineering, member of IEEE ®CERT is a registered trademark of working group for draft recommended Carnegie Mellon University. Conclusion ractices for Establishing and Managing Software Development Efforts Using Since common software defects Agile Methods, and a member of the are a leading cause of vulnerabilities, IEEE and the ACM. the overall defect content of software must be reduced. Next, security must be systematically addressed throughout References the software development life cycle. [Computer World] “Congress’ role There must be a shift in attitude from in IT security debated”, November 6, “bolting security on” after the fact, to 2003. “building security in” as the product http://www.computerworld. is being developed. This requires that com/securitytopics/security/sto- good software engineering practices are ry/0,10801,86902,00.html?nas=PM- followed while the software is being 86902 developed, including multiple defect removal activities. [Davis] Davis, Noopur and Mul- laney, Julia, “The Team Software Pro- cess in Practice: A Summary of Recent Biography of Noopur Davis Results.”, Technical Report CMU/SEI- Noopur Davis is a Visiting Scientist 2003-TR-014, September 2003. at the Software Engineering Institute [Kirwan] Kirwan, Mary, “The Quest of Carnegie Mellon University, where For Secure Code”. Global Technol-

Data & Analysis Center for Software (DACS) 7 The Challenge of Low Defect, Secure Software – too difficult and too expensive? By Martin Croxford, Praxis High Integrity Systems Limited

The software industry is in crisis. port, an approach known as Correctness tional behavior is observed! A strong claim? The National Institute by Construction developed by Praxis It is the use of precision which dif- of Standards and Technology (NIST) High Integrity Systems Limited in ferentiates Correctness by Construction reports that poor quality software costs England, has been used for over fifteen from other approaches. While perhaps the US economy $60 Billion per year years to produce very low defect soft- relying only on good process, many [1]. According to the aptly named ware in critical applications. As well other software development approaches Chaos report only a quarter of software as being low defect, such software has endure a lack of precision which makes projects are judged a success [2]. also proved to be highly cost-effective it very easy to introduce errors, and Failures due to “computer glitches” are to develop and maintain in operation. very hard to find those errors early. commonplace, and seem to be viewed Two examples are cited below, includ- Evidence for this may be found in the by the public (if not by the software ing a zero defect security application. common tendency for development industry itself) as inevitable. In any However, given that Correct- lifecycles to migrate to an often-repeat- other engineering discipline, or indeed ness by Construction (and the other ing “code-test-debug” phase, which can any field, engineering or otherwise, best practice approaches cited in the lead to severe cost and timescale over- this would be unacceptable. But in the report) has been used successfully for a runs. So, what kind of precision does safety and security sector, where reli- number of years, some questions arise. Correctness by Construction use? ance on correctly functioning software Why is there still so much poor qual- At the requirements step (a source is increasing, and where such software ity software around? Why are these of half of project failures [2]) a clear is becoming ever larger and more com- approaches not in more widespread distinction is made between user re- plex, this state of affairs is unsustain- use? Perhaps the real challenge for the quirements, system specifications and able. software industry is to find a way of domain knowledge, and “satisfaction The challenge for the software in- systematically applying known technol- arguments” are used to show that each dustry has been neatly summarized by ogy? user requirement can be satisfied by Martyn Thomas, visiting Professor of Before exploring these questions, it an appropriate combination of system Software Engineering at Oxford Uni- is worth summarizing the Correctness specification and domain knowledge. versity in England, as follows: “The by Construction software development The emphasis on domain knowledge is only way to reduce costs and to keep approach, and some examples of its key; half of all requirements errors are projects within plans is to dramatically results. The underlying principles are related to domain [5], yet the vast ma- reduce the error rate at every stage in straightforward: firstly to make it diffi- jority of requirements processes do not the development. If you do that, the cult to introduce errors, and secondly to explicitly address issues in the domain. product is not only cheaper, but higher remove any errors as soon as possible At the specification and design quality: more secure, more reliable, and after introduction. The key to achiev- stages, mathematical (or formal) meth- easier to maintain.” ing this is to introduce sufficient preci- ods and notations are used to define Thomas’s emphasis on reducing sion at each step of the development precisely the behavior of the software, errors has been backed up by recent of the software to enable reasoning and to model its characteristics (for work on behalf of the National Cyber about the correctness of that step. The example demonstrating that a multi- Security Partnership, formed in 2003 in correctness of the software can then be process design cannot deadlock). Such response to the White House National demonstrated in terms of the manner techniques can allow precise verifica- Strategy to Secure Cyberspace [3]. The in which it has been produced (the “by tion of consistency and accuracy. Partnership’s Secure Software Task construction”) rather than just by ob- At the detailed design and imple- Force reported that a primary cause serving operational behavior. An anal- mentation stages, information and of security problems is software with ogy may be drawn with aeronautical data flows are explicitly modeled and vulnerabilities caused by defects, or engineering, where the demonstration statically analyzed (for example, to errors in software [4], and that practices of correctness during the specification, demonstrate the separation of secure which lead to low defect software are design and implementation phase is state). Where applicable, the code is therefore to be encouraged. such that it is rare for a new aircraft to written in a mathematically verifiable One such practice cited in the re- work incorrectly the first time opera- continues on page 9 8 STN 8-2: Software Cost, Quality and Productivity Benchmarks The Software Challenge Continued from page 8.

and statically rigorous independent reliability testing such as this, or papers such as those analyzed (for example, to demonstrate which identified zero defects, and was cited. Or there may be a view that such the absence of run-time errors, such as developed at a productivity of well best practice “could never work here” buffer overflows, which are the bane of over 30 loc/day. for a combination of reasons. These secure systems). More generally, Correctness by reasons are likely to include perceived Correctness by Construction Construction delivers software with capability of the in-house staff, belief is cost-effective because errors are defect rates of 0.1 defects/kloc and about applicability to the organization’s eliminated early or not introduced in lower; this compares very favorably product or product development ap- the first place, dramatically reducing with defect rates reported by Capabil- proach, prevalence of legacy software the amount of rework needed later in ity Maturity Model (CMM) Level 5 which is viewed as inherently difficult the development. The precision means organizations of 1 defect/kloc [9] (see and therefore expensive to maintain, or that the requirements are more likely to chart Figure 1). concern about the disruption and cost be correct, and the system more likely � to be the correct system to meet the Correctness by Construction defect rates compared to requirements, and to work correctly in Capability Maturity Model data operation. Software developed using Correctness by Construction has also proved to be very cost-effective to 8 maintain. 7

c 6 o

The results speak for themselves. l k

/ 5

Correctness by Construction was used s t 4 c

by Praxis to develop the Certifica- e f 3 tion Authority system to support the e D MULTOS multi-application smart 2 card developed by 1 0

Mondex International (now part of 1 2 3 4 5

n s l l l l l o s e e e e e i t e

Mastercard) [6]. Developed to the v v v v v c n e e e e e t u y L L L L L r standards of the IT Security Evaluation c

t b e s r M M M M M r Criteria (ITSEC) [7] Level E6 (roughly n o M M M M M o C equivalent to Common Criteria [7] C C C C C C Evaluation Assurance Level (EAL) 7), CMM data from Jones, Capers [9] the system had to meet both stringent security requirements and demanding Figure 1 availability requirements. The system was delivered with a warranty against So, given the apparent success of of introducing new approaches. defects, and had an operational defect best practice approaches such as Cor- rate of 0.04 defects/kloc (thousand rectness by Construction, why is there Secondly, where the need for lines of code), yet was developed at a still so much poor quality software improvement is acknowledged and productivity of almost 30 loc/day (three around, and why is such best practice considered achievable there are usually times typical industry figures). not in more widespread use? practical barriers to overcome, such as how to acquire the necessary capability In the US, Praxis used Correct- There seem to be two kinds of bar- or expertise, and how to introduce the ness by Construction to develop a riers to the adoption of best practice. changes necessary to make the im- demonstrator biometrics system for Firstly, there is often a cultural mindset provements. Introducing such change the National Security Agency (NSA), or awareness barrier. Many individuals may be challenging for a combina- aimed at showing that it is possible and organizations do not, or do not tion of technical, political and social to produce high-quality, low defect want to, recognize or believe that it is reasons. software conforming to the Common possible to develop software that is low Criteria [6] requirements for Evaluation defect, secure and cost-effective. This These are reasonable, common, Assurance Level (EAL) 5 and above may simply be an awareness issue, in but not insurmountable barriers, and [8]. The software was subjected to principle readily addressed by articles continues on page 10

Data & Analysis Center for Software (DACS) 9 The Software Challenge Continued from page 9.

overcoming them requires effort from plex, the prevalence of requirements 20 Manvers Street, Bath BA1 1PX, UK suppliers, procurers and regulators and for EAL 5 and above will increase, and http://www.praxis-his.com/ involvement at the individual, project the software industry will need to adapt +44 1225 466991 (switchboard) and organizational level. Typically, its development approaches in order to +44 7881 516750 (cell) strong motivation and leadership will meet these requirements. The situation [email protected] be required at a senior management is analogous to the safety-critical sec- level, where the costs to the business tor, particularly in Europe, where the References of poor quality (high defects, low key safety regulatory requirements now 1 US NIST Report 7007.011, May productivity) are most likely to be require such approaches. This is the 2002 experienced. “stick” incentive, and there is a view At a supplier level, a typical way that if the industry persists in producing 2 The Chaos Report http://www. forward is for organizations (and in- insecure software then regulation will standishgroup.com dividuals within them) to take a fresh, increase. 3 http://www.cyberpartnership.org/ open-minded look at what is possible But there is also the “carrot” incen- about-overview.html by comparing current approaches to tive. There is plenty of evidence from 4 Processes to Produce Secure best practice and, where appropri- a range of sectors that shows that best Software www.cyberpartnership. ate, adopting step-wise, prioritized practice software engineering produces org/SDLCFULL.pdf improvements based on assessments of high quality software cost-effectively. the Return On Investment. - When organizations recognize that 5 Hooks and Farry, Customer Cen- ing decisions on process, methods and low defect software really can have tred Products, Amacom, 2000 tools need to be premised on the basis through-life cost benefit (even tak- 6 Correctness by Construction: of logic and precision (for example by ing into account the costs of the time Developing a Commercial Secure asking “how does this choice help me and effort to acquire the capability to System, Anthony Hall and Roder- reason about my software?”), rather deliver it) then the business driver will ick Chapman. Published in IEEE than on silver bullets or fashion (char- take over – the $60 billion reported by Software January/February 2002 acterized by questions such as “how NIST [1] is a big prize! pp 18-25. http://www.praxis-his. many developers already know this Perhaps the real challenge for the com/pdfs/c_by_c_secure_system. particular technology?”). software industry is to recognize and pdf Procurers and regulators can help eat the “carrot” before being beaten by 7 Information about ITSEC and the by adopting an attitude of not settling the “stick”. Common Criteria may be refer- for less, by demanding a warranty, by enced from http://www.cesg.gov. awarding contracts to organizations About the author uk/site/iacs/index.cfm with the capability to deliver low-de- 8 Fourth Annual High Confidence fect software, and by using contracting Martin is Associate Director for Software and Systems Conference arrangements such as gain-share that security with Praxis High Integrity Proceedings, National Security encourage partnership and improve- Systems Limited, a UK-based systems Agency, April 13-15 2004 ment. engineering company specializing in mission-critical systems. Martin Fundamentally, however, the main 9 Jones, Capers: Software Assess- Croxford is a chartered engineer with drivers for change will come from two ments, Benchmarks, and Best 15 years experience in the software in- directions. Practices. Reading, MA: Addison- dustry. Martin has worked on software Wesley, 2000 Regulation, such as in the form development projects in a range of of the Common Criteria [7], at least organizations, and as a software devel- at EAL 5 and above, already requires opment manager has used Correctness the adoption of techniques that provide by Construction to successfully deliver demonstration of correctness through a multi-million pound security-critical the way software is developed. As system. reliance on correctly functioning soft- ware-intensive security applications increases, and where such software is Contact Details becoming ever larger and more com- Praxis High Integrity Systems Limited

10 STN 8-2: Software Cost, Quality and Productivity Benchmarks Enhancing Customer Security: Built-in versus Bolt-on

Glenn Schoonover CISSP MCSE, Security Solutions Specialist

Introduction new territory and people tended to trust operating system because they did not Can you really bolt on security after each other when conducting business on understand the impact that turning off a the fact? I don’t think so, at least not the nascent Internet. The result was an service would have and many times the effectively. That is a question that soft- attack path that individuals with mali- only way to recover was to reinstall the ware developers and security specialists cious intent could use to run executable OS from scratch. have been discussing for quite some code on an unsuspecting user’s machine. With the implementation of Trust- time and with the increasing number of Writing secure code is not a challenge worthy Computing, security has become vulnerabilities and the reduction in num- that is unique to Microsoft. All software the number one priority. Default ber of days between vulnerability and vendors are faced with the challenge of installations aimed at ease of use are patch the best answer is to get it right the building secure products, but as part of now not always sufficiently secure, but, first time. At Microsoft there have been their Trustworthy Computing and Secu- going forward, security in Microsoft’s a number of significant changes in the rity Mobilization Initiatives Microsoft is products will take precedence over ease- past 3 years to address the problem of doing something about it. The goal of of-use. our Security Mobilization is to address building software that is secure “out of For instance, in Windows Server five key issues: 1) Build Security into the box” and resistant to attack even if 2003, IIS6 is turned off by default. It our products, 2) Address Customer Pain, unpatched. will need to be specifically chosen for 3) Demonstrate Leadership, 4) Mobilize installation, and when installed will the company, and 5) Provide Security only serve up static HTML pages by What is the Problem? and Assurance for computer services default. All other functionality (ISAPI One of the problems with building and products that are built on Microsoft filters, Active Server pages capabilities) secure software is spelling out the re- Products. must be turned on by the administra- quirements for the developers as early as tor after installation. Also, the Outlook possible. “ can be taught Security Philosophy: Past and Present Security Patch functionality, introduced to avoid creating buffer overflows and as a download for Outlook 2000, is now other well-known vulnerabilities found Until recently Microsoft’s philoso- built-in to Outlook XP and 2003. This in commercial software,” said Lawrence phy has been to build products that were security patch blocks access to poten- Hale, speaking at this year’s FOSE easy to use and that worked seamlessly tially dangerous attachments, and warns conference on government technology. across the platform. This meant that when programs try to send mail on the The problem is that, historically, most many services were enabled by default users’ behalf. developers did not spend much time when the operating system was installed. worrying about buffer overruns nor did For example, in Windows 2000 Server, Microsoft has committed unprec- they do threat modeling against applica- the Internet Information Services are edented resources to achieving the tions except in very tightly controlled installed by default, set to start auto- highest level of security possible in all government environments. If they did matically, with the Internet Printing of our products. The goal is to become consider the potential for a threat they ISAPI filter enabled. Security was often the leader in the industry both in terms were often not trained in writing secure thought of in terms of “features” such of product security, and in response to code. An example I like to use is the as IPSEC and EFS (Encrypting File security issues that arise. URL injection exploit where a hacker System). While this gave system admin- istrators and home users alike the ability can force a buffer overrun by inserting Trustworthy Computing an exceptionally long character string. to run a wide range of applications with In 2002, Bill Gates announced the While I was not present those many minimum intervention, it did nothing to Trustworthy Computing Initiative. This years ago when Mosaic, Netscape, and, enhance security. In the Department of was the first step in a 180 degree turn later, Internet Explorer were first being Defense it took many hours of testing in building secure products. Success coded, I’m pretty sure that none of the and evaluation to develop configuration with Trustworthy Computing (TWC) developers ever stopped to consider templates that would allow organizations is not going to be an easy task. It will what would happen if someone did in- to meet our Certification and Accredita- take several years - perhaps a decade sert an extra long string into their brows- tion requirements. System administra- tors would lock themselves out of the or more, before technology is trusted. er forcing a buffer overrun. This was continues on page 12 Data & Analysis Center for Software (DACS) 11 Enhancing Customer Security Continued from page 11.

The initiative is predicated on four key are necessary to manage the security turned off by default. We implemented pillars: business risk. a stronger security policy, • Security: Operating systems and The security of our customers’ list defaults and new “low privilege” ser- applications must be resilient to and networks is a top priority vice accounts. Windows Server 2003 is attack; confidentiality, integrity and for Microsoft. Security is an industry- also the first product to ship “Secure in availability of data and systems are wide issue and although there is no one Deployment.” We improved the power protected, enabling customers to solution, our approach to security spans and simplicity of Security Management safeguard critical information. across both technology and social as- Tools & Services, including software pects. In technology, we’re focused on: restriction policies. Secure communica- • Privacy: Products and online tions (VPN/Wireless) is now easier to • Building greater isolation and services adhere to fair information deploy with IEEE 802.1X protocol sup- resiliency into the computing plat- principles, while protecting the port, and integrated certificate services form. individual’s right to be left alone. with auto-enrollment. There is greater • Reliability: Ensuring systems and • Providing customers with the lat- breadth of Patch Management Solutions services work the way customers est and most effective advanced within and outside of the product, in- expect; dependable, performing updating methods. cluding Software Update Services (SUS) and available when needed. • Enabling new business scenarios 2.0, and we offer much more extensive • Business Integrity: Open, transpar- through integrated authentica- Prescriptive Guidance so system admin- ent and responsive with customers, tion, authorization and access istrators can easily get information on with an internal focus on excel- control options. how to deploy the product securely. lence in our decision-making and • Improving quality by enabling These security engineering practices processes. engineering excellence. have been recognized by organizations Goal: To be everyone’s trusted such as RSA and the SANS Institute who have given Microsoft awards on supplier of secure, private, and reliable Progress computing. our training, tools, and product update The first product to ship as part of investments. In short, with the degree of Security is a core tenant and Micro- the Trustworthy Computing Initiative soft is committed to building software customer engagement, early production was Windows Server 2003. We focused deployments across all customer seg- and services to help better protect our on making the product secure by design, customers and the industry. ments and workloads and the measures by default, and in deployment. This of quality, especially security, Windows represents huge progress on security, Server 2003 is a product that can be Commitments and the processes we use have begun deployed today, without waiting for a At the Worldwide Partner Confer- to win recognition in the industry and service pack. ence in October 2003, Steve Ballmer even awards. In the area of “Secure by announced Microsoft’s commitment to Design,” we made a $200M Investment RSA Industry Innovation Award “Build software and services that will in security engineering covering tools, help better protect our customers and the training, and the process of software As members of Microsoft’s elite Se- industry.” development. We instituted better cure Windows Initiative team, Michael developer accountability - each line of Howard and David LeBlanc published In developing and refining our code is owned by a particular developer “Writing Secure Code” (now in its’ approach to security over the past few who is responsible for ensuring security second edition) to provide software years, the largest set of stakeholders compliance. We developed perva- developers with a better understanding that have influenced us has been our sive threat-modeling techniques and of the processes and practices needed to customers. Security sometimes seems automated code analysis to analyze the produce sound software code. Howard too simple a term for the many aspects design. Another key development is and LeBlanc’s book is the cornerstone of of business and information technol- shipping Windows Server 2003 in a the security training programs developed ogy that it touches. Even just looking “” mode. The product during the implementation of the Trust- at security from an IT viewpoint, we uses locked-down configurations so that worthy Computing initiative. During want to protect networks, systems, data, only the features you need are enabled, the Windows security push, product processes and users. For each of those reducing the attack surface to less than development halted for more than two areas, people, processes and technology half of what it was in NT 4.0. IIS 6.0 is continues on page 13

12 STN 8-2: Software Cost, Quality and Productivity Benchmarks Enhancing Customer Security Continued from page 12.

months as Microsoft software develop- patching of all systems managed finding bugs, it should be to educate, ers attended, and then implemented, by the Update Server, both locally and attempt new attack techniques and mandatory security training, all based on and remote. methods on an ongoing basis. If you the “Writing Secure Code” book. • The Award for Leadership in give people the time to do this they will Thousands of product developers Security Training of Software find new issues. and testers from across the company Developers for Microsoft’s nascent have now been trained in writing program of requiring all Microsoft Security Push secure code as part of the Trustwor- software developers to become fa- thy Computing Initiative. Since being miliar with common security errors A security push occurs at beta time introduced internally at Microsoft, made by programmers and how to and is a team-wide stand down to focus “Writing Secure Code” has become the avoid them. on threat model updates, code review, testing and documentation scrub. Note definitive security resource for software • The Award for Leadership in Test- that the push is not a quick fix for a developers and at Microsoft. ing Software for Security Vulner- process that lacks security discipline; In addition, “Writing Secure Code” is abilities for Microsoft’s extensive it is simply a concerted effort to being adapted into textbook format for automation of the software testing eradicate bugs before ship. Note, in the university courses process. by Addison Wesley. The success of short-term, a security push is a length What are these changes? The Secu- Howard and LeBlanc’s book and cur- milestone. rity Development Lifecycle is the pro- riculum underscores the industry’s need cess that is used internally at Microsoft for guidelines and the to build more secure software. This is a Security Audit importance of educating developers sophisticated process, with threat model- Once the end of the project draws about the value of secure software in ing, audit, testing and signoff stages, near, a very important question must today’s computing landscape. coupled with developer education and be asked, “from a security viewpoint, tools. At Microsoft, we have trained are we ready to ship.” The only way to SANS 2003 over 13,000 engineers in the rigorous answer this question is to have an end Leadership Awards process. (See Figure 1) of project security audit. The process Microsoft earned recognition in is well understood – the three main analysis points are: (1) Have the threats three categories of SANS 2003 Informa- Security review tion Security Leadership Awards, includ- changed? (2) Perform a root cause Once the product design is under- ing automated patching and training analysis of incoming security vulner- stood, the specs complete and the threat programmers to write safer code. Red abilities that require code modifications models are done, it’s time to have the Hat also was recognized for automated in the current code base. Why were design reviewed. The product group patch notification. they missed? What needs changing? (3) should set aside a day or more for such Penetration work; The Secure Windows http://www.computerworld. reviews. At this meeting (taking a day Initiative (SWI ) (and outside contrac- com/securitytopics/security/sto- at a minimum), component owners will tors and the product team) review ry/0,10801,79164,00.html. present their and the securi- default settings, attempt to compromise Microsoft won three of the awards ty implications, threats and countermea- the system. - demonstrating that its Trustworthy sures pertaining to their component. The Computing Initiative is beginning to team will provide experienced feedback bear fruit: and, if need be, the product group makes Security Response • The Award for Leadership In Au- adjustments to the product. You can only design, write and test tomated Updates for Microsoft’s for the security issues you know today; no matter how rigorous the process automated patching service (for Develop and Test Windows XP and Windows 2000 security, issues will arise simply because The purpose of the Security Days SP3 and above) that helps protect the threat landscape changes each week. is simply to keep everyone on their users who are not security experts Because of this, each team needs a toes, and to provide updated educa- and for the Update Server that group of people to handle security vul- tion and security analysis. In the past, allows security experts inside nerabilities as they are discovered after many groups held such “bugbashes,” organizations to test patches and the product has shipped. The team must but the focus should not be simply on continues on page 14 then release them for automated Data & Analysis Center for Software (DACS) 13 Enhancing Customer Security Continued from page 13. focus on addressing the vulnerabilities software we can, while still building for developing and executing on the found, and also on performing a root products that customers will want and Infrastructure and Security strategy cause analysis on each vulnerability so be able to use. Beyond that, we are tak- for the Federal District and spends as to find and modify potential vulnera- ing steps to help protect our customers most of his time helping government bilities proactively – before they are also in a world where vulnerabilities are customers build a secure, connected found in the field. The team must meet inevitable and the threats are evolving. infrastructure. Prior to arriving at common standards for response time, This means investing in new technolo- Microsoft he was the Director of quality, patch packaging and release. gies; investing in training, guidance and Security for a global Internet Service communications to help our customers Provider and Chief of Network Se- get the expertise they need; and partner- curity for the Army at the Pentagon. Challenges Ahead ing with industry leaders, customers, A 1986 graduate of the United States At Microsoft, we are thinking governments, and law enforcement to Military Academy at West Point, he about the “big picture” of security, and address the challenge. is an authority on working to help customers in a variety architecture, vulnerability assess- of ways. First and foremost, we remain ments and intrusion detection with Biography deeply committed to building software experience using a wide variety of and services that will help better Glenn Schoonover CISSP MCSE, commercial and open source tools. protect our customers and the industry. is currently a Security Specialist Our goal is to build the most secure with Microsoft. He is responsible

Figure 1. Security Development Life Cycle 14 STN 8-2: Software Cost, Quality and Productivity Benchmarks Software Development Security: A Risk Management Perspective

By Ellen Walker, DACS Analyst

A synopsis of the Government Ac- toward globalization. be available for use in source selection. countability Office (GAO), (formally Concurrent with this software Figure 1 depicts how supplier the Government Accounting Office) development paradigm shift, we are expansion is occurring and how it may report to congressional requesters titled seeing increasing attempts by foreign encompass foreign involvement in the “Defense Acquisition: Knowledge of entities to access U.S. technology and development process. Software Suppliers Needed to Manage information, and countries and orga- The spider-like image also Risks”, (GAO-04-678), published in nizations hostile to the United States conveys the complexity involved in May 2004. focusing on . identifying and tracking all suppliers The Department of Defense (DOD) Do we know who is actually devel- and, specifically, sources of foreign is concerned about the expansion of oping the software used in our weapons involvement. The shaded oval identi- opportunities for exploiting vulner- systems programs? Is there a signifi- fies the current scope of control from abilities in defense weapon systems cant risk resulting from the expansion the perspective of the Program Office. software that may result from increased of suppliers and the unknowns relating A solid, managed relationship exists reliance on prime contractors who, in to the origins and security of the actual between the prime contractor and the turn, are outsourcing the development, developers and the respective develop- Program Office, but the remaining ac- implementing reuse, using COTS, ment environment? DOD thinks there tivity and information, which is primar- and acquiring software. Additionally, is a risk that it needs to be identified ily contractor driven, is essentially not contractors are growing through acqui- and managed at the program level and visible to the Program Office. sitions, mergers, and a general trend that knowledge of all suppliers needs to continues on page 16

Figure 1. Scope of Supplier Expansion and Foreign Involvement

Data & Analysis Center for Software (DACS) 15 Software Development Security Continued from page 15.

To address DOD concerns, Con- did not view any risk associated with in software development. gress asked the GAO to examine and foreign involvement as significant. GAO found that DOD policy and report on DOD’s efforts to identify They relied on the competence guidance is not currently addressing and manage the risks associated of their prime contractors to ensure the issues of software development with foreign involvement in and security and make good security and adopting a risk strategy for development in individual weapon decisions about subcontractors, but foreign involvement. Security policies systems programs. few of the programs included specific for weapon systems software focus For this study, conducted from software security requirements in their primarily on operational threats, not April 2003 to May 2004, GAO selected contractual agreements with the prime insider threats such as the insertion of 16 weapon systems varying in age and contractor. In the absence of specific malicious code by software developers. capability; reviewed relevant DOD requirements to address security risks Additionally, security procedures that guidance, policies, regulations and associated with foreign involvement, are in place tend to be applied after the procedures; met with experts from the contractors are not dealing with it, software suppliers have been selected the SEI and the weapons engineering electing instead to focus on meeting the and, thus, do not provide the manager community; and reviewed or solicited specified contractual requirements. the opportunity to evaluate whether the information from the Program Offices Those programs that did identify risks associated with using a supplier and their respective prime contractors. software security as a risk focused are acceptable. Appendix I of the GAO report provides on operational threats (e.g., limiting Some officials noted that acceptance further details of the scope and method- foreign access to software development testing for reused and COTS software ology of this study. facilities and denying foreign access to limits its focus to proving functionality software code), not insider threats that and, thus, closes the door to supplier Summary of GAO Findings might come from foreign involvement information for those products. GAO found that software security issues in general, and the risk associ- ated with foreign involvement in particular, are taking a back seat to the main topics of focus on weapon systems programs – performance, cost and schedule. Reasons for this tend to fit into the following categories: • Lack of policy to address the risk of foreign involvement • Lack of communication among organizations who possess knowl- edge of foreign suppliers • Lack of prioritization of software security relative to issues of cost, schedule, and performance • Lack of clear accountability for ad- dressing software security related risks Figure 2 presents some of the quantifiable key findings with respect to the actions and viewpoints of the program officials. In general, program officials lacked awareness of foreign involvement, in either COTS, or their custom software. Consequently, they Figure 2. GAO Key Findings continues on page 17

16 STN 8-2: Software Cost, Quality and Productivity Benchmarks Software Development Security Continued from page 16.

GAO reported several situations in manager’s assessment of risks. changes in supplier status and to ad- which knowledge of foreign involve- Therefore, unless the program just program security requirements ment exists at some level, but is not manager has identified foreign 3. Require the Office of the Assistant routinely shared with the Program Of- software development as a risk, the Secretary of Defense for Networks fice, either because there is no require- process will not address it. and Information Integration ment to do so, or because the knowl- • Better software development (OASD-NII) and the Office of the edge is acquired by other agencies practices alone (such as those Undersecretary of Defense for relative to other functions, such as the represented by the SEI CMM levels Acquisition Technology and Lo- export licensing process. Contractors of maturity) may reduce defects and gistics (OUSD-ATL) to work with request approval from the State Depart- improve overall software quality, other organizations to ensure that ment, but the State Department does but cannot be expected to address weapons program risk assessments not automatically refer the application malicious software development ac- include attention to software devel- to DOD or the Program Office. tivities intended to breach security. opment risks and threats. Some additional insights from the • Program managers are encouraged DOD’s concerns, in response to study are as follows: (under the blanket of using sound this report, are that the recommenda- • Although we know that practices practices) to tions place too much responsibility on like peer reviews and dedicated develop open software systems the program managers, and that insider software testing can uncover mali- , use COTS products, threats are not limited to foreign suppli- cious code and minimize defects, and make incremental improve- ers. DOD believes that program man- 50% of the programs made deci- ments through code reuse. How- agers should be able to rely on external sions about what code to test based ever, all of these practices have resources to gain threat information on the risks and benefits to the potential for introducing malicious on suppliers, and that formulation and functionality of the system, not on code from unknown software de- oversight of security practices should security. Experts agree that com- velopment sources. be a collaborative function among prehensive testing (every line of several offices. Furthermore, a central- code) to ensure complete security is ized information repository on software GAO Recommendations perhaps physically impossible and suppliers (including but not limited to would require immense resources. The GAO concluded the DOD foreign suppliers) is necessary because needs to take steps to ensure that the cost of collecting and maintaining • 75% of the programs reported software security is an integral element this information would require re- using the Technology Assessment/ in decision-making and that program sources and assets beyond the scope of Control Plan and/or the Program managers mitigate risks accordingly. individual program managers. Perhaps Protection Plan (documents that They recommended that the Secretary identifying, tracking, and maintaining address the release of information of Defense take the following three intelligence on security risks of soft- to foreign governments through actions to address risks attributable to ware suppliers is best done at the DOD cooperative programs and military software vulnerabilities and threats: level so that it can be shared among sales), but these documents do not the programs. Threat analysis, which provide specific information on 1. Require program managers (work- drives the development of security suppliers who will be performing ing with others as necessary) to requirements, should be carried out at the work. define software security require- ments, including identifying and the subsystem, system, and system-of- • 69% of the programs reported us- managing software suppliers, and systems levels and not be limited to the ing the Defense Information Tech- then communicate the require- scope, expertise, and resources of the nology Security Certification and ments through the prime develop- individual program managers. [Source: Accreditation Process (DITSCAP) ment contract to ensure that they GAO Report Appendix II, DOD Com- to address general software secu- are used in selecting suppliers ments to the Recommendations] rity; however, this process does not Reading the actual report begs govern contractors in cases where 2. Require program managers to questions such as the following: the requirements were not included collect and maintain information in the original contract. In addi- on software suppliers (including • Is the issue of malicious code tion, the DITSCAP process bases software from foreign suppliers) potentially being inserted into a its requirements on the program and use the information to assess continues on page 18

Data & Analysis Center for Software (DACS) 17 Software User Comment Development By Bob Ferguson, Security Carnegie Mellon University Continued from page 17. I was reading part of the July 2004 software component really a threat Software Tech News. I noted your criti- here and now? cism of “software testing as an art.” I do understand why you would say that • How far could someone get with this before it is discovered? -- that certain people claim they are art- ists as a way of saying they do not care • Are foreigners the only people who to have a disciplined process. could do this? I claim that nearly every artist • What would it take to test for malicious code insertions (insider does have a disciplined process. I have threats)? observed my wife doing watercolor painting. Whenever she changes paper, • Whose job is it to verify that all the software used in a weapon system brush or paint, she does some number does not represent a security risk? of test drawings. She has to have a detailed understanding of how the • What level of risk is acceptable (if any)? And at what cost? materials work together and master the techniques she will use. It is important • What do we need to know about to understand the mistakes that are software suppliers, or the software inherent in the process. Then it is pos- development environment, in order to be able to thwart such threats? sible to execute so that the mistakes do not compromise the end product. • How could we be sure that the information we collect on suppliers If she happens to be in a rush and is, in fact, valid? skips the test drawing, she inevitably These and many more questions, has to throw away the first version -- collectively, hint at the complexity of usually before it is finished. the issue and its proposed solutions. Are Michelangelo and Leonardo Da GAO’s recommendations simplistic Vinci were famous for making many given the realities of the issue? If imple- detailed sketches in preparation. They menting software security measures re- mixed and tested materials they would quires continuous tracking of all suppli- use in their frescos. I also know a ers of all software products and results couple of sculptors who like to work in enormous costs, who decides what with new and different materials. They the priority of security should be relative spend a great deal of time learning how to overall cost, schedule and software capability requirements? Perhaps these to work with the new material before questions have wetted your appetite for attempting a product. reading the report in its entirety. It is Let’s not demean artists by available for download at http://www. claiming we can be creative without gao.gov/new.items/d04678.pdf a process. We should realize that true artists do follow a disciplined process Author Contact Information and calibrate their work. Good testers Ellen Walker would do likewise. Data and Analysis Center for Software (DACS) Bob Ferguson Ph: 315-334-4936 Sr. Member of the Technical Staff E-mail: [email protected] Software Engineering Institute Carnegie Mellon University

18 STN 8-2: Software Cost, Quality and Productivity Benchmarks Lessons Learned By Thomas McGibbon, DACS Director

The four articles in this issue of - Need to sensitize designers, • From a supplier perspective: Software Tech News have provided developers and testers to security - Suppliers need to ship software excellent guidance and a wealth of issues through training.1, 3 where default settings are information about “Secure Software - Tools are available to detect secure.3 Engineering” from a development, some security vulnerabilities.1, 3 - Perform a security audit prior to management, supplier, and acquisition - Best practices for developers and release.3 perspective. Being a software testers includes threat modeling, engineer myself and given the Fuzz testing, Ballista, penetration • From an acquirer’s perspective: increased importance now of security analysis static code analysis.1, 3 - Acquirers, suppliers, and and trustworthiness of software - Developer accountability helps program managers need to intensive systems, my perspective in to ensure security compliance.3 identify and manage risks reading these articles is to understand - Correctness by Construction associated with foreign how what I perceive as “best” achieves significant defect involvement in development of practices in software engineering are reduction through rigorous software (including COTS) for impacted by security issues. Here are requirements analysis, use of weapon system programs.4 the highlights of what I learned: formal design methods and - Acquirers need to communicate information flow models for security requirements through • From a development and lifecycle design, and verifiable code prime development contracts.3, 4 perspective: development (when needed).2 - Acquirers should demand - Need to significantly reduce software warranties, award defects induced and improve • From a management perspective: contracts to organizations that methods for detecting software - >90% of all vulnerabilities are deliver low defect software, and defects throughout the lifecycle. caused by defects resulting from provide contract incentives for Correctness by Construction inadequate, normal software partnership and improvement.2 is a rigorous methodology engineering methods.1 - Change, in reality, will come which results in very low defect - In building a business case for from regulations and financial software (<0.1 defects/ksloc). secure software engineering, incentives.2 TSP-Secure provides a set of need to consider (add) costs defined and measured best of fixing and releasing patches 1 See “Developing Secure Software” practices for low-defect Secure from a supplier, acquirer, and by Noopur Davis software development.1, 2 consumer perspective. Not 2 See “The Challenge of Low - Need to understand common addressing security from a Defect, Secure Software” by causes of vulnerabilities and supplier perspective could Martin Croxford focus on defect reduction impact customer satisfaction and 3 See “Enhancing Customer techniques for defects that lead result in lawsuits.1, 2 Security: Built-in versus Bolt-on” to or cause vulnerabilities.1, 3 - Need senior management by Glenn Schoonover - Security must be built-in to the vision, leadership, support, and 4 See “Software Development software development lifecycle oversight of implementation Security: A Risk Management with appropriate checkpoints of security policies and best Perspective” by Ellen Walker and reviews.1, 2, 3 practices.1, 2, 3 - Need to define a set of best - Security measures need to be practices that development planned, measured, and tracked.1 teams can use. Correctness - Program managers should by Construction and TSP- collect and maintain information Secure provides best practice on suppliers used to perform approaches.1, 2 software development.4

Data & Analysis Center for Software (DACS) 19 20 STN 8-2: Software Cost, Quality and Productivity Benchmarks The first 50 people to send in a completed survey will receive a FREE DoD/IT Acronym CD from the DACS. This valuable CD-ROM contains over 9,000 Department of Defense and Information Technology acronyms. There are hun- dreds of acronym lists available but none are as well done as this CD AND specifically targeted towards DoD and Information Tech- nology. This unique-shaped CD-ROM plays in your computer’s regular, hub-mounted, CD drive. You’ll use this great resource over and over again. It’s FREE, just for filling out our brief survey on the next page!

� Fold Here �

Data & Analysis Center for Software (DACS)

http://iac.dtic.mil/dacs/

� Fold Here �

Data & Analysis Center for Software (DACS) 21 Software Tech News Subscriber Survey

1. Which volume of the Software Tech News did you receive? ______STN 8:2 Secure Software

2. When did you receive the newsletter? (month/year) ______

3. How satisfied were you with the CONTENT of the newsletter? (Article Quality) � Very Satisfied � Satisfied � Neither Satisfied nor Dissatisfied � Dissatisfied � Very Dissatisfied

4. How satisfied were you with the APPEARANCE of the newsletter? � Very Satisfied � Satisfied � Neither Satisfied nor Dissatisfied � Dissatisfied � Very Dissatisfied

5. How satisfied were you with the OVERALL QUALITY of the newsletter? � Very Satisfied � Satisfied � Neither Satisfied nor Dissatisfied � Dissatisfied � Very Dissatisfied

6. How satisfied were you with the ACCURACY of the address on the newsletter? � Very Satisfied � Satisfied � Neither Satisfied nor Dissatisfied � Dissatisfied � Very Dissatisfied

7. Approximately how much of the newsletter do you read? � The entire issue � Most of the content � About half the content � Briefly Skimmed � Didn’t Read

8. Would you read this newsletter in an E-mail newsletter format? � Definitely � Probably � Not Sure � Probably Not � Definitely Not

9. How did you request the product or service? � Phone Call � E-mail � DACS Website � Subscription Form Other ______

10. Would you recommend the DoD Software Tech News to a colleague? � Definitely � Probably � Not Sure � Probably Not � Definitely Not

11. What topics would you like to see this newsletter devoted to?

Comments (Optional)

Contact Information (Optional*)

Name: Position/Title:

Organization: Office Symbol:

Address:

City: State: Zip Code:

Country: E-mail:

Telephone: Fax:

Functional Role:

Organization Type: � Air Force � Army � Navy � Other DoD ______� Commercial � Non-Profit � Non-US � US Government � FFR&D � Other ______

*Note: You must give us your address to receive the CD.

22 STN 8-2: Software Cost, Quality and Productivity Benchmarks About the DoD Software Tech News

Article Reproduction STN Editorial Board About This Publication: Images and information presented The DoD Software Tech News is in these articles may be repro- Philip King published quarterly by the Data duced as long as the following Editor & Analysis Center for Software message is noted: ITT Industries, DACS (DACS). The DACS is a DoD “This article was originally printed sponsored Information Analysis Paul Engelhart in the DoD Software Tech News, Center (IAC), administratively DACS COTR Vol. 8, No. 2. Requests for copies managed by the Defense Technical Air Force Research Lab (IFEA) of the referenced newsletter may be Information Center (DTIC). The submitted to the following address: DACS is technically managed by Morton A. Hirschberg Philip King, Editor Air Force Research Laboratory, Editorial Board Chairman Data & Analysis Center for Soft- Rome, NY and operated by ITT Army Research Lab (retired) ware Industries, Advanced Engineering P.O. Box 1400 and Sciences Division. Ellen Walker Rome, NY 13442-1400 ITT Industries, DACS Phone: 800-214-7921 Fax: 315-334-4964 Thomas McGibbon E-mail: [email protected] DACS Director An archive of past newsletters is ITT Industries, DACS available at www.SoftwareTech- News.com. David Nicholls In addition to this print message, DACS Deputy Director we ask that you send us three cop- ITT Industries, DACS ies of any document that refer- ences any article appearing in the DoD Software Tech News.

Cover Design by Joseph Barbaccia, ITT Industries

To Subscribe to this Publication Contact: Phone: 800-214-7921 Fax: 315-334-4964 DACS E-mail: [email protected] P.O. Box 1400 Web: www.dacs.dtic.mil Rome, NY 13442-1400 Phone: 800-214-7921 Fax: 315-334-4964 E-mail: [email protected] Distribution Statement: URL: http://iac.dtic.mil/dacs/ Unclassified and Unlimited

Data & Analysis Center for Software (DACS) 23 Advertisement STN Vol. 8, No. 2 The DoD Software Tech News is now ac- In This Issue cepting advertisements for future newsletters. In addition to being seen by the thousands of Developing Secure Software ...... 3 people who subscribe to the DoD Software Tech News in paper copy, the Tech News will The Challenge of Low Defect, Secure also be placed on the Data & Analysis Center Software ...... 8 for Software’s website (http://iac.dtic.mil/dacs/), exposing your prod- Enhancing Customer Security ...... 11 uct, organization, or service to hundreds of Software Development Security ...... 15 thousands of additional eyes. Interested in learning more? For rates, lay- out information, and requirements contact: Philip King, STN Editor Data & Analysis Center for Software P.O. Box 1400 Rome, NY 13442-1400 Phone: (800) 214-7921 Fax: (315) 334-4964 E-mail: [email protected]

Data & Analysis Center for Software PRSRT STD P.O. Box 1400 U.S. Postage P A I D Rome, NY 13442-1400 Permit #566 UTICA, NY Return Service Requested

24 STN 8-2: Software Cost, Quality and Productivity Benchmarks