The Costs and Benefits of Pair Programming Alistair Cockburn Laurie Williams Humans and Technology University of Utah Computer Science 7691 Dell Rd 50 S. Central Campus #3190 Salt Lake City, UT 84121, USA Salt Lake City, UT 84112, USA [email protected] 801.947.9277 [email protected] 435.649.7931
“Only if the various principles - names, However, convention speaks against having two definitions, intimations and perceptions - are people work together to develop code -- having laboriously tested and rubbed one against the “two do the work of one”, as some people see it. other in a reconciliatory tone, without ill will · Managers view programmers as a scarce during the discussion, only then will insight and resource, and are reluctant to "waste" such by reason radiate forth in each case, and achieve doubling the number of people needed to what is for man the highest possible force...” develop a piece of code. -Plato · Programming has traditionally been taught “Knowledge is commonly socially constructed, and practiced as a solitary activity. through collaborative efforts toward shared objectives or by dialogues and challenges · Many experienced programmers are very brought about by differences in persons' reluctant to program with another person. perspectives." -Salomon [1] Some say their code is "personal," or that another person would only slow them down. ABSTRACT Others say working with a partner will cause Pair or collaborative programming is where two trouble coordinating work times or code programmers develop software side by side at versions. one computer. Using interviews and controlled At the same time: experiments, the authors investigated the costs and benefits of pair programming. They found · Several well-respected programmers prefer that for a development-time cost of about 15%, working in pairs, making it their preferred pair programming improves design quality, programming style [2] [3]. reduces defects, reduces staffing risk, enhances · Seasoned pair programmers describe working technical skills, improves team communications in pairs as "more than twice as fast" [3]. and is considered more enjoyable at statistically · Qualitative evidence suggests the resulting significant levels. design is better, resulting in simpler code, Keywords easier to extend. pair programming, collaborative programming, · Even relative novices contribute to an expert's extreme programming, code reviews, people programming, according to interviews. factors, collaborative programming This raises some provocative questions. Is pair 1 INTRODUCTION programming really more effective than solo programming? What are the economics? What When pair programming, two programmers work about the people factor - enjoyment on the job? collaboratively on the same algorithm, design or programming task, sitting side by side at one Based on recent interest in pair programming, the computer. This practice has been nominated authors examined interview and experimental several times in the last decades as an improved data to understand the costs and benefits of way of developing software [2] [3]. practice. This paper presents the results of that
Page 1 investigation. Previous publications [4] [5] [6] more confident in the correctness of the results. have demonstrated that pair programming is When we finally made it to the first checkpoint beneficial. The purpose of this paper is to re- application, we zoomed through QA with hardly a examine these results and to further explain why hitch. Everyone, myself included, was amazed that pair programming is beneficial. it didn't take weeks to debug, especially given that one of the trees had recently spent SIX WEEKS in 2 A PROJECT EXPERIENCE QA hell. It was obvious that the pairs had dramatically reduced the defect rate. The following excerpt comes from an experienced programmer describing his As the merge progressed the pairs worked together organization’s first venture into pair even more closely. programming. It reveals many features of the pair ... As each subsystem was complete the pairs programming experience, as will be discussed in would get rearranged based on knowledge of the next task. This slowed things down because the this paper. new pairs would have to spend time getting in In early December my team began a high-risk phase with one another before working effectively. activity: it involved touching just about every file, But by August the pairs were fairly well cemented, merging the code together, and trying to keep to the point where they would routinely speak for everything working. Furthermore, one tree involved each other in our twice-weekly team meetings. fairly deep rearchitecture. So the merge would be Subsequent releases, both internal and external, composed of both utter tedium and massive thought went smoothly and we rarely hit massive show- effort. stopper bugs. The continual review caught many The staff agreed with my points that pair serious issues midstream, including some major programming: design problems that hadn't been noticed before. - Should significantly reduce the risk of subtle I wasn't involved in a pair until fairly late in the errors that would make debugging excruciating; game. Once my partner and I synchronized our brains, it was a great experience. - Would give us a much broader code review than we'd ever had; and He was relatively junior, but he asked the right questions and, by struggling for answers, we usually - Would provide an opportunity to communicate forced ourselves to discover the best solution to knowledge between coders. each problem. For the first few weeks, things didn't work out as ... The team members decided to do it for envisioned. Instead of doing side-by-side pair themselves. None of my pontification had as much programming, the first few people did what I called effect as experience. "partner programming": they coded individually for awhile, and then reviewed the changes with their 3 INVESTIGATIVE PATHS partner before checking-in the modifications. They We explore eight paths of software engineering reported that they were catching errors early. This was encouraging, but I was disappointed that they and organizational effectiveness. Surprisingly, weren't working together consistently. all paths point to pair programming. These investigative paths are briefly described: But about four months into the merge, I began to notice that things were changing. One pair in Economics. A recent controlled experiment [4] particular spent their whole day together, doing found only a small development cost for adding honest-to-goodness pair programming, and the other two pairs were getting much closer to that the second person. However, the resulting code ideal. In discussions it was clear that they knew why also had fewer defects. The defect removal the change was happening. It simply worked better! savings should more than offsets the development They discovered that it took longer to work cost increase. independently and then review the changes, since Satisfaction. People working in pairs found the the review process involved teaching your partner everything you had learned in making the changes. experience more enjoyable than working alone. And that took almost as long as making the Design quality. The study [4] also found that the changes to begin with. By working together they could avoid "doing it twice", the coding went faster pairs produced shorter programs than their solo due to the two-brain effect, and they were much peers, indicating superior designs. Interviewees
Page 2 made the same comments. “jelling” assignment), together the pairs only Continuous Reviews. Pair programming’s spent about 15% more time on the program than shoulder-to-shoulder technique serves as a the individuals [4]. Development costs certainly do not double with pair programming! continual design and code review, leading to most efficient defect removal rates. Relative Time: One Individual vs Two Problem solving. Interview participants Collaborators constantly refer to the team's ability to solve 200.0% "impossible" problems faster. 150.0% Learning. Pair programmers repeatedly cite how 100.0% much they learn from each other. 50.0% Team Building and Communication. Interview 0.0% participants describe that people learn to discuss Program 1 Program 2 Program 3
and work together. This improves team One Individual Two Collaborators communication and effectiveness. Staff and Project Management. Since multiple Figure 1: Programmer Time people have familiarity with each piece of code, Significantly, the resulting code has about 15% pair programming reduces staff-loss risk. fewer defects [4]. (These results are statistically In this paper, each of these investigative paths significant.) Figure 2 shows the post- will be further discussed -- reviewing the development test cases the students passed for supporting statistical and interview data and each program – essentially the percentage of the highlighting the costs and the benefits. instructor’s test cases passed. 4 ECONOMICS Post Development Test Cases Passed The affordability of pair programming is a key issue. If it is much more expensive, managers 100.0% simply will not permit it. Skeptics assume that 80.0% incorporating pair programming will double code 60.0% development expenses and critical manpower 40.0% needs. Along with code development costs, 20.0% however, other expenses, such as quality 0.0% Program 1 Program 2 Program 3 assurance and field support costs must also be considered. IBM reported spending about $250 Individuals Collaborators million repairing and reinstalling fixes to 30,000 customer-reported problems [7]. That is over Figure 2: Code Defects $8,000 for each defect! The initial 15% increase in code development In 1999, a controlled experiment run by the expense is recovered in the reduction in defects, second author at the University of Utah as the following example illustrates. Let a investigated the economics of pair programming. program of 50,000 lines of code (LOC) be Advanced undergraduates in a Software developed by a group of individual programmers Engineering course participated in the and by a group of collaborative programmers. At experiment. One third of the class coded class a typical rate of 50 LOC/hour, the individuals projects as they had for years – by themselves. will develop this code in 1000 hours. It will take The rest of the class completed their projects with the pairs 15% longer or 1150 hours -- a cost of a collaborative partner. The results of how much 150 hours. Based on representative statistics time the students spent on the assignments are reported in [7], programmers inject 100 defects shown below in Figure 1. After the initial per thousand lines of code. A thorough adjustment period in the first program (the development process removes approximately Page 3 70% of these defects. Therefore, the individuals enjoyed programming collaboratively. would be expected to have 1,500 defects remaining in their program; collaborators would ENJOY the work more because of Pair have 15% less or 1,275 – 225 less defects. Programming
In some organizations, developers’ code is passed 100% 80% to a test or quality assurance department, which 60% 40% finds and fixes many of the remaining defects. 20% Typically, in systems test it takes between four 0% [7] and 16 [8] hours per defect. Using a fairly conservative factor of 10 hours/defect, if testing PROF SUM1 SUM2 SUM3 FALL1 FALL2 FALL3 finds these “extra” 225 defects they will spend Agree Disagree 2,250 hours – fifteen times more than the collaborators “extra” 150 hours! Figure 3: Pair Enjoyment If the program is sent directly to a customer A programmer comments, instead of a test department, pair programming is “It is psychologically soothing to be sure that that no even more favorable. Industry data reports that major mistakes had been made . . . I find it between 33 and 88 hours are spent on each defect reassuring to know that [partner] is constantly found in the field [7]. Using a fairly conservative reviewing my code while I drive. I can be sure I had factor of 40 hours/defect, if the customer is done a good job if someone else I trust had been watching and approved.” plagued by these “extra” 225 defects, field support will spend 9,000 hours – sixty times Another says, more than the collaborators “extra” time! “It’s nice to celebrate with somebody when Doubtlessly, pair programming can be justified something works.” on purely economic grounds. But there are more Students prefer the 15% overhead aspects to consider. In the study previously discussed, the class was 5 SATISFACTION divided into two groups: an “individual” group in which solo programmers did all programming, If pair programming is not satisfying, and a “collaborative” group in which all programmers won't practice it. programming was done in pairs. Each Many programmers are initially skeptical, even programming assignment cycle, the individuals resistant, to programming with a partner. It takes were given one program to complete while the the conditioned solitary programmer out of their pairs were given two programs to complete. “comfort zone.” One programmer wrote, After several programming cycles, one pair “The adjustment period from solo programming to complained that this arrangement was unfair. collaborative programming was like eating a hot They felt they had to work harder than the pepper. The first time you try it, you might not like it because you are not used to it. However, the more individuals during each cycle. The instructor you eat it, the more you like it.” suggested that the students split up and work as solo programmers as part of the “individual” In statistically significant results, pair group so the would no longer feel they were programming teams who had earlier programmed being unjustly overworked. Both students alone reported that they enjoyed pair rejected this offer almost instantaneously. They programming more and that they were more did not complain about the "unjustness" of the confident in their programs than when they additional workload again. programmed alone (as the defect rates show they are entitled to be). The graph (Figure 3) shows We feel that this is a strong indicator of the results of anonymous surveys of professional pair satisfaction of pair programming. programmers and of student pair programmers at the University of Utah. Most of the programmers Page 4 6 DESIGN QUALITY the design should go and the result of arguing over which one was better often led to a truly superior The following comments came from a team lead hybrid design. who had never heard of pair programming. He described why all of his designer-programmers In the quantitative study at the University of working together at the same terminal. Utah, the pairs not only completed their programs with superior quality, but they consistently As we proceeded with our work, I started to notice implemented the same functionality as the that one team consistently produced designs of distinctly better quality than the others. I asked individuals in fewer lines of code. Details are them about this. shown in Figure 4. We believe this is an indication that the pairs had better designs. They said that they had taken to working together, on both the design and the programming. They found that their designs and programs were better Lines of Code this way. I agreed with them and made it standard for all teams to work in pairs. The design quality is 120.0% now better. 100.0% 80.0% [from the interview files of A. Cockburn] 60.0% In 1991 Nick Flor, a masters student of Cognitive 40.0% 20.0% Science, reported on distributed cognition in a 0.0% collaborative programming pair he studied. Program 1 Program 2 Program 3 Distributed cognition is a field of cognitive science based on the belief that “Anyone who has Individuals Collaborators closely observed the practices of cognition is Figure 4 struck by the fact that the “mind” rarely works alone. The intelligences revealed through these 7 CONTINUOUS REVIEWS practices are distributed – across minds, persons, Inspections were introduced more than twenty and the symbolic and physical environment [1].” years ago as a cost-effective means of detecting and removing defects from software. Results Flor recorded via video and audiotape the [10] from empirical studies consistently profess exchanges of two programmers working together the effectiveness of reviews. Even still, most on a software maintenance task. In [9], he programmers do not find inspections enjoyable or correlated specific verbal and non-verbal satisfying. As a result, inspections are often not behaviors of the two under study with known done if not mandated, and many inspections are distributed cognition theories. One of these held with unprepared inspectors. theories is “Searching Through Larger Spaces of Alternatives.” “Despite a consistent stream of positive findings over 20 years, industry adoption of inspection “A system with multiple actors possesses greater appears to remain quite low, although no definite potential for the generation of more diverse plans data exists. For example, an informal USENET for at least three reasons: (1) the actors bring survey we conducted found that 80% of 90 different prior experiences to the task; (2) they may respondents practiced inspection irregularly or not have different access to task relevant information; at all. [11]” (3) they stand in different relationships to the problem by virtue of their functional roles. . . An The theory on why inspections are effective is important consequence of the attempt to share based on the prominent knowledge that the goals and plans is that when they are in conflict, the earlier a defect is found in a product, the cheaper programmers must overtly negotiate a shared it is to fix the defect. Many sources, including course of action. In doing so, they explore a larger number of alternatives than a single programmer [8] state that it is ten times more expensive to alone might do. This reduces the chances of remove a defect for each additional process step. selecting a bad plan. [9]" This exponential cost growth is easy to A pair programmer's description matches Flor’s: understand. During an inspection, a programmer We often came up with different ideas about how might say, “The 'if' statement starting on line 450 Page 5 should also have an 'else' clause.” The design techniques, application domains, and so programmer marks the listing and promptly forth. [11].” makes the change at their computer. However, if In keeping with the known characteristics of code the software is already in the field, a customer reviews, we find practitioners citing: makes an irate call to the software shop, “It’s Christmas Eve, and all of my cash registers just · Mistakes are found as they are entered, saving crashed. I can’t sell anything!” even the cost of compilation, and providing the economic benefit of early defect In the first case, the programmer looks directly at identification and removal. the problem that was just identified to them. In · Coding standards are followed more the second case, the field maintenance team must accurately with the peer pressure to do so. work to translate the symptom (all the cash registered crashed) back to the problem (which · Team members learn to talk together and exact line(s) of code caused the crash). It is easy work together. to see why the translation of the symptom back to 8 PROBLEM SOLVING the problem costs exponentially more than direct There were times we felt that we would have given problem identification. With pair programming, up except that we “tag teamed.” I’d be on the ropes this problem identification occurs on a minute- and I’d describe the problem in such a way that he had a valuable insight. Then he’d fight on as long by-minute basis. These continual reviews not as he could and stop . . . then I’d have an insight . . . only outperform formal reviews in their defect and so on. I suppose others would call it removal speed, but they also eliminate the brainstorming, but it feels different to me. programmer’s distaste for reviews. [-David Wagstaff, Salt Lake City] The following, sardonically worded description Pair relaying is our name for the effect Wagstaff from a senior (originally skeptical) programmer describes. Indeed, pairs consistently report that shows how pairing with even a novice they solve problems faster, and that it is different contributed to his programming. from improving design quality, or detecting I was sitting with one of the least-experienced typing errors, or brainstorming. By "problem developers, working on some fairly straightforward solving", we refer to when the two are puzzled as task. Frankly, I was thinking to myself that with my to why something doesn't work as expected, or great skill in Smalltalk, I would soon be teaching this simply can't figure out how to go forward. young programmer how it’s really done. We hadn’t been programming more than a few In interviews and in off-hand remarks, minutes when the youngster asked me why I was practitioners describe contributing their doing what I was doing. Sure enough, I was off on a knowledge to the best of their abilities, in turn. bad track. I went another way. Then the They share their knowledge and energy (and also whippersnapper reminded me of the correct method name for whatever I was mistyping at the time. brainstorming) in turn, chipping steadily away at Pretty soon, he was suggesting what I should do the problem. next, meanwhile calling out my every formatting Combining brainstorming and pair relaying is error and syntax mistake. powerful. One seasoned programmer wrote, [-Ron Jeffries, from [4]] I have found that, after working with a partner, if I go A final benefit of code reviews is that reviewers back to working alone, it is like part of my mind is learn new coding idioms and language features, gone. I find myself getting confused about things. as well as more about the system. 9 LEARNING The continuous reviews of collaborative Knowledge is constantly being passed between programming create a unique educational partners, from tool usage tips (even the mouse), capability, whereby the pairs are endlessly learning from each other. “Indeed, review has a unique to programming language rules, design and educational capability: The process of analyzing programming idioms, and overall design skill. and critiquing software artifacts produced by others Learning happens in a very tight apprenticeship is a potent method for learning about languages, Page 6 mode. The partners take turns being the teacher Expert In Earshot and the taught, from moment to moment. Even The first author obtained a project management unspoken skills and habits cross partners. pattern from a set of workshops with 10 senior Line of sight learning in apprenticeships project managers. The full pattern, listed in the [12] discusses apprenticeship case studies. These Appendix, includes examples and caveats; it is summarized as: studies range from tailors to flag signalmen in the U.S. Navy to butchers in modern supermarkets. Use Expert In Earshot when novices are not learning good techniques and habits very well, but The book is subtitled "legitimate peripheral you don't want to turn the expert into a full-time participation" to highlight three key aspects of teacher. Put the expert or leader in the same apprenticeship: that the novice actively workspace as the more novice workers, so that the participates; that the novice have legitimate work novices can learn by watching and listening, while the expert does his or her usual work. Novices will to do; and that the novice works on the periphery, pick up expertise and (hopefully good) habits from steadily moving toward some higher rank. The the expert. (Your expert will be disturbed more novice's work is initially simple and non-critical. often, so you will have to set up ways to create Later work is more critical. personal quiet time.) One of the most distinctive characteristics they Note the overlap with the apprenticeship studies. note of successful apprenticeship environments is It is significant that this pattern had to be that the novice work in a "line of sight" of the accepted by all 10 senior project managers, since expert, that expertise is transmitted, in part, they were going to apply it within their company. through the ongoing visual (and auditory) field. Pair programming is an example of both Expert They describe successful apprenticeship learning In Earshot and of legitimate peripheral in both tailors and Navy signalmen where “line of participation, with line-of-sight/hearing access. sight” is available. The beginner explicitly picks We can expect, therefore, that the learning that up skills from hearing and/or seeing the expert. occurs in pair programming is more significant The most interesting of the case studies, for this than merely learning new tool usage or discussion, is that of the butchers in programming language idioms. This expectation supermarkets. They did not have line of sight matches off-hand reports from practitioners. access to their local expert. The beginners were Statistical confirmation given simple cuts to perform, but did not have a Pair programming was used exclusively in a web way to learn how to do more difficult cuts, which programming class at the University of Utah, were being done by the senior butcher in another taught by the second author. The class consisted room. Lave and Wenger present this as a of 20 juniors and seniors, familiar with situation in which apprenticeship learning does programming, but not with web programming not happen. languages and tools. The majority of the students It should be obvious that most project had only used WYSIWYG web page editors prior programming environments match the butcher to taking the class. During the eleven-week situation, not the tailor or signalmen situation. semester, the students learned advanced HTML, Indeed, we have found it extremely difficult to JavaScript, VBScript, Active Server Page set up programming environments in which line- Scripting, Microsoft Access/SQL and some of-sight and line-of-hearing from expert to novice ActiveX commands. In many cases, they had to can be accommodated. The novice programmer intertwine statements from all these languages in generally sits in their workspace working on one program listing. simple code; the expert sits in their own Unusual for such students, they produced their workspace creating complex code and making programs with minimal questions of the teaching architectural decisions. Pair programming makes staff. The students were queried about the a better apprenticeship situation. reasons for their independence in an anonymous
Page 7 survey on the last day of class. The Psychology of Computer Programming [13] and Peopleware [14], written 20 and 30 years · 74% wrote “between my partner and me, we ago, respectively, still have not been replaced on could figure everything out.” the topic of teamwork. More recently, eXtreme · 84% of the class agreed with the statement “I Programming [2], the Crystal Light learned Active Server Pages faster and better methodologies [15] and Adaptive Software because I was always working with a Engineering [16] have strengthened attention partner.” given to team building and communication. In We would attribute part of that result to enhanced [17], Cockburn goes farther, and argues that these problem solving in pairs, as described above, and are first-order project drivers, not side issues. part to enhanced pair learning. Pair programming contributes in three ways. 10 TEAM BUILDING&COMMUNICATION People learn to work together, as illustrated in the When I arrived, I saw a disheartening sight: Bill above quote. In the University of Utah study, didn't have a team; he had a random collection of six bright, talented individuals who didn't work none of the 14 pairs ran into an insurmountable together. They didn't sit near each other. They didn't personality clash. In industry, off-hand even like each other. Here is a scene from a weekly comments indicate personality clashes staff meeting occasionally happen, perhaps because there is not 'Let's talk about pair programming. (benefits of pair sufficient pressure or motivation for the people to programming enumerated)
Page 11