Improving the Internal Quality of Software Through Reputation-Based Gamiﬁcation

Improving the Internal Quality of Software through Reputation-based Gamiﬁcation

Von der Fakultat¨ fur¨ Mathematik, Informatik und Naturwissenschaften der RWTH Aachen University zur Erlangung des akademischen Grades eines Doktors der Naturwissenschaften genehmigte Dissertation

vorgelegt von

Diplom-Informatiker

Christian Reinhard Prause

aus Siegburg

Berichter: Prof. Dr. Matthias Jarke Prof. Dr. Volker Wulf

Tag der mundlichen¨ Prufung:¨ 21.3.2013

Diese Dissertation ist auf den Internetseiten der Hochschulbibliothek online verfugbar.¨

Abstract Zusammenfassung

Developers are notorious for their dislike of writing in- Entwickler sind beruchtigt¨ fur¨ ihre Abneigung gegen ternal documentation. It has few value to them because das Schreiben von Dokumentation. Der personliche¨ its benefits are too far away in the future and distributed Nutzen hiervon liegt zu weit in der Zukunft und wird on the whole team, while the efforts of documenting mit anderen geteilt, wahrend¨ der Schreibaufwand auf are burdened on the individual himself. The resulting dem Einzelnen lastet. Das Ergebnis sind Phanomene¨ phenomena have names like free riding or tragedy of the wie Trittbrettfahren oder die Tragik der Allmende. commons. Software ist eine Ansammlung kodierten Wissens Software is an accumulation of encoded knowledge und ein Auffangbecken fur¨ Komplexitat.¨ Ihr Entste- and a collecting pond for complexity. Its development hungsprozess gleicht eher Forschung und Evolution als is more akin to research and evolution than to an agreed einem festen Weg mit klarem Ziel. Die gemeinsame Ar- path with an established goal. Multiple developers are beit mehrerer Entwickler wird von unterschiedlichsten engaged in an intricate joint endeavor, building on dis- Interessenten begleitet, und besteht aus Entdeckung, covery, invention and decision making, flanked by sev- Erfindung und Entscheidung. Um die Software an sich eral stakeholders. A developer must constantly learn laufend andernde¨ Anforderungen anpassen zu konnen,¨ and relearn the software to be able to adapt it to ever- muss ein Entwickler ihr Inneres standig¨ (wieder) er- changing requirements. Instead of being “soft” and easy lernen. Anstatt leicht anpassbar zu sein, ist Software to adapt, software is difficult to modify. If knowledge is schwer zu andern;¨ ihre Entwicklung ist daher teuer. not preserved in source code, wiki articles, and other doc- Wird das Wissen nicht in Quellcode, Wiki-Artikeln umentation, then software development becomes costly oder anderer Dokumentation wohl bewahrt, droht ein even up to the point of a total economic loss. wirtschaftlicher Totalschaden. This research presents CollabReview, which ad- Diese Arbeit prasentiert¨ CollabReview, das anstatt dresses the developers’ motivation to invest in internal strenge Regeln vorzugeben, die Motivation der Entwick- quality without strict regulation. It re-balances intrap- ler anspricht, in interne Qualitat¨ zu investieren. Es bee- ersonal cost-benefit considerations in favor of internal influsst intrapersonelle Kosten-Nutzen-Uberlegungen¨ quality using gamification. Personal reputation scores zu Gunsten interner Qualitat¨ mittels Spielelementen. for developers are calculated from ownership-based Dazu werden personliche¨ Reputationspunkte fur¨ En- responsibility and quality ratings for collaboratively twickler aus Qualitats-¨ und Zustandigkeitsinformationen¨ written software process artifacts. The scores are tied to von gemeinschaftlich entwickelten Software-Artefakten social games and rewards to make scores worthwhile to berechnet. Um die Punkte erstrebenswert zu machen, achieve. The implemented prototype of the CollabRe- sind sie an soziale Spiele und Belohnungen gekoppelt. view concept mines responsibility information from Der CollabReview Prototyp gewinnt seine Informatio- revision histories, and combines it with manual and nen aus Entwicklungshistorien, und kombiniert sie mit automated assessments of artifact quality. manuell und automatisch erzeugten Qualitatsreviews.¨ The CollabReview concept and prototype were eval- Das Konzept und der Prototyp von CollabReview wur- uated in several studies including an analytical con- den in Expertenanalysen, technischen Validierungen und cept evaluation, technical validations and field exper- Feldversuchen evaluiert. Letztere brachten den Prototyp iments. The latter deployed the prototype in knowledge in agilen, quelloffenen, verteilten Konsortial- und Wis- management, agile, open source or distributed consor- sensverwaltungsprojekten zum Einsatz. Die Beitrage¨ tium projects. It successfully amplified contributions to zur Wissensverwaltung konnten so erfolgreich gesteigert knowledge management wikis, and improved the read- und die Lesbarkeit von Quellcode verbessert werden. ability of source code. The results show that CollabRe- Die Ergebnisse zeigen, dass CollabReview zu besserer view leads to better internal documentation and quality interner Dokumentation und Qualitat¨ der Software fuhrt.¨ of the whole software. At the same time, it leaves devel- Gleichzeitig lasst¨ es Entwicklern die Freiheit, Qualitat¨ opers their freedom to neglect quality when necessary, wenn notig¨ zu vernachlassigen,¨ belastet sie nicht mit does not burden them with unnecessary additional ef- unnotigem¨ Mehraufwand, und hat folglich besonderen forts, and hence has particular value in self-organizing Wert in selbst-organisierenden Projekten mit schwacher project environments where code ownership is weak. Quellcode-Eigentumerschaft.¨

iii iv Contents

1 Introduction 11

2 Problem Scope: On the Quality of Software Process Artifacts 13 2.1 Introduction to Software Process Artifacts ...... 13 2.1.1 Software as collection of knowledge ...... 13 2.1.2 Software development as costly knowledge creation process ...... 15 2.1.3 Software process artifacts document created knowledge ...... 17 2.1.4 Source code and other documentation, and how to best make it ...... 20 2.1.5 Summary of software development as knowledge creation ...... 21 2.2 The Case for Internal Quality ...... 22 2.2.1 Source code as the project’s most valuable output ...... 22 2.2.2 Protecting the value in source code, and the cost of maintenance ...... 22 2.2.3 Why and where internal quality matters ...... 25 2.2.4 Summary of the importance of internal quality ...... 27 2.3 An As-is Analysis of Internal Quality ...... 27 2.3.1 The software crisis in literature ...... 28 2.3.2 Evidence of the problems in popular developer culture ...... 29 2.3.3 Interviews with Software engineering experts ...... 29 2.3.4 The Karakos EU project case study ...... 31 2.3.5 Karakos is not a black swan – EU project survey ...... 38 2.3.6 Diverse Own Experiences ...... 40 2.3.7 Summary ...... 44 2.4 Analysis of Problem Causes ...... 44 2.4.1 How people, processes and technology affect quality ...... 44 2.4.2 Resistance against new processes and technologies ...... 46 2.4.3 Economic goods and the tragedy of commons ...... 46 2.4.4 The tragedy of commons for internal quality ...... 47 2.4.5 Summary of origins ...... 50 2.5 Summary of the Problem ...... 50

v CONTENTS

3 State of the Art 53 3.1 Software Engineering ...... 53 3.1.1 Introduction ...... 53 3.1.2 Computer-aided software engineering ...... 54 3.1.3 Development methodologies ...... 56 3.1.4 Summary ...... 57 3.2 Software Quality ...... 57 3.2.1 The nature of quality ...... 58 3.2.2 Reducing internal failure cost with internal quality ...... 60 3.2.3 Quality management ...... 63 3.2.4 Summary ...... 66 3.3 Testing ...... 67 3.3.1 Map of testing ...... 67 3.3.2 Manual dynamic testing ...... 68 3.3.3 Automatic dynamic testing ...... 68 3.3.4 Manual static testing ...... 69 3.3.5 Automatic static testing ...... 70 3.3.6 Summary ...... 72 3.4 Mining Software Repositories ...... 73 3.4.1 Contribution, knowledge and ownership ...... 73 3.4.2 Determining contributors in evolving artifacts ...... 73 3.4.3 Concepts of contribution mining ...... 76 3.4.4 Software process artifact halo ...... 79 3.4.5 Information retrieval ...... 79 3.4.6 Summary ...... 81 3.5 Collaboration, Organization and Motivation ...... 81 3.5.1 Introduction ...... 81 3.5.2 Collaboration ...... 81 3.5.3 Organization ...... 84 3.5.4 Management and motivation ...... 86 3.5.5 Summary ...... 90 3.6 Comparable Works ...... 90 3.6.1 In ubiquitous computing ...... 90 3.6.2 In online collaboration ...... 91 3.6.3 In software development ...... 91 3.6.4 Summary of comparable work ...... 91 3.7 Summary ...... 92 vi CONTENTS

4 Thesis Statement 93

5 Design Process and CollabReview Concept 95 5.1 The Research Process ...... 95 5.1.1 The iterative process of this research ...... 96 5.1.2 List of requirements ...... 96 5.1.3 Details of the research iterations ...... 99 5.1.4 Summary of the research process ...... 105 5.2 The Computational Model ...... 105 5.2.1 Modeling software process artifacts ...... 105 5.2.2 Importance weight of software process artifacts ...... 107 5.2.3 Responsibility of contributors for software process artifacts ...... 107 5.2.4 Quality of software process artifacts ...... 109 5.2.5 Putting it together to karma scores ...... 111 5.2.6 An example of karma scores ...... 112 5.2.7 Summary ...... 113 5.3 Authorship Algorithm ...... 113 5.3.1 Characteristics of the algorithm ...... 113 5.3.2 High-level description of the algorithm ...... 114 5.3.3 Details of algorithm Step 2: Find the artifact’s parent ...... 116 5.3.4 Details of algorithm Step 5: Handle copied code ...... 118 5.3.5 Summary of authorship algorithm ...... 120 5.4 Rewarding Schemes ...... 120 5.4.1 Karma scores and exchange value ...... 120 5.4.2 Reward patterns ...... 122 5.4.3 Summary of rewarding schemes ...... 123 5.5 Summary of Design ...... 123

6 The CollabReview Prototype 125 6.1 Architecture ...... 125 6.1.1 Overview of the prototype’s architecture ...... 125 6.1.2 The major subsystems of the CollabReview prototype ...... 128 6.1.3 Summary of architecture ...... 132 6.2 Application and User Interfaces ...... 132 6.2.1 Web user interfaces ...... 132 6.2.2 Web services ...... 137 6.2.3 IDE plug-ins ...... 140 6.2.4 Email notiﬁcations ...... 140 6.2.5 Summary of CollabReview prototype interfaces ...... 142 6.3 Summary of the prototype ...... 142

vii CONTENTS

7 Validation and Evaluation 145 7.1 Procedure ...... 145 7.1.1 Evaluation methods ...... 146 7.1.2 Statistical tools ...... 148 7.1.3 Ethics ...... 149 7.1.4 Terminology of people ...... 149 7.1.5 Targeted user group ...... 150 7.1.6 Study Environments ...... 151 7.2 Evaluation of Authorship Algorithm ...... 156 7.2.1 Purpose ...... 157 7.2.2 Method ...... 157 7.2.3 Data ...... 157 7.2.4 Results ...... 158 7.2.5 Summary ...... 160 7.3 Consistency of Karma Scores ...... 161 7.3.1 Purpose ...... 161 7.3.2 Method ...... 161 7.3.3 Data ...... 163 7.3.4 Results and discussion ...... 165 7.3.5 Summary ...... 167 7.4 Validity of Computed Karma Scores ...... 167 7.4.1 Purpose ...... 167 7.4.2 Method ...... 168 7.4.3 Data and results ...... 168 7.4.4 Summary ...... 171 7.5 Analytical Concept Evaluation ...... 171 7.5.1 Purpose ...... 171 7.5.2 Method ...... 171 7.5.3 Results ...... 172 7.5.4 Summary ...... 173 7.6 Moknowpedia Field Experiment ...... 173 7.6.1 Purpose ...... 173 7.6.2 Method ...... 173 7.6.3 Data and results ...... 174 7.6.4 Discussion and threats to validity ...... 178 viii CONTENTS

7.6.5 Summary ...... 179 7.7 AgileLab Field Experiment ...... 179 7.7.1 Purpose ...... 179 7.7.2 Method ...... 179 7.7.3 Data ...... 181 7.7.4 Results and Discussion ...... 181 7.7.5 Summary ...... 187 7.8 TornBond Design Experimentation ...... 187 7.8.1 Purpose ...... 187 7.8.2 Method ...... 188 7.8.3 Data ...... 188 7.8.4 Results ...... 188 7.8.5 Summary ...... 190 7.9 ReputationForge Field Experiments ...... 190 7.9.1 Purpose ...... 190 7.9.2 Method ...... 191 7.9.3 Data ...... 191 7.9.4 Results ...... 192 7.9.5 Summary ...... 192 7.10 MicroEditCastigator Field Experiment ...... 193 7.10.1 Purpose ...... 194 7.10.2 Method ...... 194 7.10.3 Data ...... 194 7.10.4 Results and discussion ...... 194 7.10.5 Summary ...... 196 7.11 ScrumLab Field Experiment ...... 196 7.11.1 Purpose ...... 196 7.11.2 Method ...... 196 7.11.3 Data ...... 198 7.11.4 Results ...... 205 7.11.5 Summary ...... 207 7.12 Overview of Studies ...... 208 7.13 Validation of Requirements and Concept Evaluation ...... 210 7.13.1 Validation of Requirement-1 “Improve internal software quality” ...... 210 7.13.2 Validation of Requirement-2 “Appeal to individuals’ motivation” ...... 211 7.13.3 Validation of Requirement-3 “Flexible applicability” ...... 212 7.13.4 Validation of Requirement-4 “No hazard to the team” ...... 213 7.13.5 Validation of Requirement-5 “Robustness and resistance against attacks” ...... 213 7.13.6 Validation of Requirement-6 “Involve contributors without dependence” ...... 214 7.13.7 Validation of Requirement-7 “Validity of computations” ...... 214 7.13.8 Validation of Requirement-8 “Few overhead for users” ...... 214 7.13.9 Validation of Requirement-9 “Easily understandable” ...... 215 7.13.10 Summary of requirements validation ...... 215 7.14 Evaluation of CollabReview ...... 215

ix CONTENTS

8 Conclusion 219 8.1 Retrospective ...... 219 8.1.1 Summary ...... 219 8.1.2 Contributions ...... 220 8.1.3 Lessons learned ...... 221 8.2 Future work ...... 222 8.2.1 Open questions ...... 222 8.2.2 Further research opportunities ...... 222

Danksagung (Acknowledgments) 225

Registers and lists 227 List of Acronyms ...... 227 List of Figures ...... 229 List of Tables ...... 231 Bibliography ...... 232 Index ...... 248

x T hc sipeetdi h eairo otae How- software. of behavior the in implemented is which developer a costs example, for software, of workings Introduction 1 Chapter rietesfwr rdal eoe oeadmore and more becomes Oth- gradually developers. software here, the and, erwise users to ex- software further the which plains documentation high-quality accompa- by be nied to has functionality implemented its knowledge ever, detailed of mass huge a accumulate tems the maintenance. to its incentive few for is pay there to and but individual there condition be good to a it in expects be Everybody that good. is public analogy a economic of The team. is knowledge the of for are invaluable preservation Yet and team. later the much developer with off the shared pay for benefits value Potential few has himself. but time precious his inner the about individ- knowledge the down for Writing value direct himself. few ual has be- often quality so internal doing improve cause that activities have for developers dislike that a is reason management major or Another pressure ignorance. time like projects reasons software various in for neglected But often software. is of quality quality internal internal the good determines How are system. they software the building and while obtained applied was that knowledge the preserve mentation with. contact into gets is team quality development internal the see, which that users which opposed quality As external quality. internal to on focusing by achieved is h rbe shgl eeat ag otaesys- software Large relevant: highly is problem The docu- and code source like artifacts process Software HIS ult ftesfwr hti en eeoe.It developed. being is that software the the improve of to quality is goal The development. software eerhdaswt dacn h rcs of process the advancing with deals research ainConway Damian — ouetto salv etrta o rt oyu uueself. future your to write you that letter love a is Documentation rval,ti ok gis h ifso fresponsi- of diffusion the against works this Trivially, au htterpoet a o h em Furthermore, team. the for has property their that value n eurmns n gsqiky nteed valu- end, the In quickly. ages chang- and always requirements, with ing up keep cannot modify, to difficult tbcmspsil oascaeaprevbedirect perceivable a associate to possible becomes it the in stake a have now Developers good. common privatizes the and artifacts, created collaboratively in bility responsibility. and ownership be unearth can to order artifacts in developed mined collaboratively development The of project. the history for have they for that and value artifacts, the process comes software ownership the With for it. responsibility artifact to contributing process by software gained is a is for reasoning software Ownership The follows: the quality. as internal into improve elements to project market economic scale loss. economic total The a is lost. project are imple- efforts the invested all and consequence, knowledge a mentation As discarded. implementation under- be the only to that can means costly it too then modify, are or they stand because reused projects be cannot later prototypes in such a If just probe. than design more quick being for developed are pro- which i.e. totypes projects prototypes, Research software large-scale generate components. often software of reuse eas- enables ier and costs, maintenance in software decreases quality efficient, long-term more and itself quality development makes external use, better to leads internal quality Better alone. from instructions it machine-readable recover or the to difficult requirements) too is (e.g. it impossible because even lost is knowledge detail able h prahpeetdi hstei rnsmicro- brings thesis this in presented approach The 11

I—INTRODUCTION Chapter 7 validates the prototype against defined re- Chapter 8 concludes the research by summarizing Chapter 4 therefore introduces such a new approach. Chapter 5 explains how the concept was designed. It Chapter 6 describes the CollabReview prototype, concept and design. CollabReview isIn later this used chapter, the to and focus the is interfaces and on userprototype. software interfaces architecture, provided by the quirements, and evaluates its benefit.the First, validation it and evaluation specifies procedure, the targetedgroup, user and summarizes the differentserved environments that as real-world contexts forstudies field were studies. conducted Several to test thatrequirements previously formulated are met. It showstion that is the computed way is how consistent reputa- from and analytical valid, reports concept results evaluations, andseveral describes field the experiments and their resultsto that trial were the used developed concept andchapter prototype. evaluates the Finally, the merit, worth,the and concept significance by of systematically reviewing andthe assessing results obtained in the studies. on its contributions and presenting an outlook to future It is based on thecan claim be internalized that with software a process reputation-basedand software artifacts can tool, be used tohimself encourage to the care individual more developer forcommon the property quality when of combined a with projectPro-social suitable team’s behavior rewards. that helps theto team, internal i.e. quality, earns contributing the developersscore. a Consequently, high documentation reputation has an immediate software quality. starts out with the researchtual process, requirements presents the that concep- werecess, gathered explains through the the computational pro- modeltion behind scores, reputa- shows how tofrom obtain collaboratively authorship written information texts, andrewarding sketches mechanisms. possible validate and evaluate the researched concept and design. value for the individual. This leads to steadily improving which is the software system that implements the work. NTRODUCTION I—I projects. There EU ) projects) because it does not EU Chapter 3 presents a survey of related work. It de- Chapter 2 elaborates the problem scope of this re- The game makes it possible to appeal to the develop- like collaboration, organization, management and moti- erature review reveals that the above problems have not is required. But ittrying also similar approaches shows in that other currentdevelopment domains as research and well. is software haps makes them particularlynomenon susceptible as compared to to this other phe- kinds of projects. scribes relevant concepts and approachesof in software the engineering, domains software qualitytesting, management, mining software repositories, and in related areas lems. The reasons arecommons, attributed a to sociological the phenomenon tragedy thatdecay of causes of the valuable the resources. Theinitiated original the research context process that were the problem reveals itself morenounced clearly organizational due maturity to of less consortia, pro- which per- search. On theples basis of of software source process code artifacts,the and importance the of other chapter internal exam- stresses qualitythough for good collaboration. internal Al- quality is knownit to is be so infamous important, for beingcauses fallen unnecessary costs, short delays, of. and This exploitation prob- situation require enforcement and control basederarchical on structures. established And hi- it stillflexibility leaves without developers forcing their them toimproved comply. internal quality In reduces the development end, costsmakes and industrial exploitation easier. nal quality. Selfishness guides developersinvisible like Smith’s hand to dothe team. more of The theabling effect the activities assists self-management that of quality software help assurancequality. process artifact by The en- concepttributed, is consortium-based especially projects interesting with flat for hierarchies dis- quality. The realization of thiscomputer-implemented concept reputation is system supported that by computes a individual scores for developers. Thesebasis for scores reinforcing are desired the behavior. ers’ selfishness to bring forth the improvement of inter- property. Taking care ofteam artifacts becomes for the a goodindividual game. developers of to the take pro-social care Selfishness of internal then motivates vation, also including economics. Additionally, the lit- yet been tackled satisfactorily and that a new approach value with the abstract value-for-the-team of their (like European Union ( 12

I—INTRODUCTION P h term The it Instead, change. to trivial not is software Today’s eomn fsfwr aeahg neeti odin- good in interest high a have software of velopment otaePoesArtifacts Process of Software Quality the On Scope: Problem 2 Chapter t agbecounterpart tangible its knowledge of collection as Software 2.1.1 preserved. knowl- is this edge how investigates and research- process, a evolution through like en- being as to comes software that defines knowledge first coded section of This workings inner software. the the re-learn and learn who constantly developers, must knowledgeable of hand the by evolves Pro- Software to Introduction 2.1 neglected. therefore that that is and is it documentation, reason writing primary like the not that do argued developers be will achieve I to goal. fail that often very they However, quality. ternal de- the in involved are that organizations most quently, OOR oko nodrt hneasfwr.Conse- software. a change to order need in they what know learn to to developers for difficult is it esArtifacts cess ult fsfwr rcs riat en that means artifacts process software of quality software a nrdcdt iciiaei from it discriminate to introduced was hre .CnelJr. Connell H. Charles — uirpormescet ipesltost ipepolm.Se- problems. simple to solutions simple create programmers Junior ra rgamr n ipesltost ope rbes The problems. complex to solutions simple find programmers Great sfiihd u ti atymr ifiutt create. to difficult more vastly is it it once but obvious, finished, appear is may programmers topnotch by written code problems. complex to solutions complex create programmers nior hardware otae“sasubset a “is Software . 99.A t rfi sf”ipis otaei easy is software implies, “soft” prefix its As 1979). sdi nweg aaeetdisciplines: management knowledge definition in the used follows episte- pragmatically of work discipline This philosophical mology. the in debate of matter terms the with information involved often is confusion Some knowl- and information data, of Pyramid 2.1.1.1 is It 2005). knowledge. (Brockhaus, encoded system tangible computer the a in of hardware counterpart immaterial the is Software Software – 1 Definition and (Cerri invention 1997). and Bailin, discovery 2007; Fuggetta, of process the development Its is encoding. its and builds. knowledge man is Software that for things sink in as complexity ever-increasing serves the software extremely Today, is change. software to complexity, became difficult its it to soon But due to software that 1997). clear used sense, (Bailin, some change” it to in least, easy that, At is belief unspoken 1979). “an al., be et (Bersoff change to al., et (Bersoff information.” called things of set the of edge n especially and knowledge hc ssila still is which , data 13 ,

II — PROBLEM SCOPE:ONTHE QUALITYOF SOFTWARE PROCESS ARTIFACTS Semantics context Syntax Pragmatics integration ) (Harman, 2010), RTIFACTS SCAM A Data Characters Knowledge Information ROCESS P is any human-readable description that also can read to build the fully executable soft- here refers to any detailed, technical plan that ¨ uller lives OFTWARE S ¨ ¨ uller uller is a last In the software domain, engineers use blueprints, too. Producing software from source code is by definition compiler Mr. M in Hamburg. He is a potential customer... M name M ’A’, ’b’, ’c’, ... an automatic process with littlecepted cost. source code If once, a then compilerover it ac- again. will As accept opposed it to overtion and other is industries, a mass matter produc- of a few Watts. Originally, a form of paper-based reproduction,blueprint the term documents an architecture, designis or not construction. necessarily It ofblueprints graphical to nature. communicate with Engineerscraftsmen each use who other, build prototypes and according with to them. tion largely borrows from the terminologyIEEE International given Working by Conference the on Source Code but does not follow itsi.e. view instructions to include immediately machine executableProcessing Unit code, by (CPU). a Central Definition 6 – Source code a most precise description of software behavior. Figure 2.1: The pyramid ofdata knowledge, (adapted information from and Bodendorf (2005)) Definition 5 – Blueprint Source code They are called source code here. The following defini- Analysis and Manipulation ( ware. Source code is the blueprint of software. It is the UALITYOF Q NTHE :O COPE S ROBLEM II — P Data is on the lowest level of abstraction. All three Finally, data is information that does not carry mean- Information is partial knowledge that lacks the seman- Engineers also communicate withblueprints, each learn other from through them, anda incrementally improve design. Only ifdustrial engineers, blueprints then are research understandable resultsand to are actually in- repeatable usable. Consequently, thesebaseline plans for are later the industrialoptimization production efforts. and continued cesses used in other industrialbe sectors easily can transferred therefore to not theand production Fuggetta, of 2007). software (Cerri However, all kindshave of in production common thatof blueprints any are innovation the phase. major output Engineers use them to give knowledge at the top (see Figure 2.1). 2.1.1.2 Software as an industrial product Software is a relatively new and unique technology. Pro- Data are sequences of characterstion that so represent that informa- they canhaus, be 2005). processed by a computer (Brock- terms are often represented in a pyramid with the most ing. Data isments, obtained statistical through surveys observations, and the measure- like (Brockhaus, 2005). Definition 4 – Data Definition 3 – Information Information is a subset of knowledgeby that a may be certain needed person orsolve group a problem in (Herget a and concrete Kuhlen, situation 1990). to decision making. Knowledgetacit, may individual be and/or collective” explicit (CWA Std. and/or 2004). 14924-1, tic integration into a knowledge context. Definition 2 – Knowledge to which is added expertto opinion, skills result and in experience, a valuable asset which can be used to aid valuable, most complex and most difficult to obtain work descriptions to craftsmen who then craft prototypes. 14 “Knowledge is the combination of data and information,

II — PROBLEM SCOPE:ONTHE QUALITYOF SOFTWARE PROCESS ARTIFACTS BugeadDti,2000). Dutoit, and (Bruegge Prototypes ain(eai,21) ti hsisac fsoftware of instance this is infor- It genetic inheritable 2011). (Nehaniv, its of mation while role software, the the plays code of source instance is fielded individual an a of by role played The population. evolving an individuals in of identification the in is requisite evolution definitional Darwinian A the individuals. beyond single also of reproduce, lifespan are and better, survive fit to likely that more Individuals pop- individuals. by of undergone ulations existence for struggle the Darwinian is 2006). evolution al., et (Nehaniv in industry efforts and academia and bio- interest Darwin’s increasing with to evolution logical compared is costly development Software as development Software 2.1.2 in fsfwr htcnltrb xeddit h full the into extended be later product. can that software of sions Prototype – 7 Definition 2005). Lewis, evolu- 1982; its Jenkins, end and can (Naumann prototype tion fitter a by supplanting environments. evolutionOnly changing then to even adaptation But through continues prototype. sup- larger or system a production plements a into evolves prototype Eventually adding the performance. iteration optimizing each with and environment functions its more for fit is long it as as expansion prototype and a modification of evolution through But continues functionality, crude. final look product’s and the bugs per- have of build, subset to a quick only and form cheap less be and may complex They less are complete. prototypes early life, early the trade-offs and alternatives process a justifications, in decisions involving to come and with solutions, experiment possible issues, identify Developers activity: experimen-tal non-algorithmic, a is development Software ex- purposeful through evolution Software 2.1.2.1 development. software of case the knowledge in encoded as of creation the is evolution logical that environment target its called of is context the in trialed is that rttpsaetilvrin fasse.Smlrto Similar system. a of versions trial are Prototypes nweg rainprocess creation knowledge prototype perimentation r nopee ha n ih-egtver- light-weight and cheap incomplete, are NuanadJnis 92.Bio- 1982). Jenkins, and (Naumann I1—Itouto oSfwr rcs Artifacts Process Software to Introduction — II-1 h lesadetnstepoetskoldebase, knowledge project’s the extends and alters who (software) A h project’s The ssr sdahadtxs otaeta al ocatch to fails that Software taxes. and death as is sure requirements as through change represented continuous its environment the improve the software, to of of able world be the will In prototype fitness. a fast determines how It and the Yet if cost. operational an users. better has their effort and i.e. developers’ environment sense, their Darwinian to the in adapted fitter often are that code. source in encoding its on focusing Developer – 8 Definition develop- by realized purpose. is on but and random ers at happen accident. not by does come It not does software in change however, a hrfr oxs ntekoldebase. knowledge the in and coexist information therefore different can store purpose, not same do the they serve related, While are artifact. requirement requirement and a implementation a as have stored to is wish feature users’ certain The information. with of artifact process pieces software other same implementa- the Its share software. may the tion across reused once be base) then knowledge and the into written (i.e. plemented of piece another from derivable information. automatically is information of it piece in no particular, in and information, of is another it contradicts If information one. of piece no and itself, is base the If case: minimal. and consistent is It knowledge. project’s of the information of pieces storing media machine-readable base Knowledge – 9 Definition 2012). Kanat-Alexander, will (e.g. software requirements a emerging that with chance cope the higher the per- changes, and developers form fitness easier The losing useless. by becomes ages eventually requirements, new with up h hne htdvlpr aela oprototypes to lead make developers that changes The metaphor, biological the in genome the to opposed As o xml,cranfntoaiysol nyb im- be only should functionality certain example, For of definition Above minimal nweg base knowledge developer consistent hni osntdpiaeaypiece any duplicate not does it then , nweg base knowledge sacnrbtrt h project the to contributor a is hni osntcontradict not does it then , ossso l ua-or human- all of consists sue h ideal the assumes 15

II — PROBLEM SCOPE:ONTHE QUALITYOF SOFTWARE PROCESS ARTIFACTS RTIFACTS ) or an issue tracker. The A IDE ROCESS P vironment OFTWARE S For a developer, there are two ways of acquiring Working on the development of software is a continu- communication is usually preferable becauseand it is of faster higher quality.ronment A maximizes good transfer personal of existing learningeases knowledge, envi- the and generation of new knowledge (Prause, 2011). requirements, searchable bug records anddelivered so to on, them that through are tools like their Integrated De- software project with tools thatcated grant access views to and the sophisti- knowledge,taining the documentation the con- information, and colleaguesformation to with exchange which in- become thelearning developers’ environments personal (Prause, 2011). 2.1.2.3 Constituents of a good personal learning en- they can employ their abilitiessoftware. to They make do changes not to need the and to over reinvent again the but wheel can over project’s efficiently knowledge expand (Humphrey, the 1997). software But whatstitutes con- a good personal learning environment? project knowledge: either through experimentation,changing i.e. something and seeing how it affects the soft- nication, i.e. exchange of existingdevelopers. knowledge In between the case of complex and abstract matters, experimenting with models, addingtions, layers and of forming vocabularies, abstrac- aretific more process like than a engineering scien- disciplineCurtis (Bailin, et 1997). al. (1988) citeone a had developer’s words to that spend “Some- aedge hundred in my million head. to It putis didn’t that too come knowl- complex free.” and Modern software toofully large understand in for detail. a But singleunderstanding developers developer for need to a performing certain changes. For example, pensive maintainers that money canmore buy than any are other worth maintainer, no ifof they the cannot necessary get knowledge. a They hold cannot use their ad- are no better. ous learning process. The developers’are learning process materials artifacts like sourcehistory, code, etc.; requirements, their bug personal learning environments consist of their individual selections of pieces of source code, velopment Environment ( velopers who are in search of information, the better vanced skills, are no faster, and the changes they make The better knowledge is preserved to be learned by de- Tryggeseth (1997) finds that even the best and most ex- ware (and the happiness of its users), or through commu- pro- UALITYOF Q NTHE :O A knowledge base COPE is purposefully not re- S but includes any support- 1 ROBLEM developer II — P ation and learning . As Bailin (1997) notes, a better term for that would then be 1 If all that what multiple developers do while they In practice, however, the knowledge base rarely is evolve software over months or yearsedge, is then to a create single knowl- softwaremore project than resembles engineering. science The developers’ activities like grammer edge. Such supportingarchitecture material designs can to convey be, a for broadersoftware, requirement picture example, descriptions of to the gainsoftware insights into needs, test casesior, that comments define that expected supplement behav- sourcehistory code, that or preserves a decision change rationale. 2.1.2.2 Software development as knowledge cre- stricted to source code alone ing material that conveys parts of the project’s knowl- Definition 10 – Fork original knowledge base it was forked from. may potentially be forked toferent serve software a project. very similar Two artifacts butpieces with dif- of contradicting information may existof legally in the different same forks knowledge base. detect. Furthermore, the knowledge basedocumentation may that contain was automatically generatedsource from code. But such knowledgeas base it is duplicates not information that minimal automatically. can be derived from it Knowledge base forking remark not cover all information.software For module example, may a modify change its inthat behavior it a in is such no a longer way to consistent with work other with. modules it Someautomatically has by inconsistencies tools may like be abetween detected compiler. behavior implemented A in discrepancy sourcehavior code expressed and in be- requirements is far more delicate to neither consistent nor minimal. Possibilitiesconsistency for and testing minimality are limited;may media overlap objects and duplicate information, or get lost or Above definition of A different knowledge base that is very similar to the 16

II — PROBLEM SCOPE:ONTHE QUALITYOF SOFTWARE PROCESS ARTIFACTS hmvri a ocr hl ae nomto needs information later while concern may it whomever need in when received is and completed is task a when eoest er h otae hpe ilcm back come will 3 Chapter software. the learn to velopers through learn and behavior software with play velopers knowl- and information confer to ability its i.e. value, r o elkona h ieo sending. of time the at known well not are to preemptively broadcast but a developers between of communication set known directed a docu- not And is (2000). mentation Dutoit and written Bruegge gets information of it e.g. event-driven, same is the creation at Its available be to time. have receiver and sender asynchronously,i.e. communicate receiver and Sender edge. didactic on emphasis with base knowledge a the is of Documentation part lost. get preservation, simply of would kind knowledge that Without knowledge. learning and informa- personal preserves tion good and a captures of it because constituent environment a sure for is It eeoesadtelnt fpoet increases. projects of of number length the the as and has especially developers relearned, often or knowledge learned that be and to software, develop- change enables to what important ers is is knowledge it that now For remember detail. to in aspects more some to 2011). (Prause, experimentation delet tests Automated part. local limits a to subsystems needs between knowledge cou- interfaces loose clean a keeping and by pling complexity information Reducing of source to. a turn others to giving place, knowl- one architectural in unifies edge The similarly architect project. an the of of knowledge role the other each develop- teach lets ers programming, pair e.g. developers, tween communication. of forms other against off traded has maintaining value be didactic to and high creating a with of documentation cost (cf. classical The information valuable 2006). preserve Prause, and carry communication also of traces can environ- as learning emails example, personal For good ment. a of constituent only asyn- is It communication. broadcast and 1994). event-driven, chronous, Dictionary, proof (Oxford or something evidence about, of information texts, as books, provided forms, papers, etc. of set the is Documentation Documentation – 11 Definition hs set r uteape fwa a epde- help can what of examples just are aspects These be- communication frequent and direct Emphasizing the not is sense classical the in documentation But is communication of form special A I1—Itouto oSfwr rcs Artifacts Process Software to Introduction — II-1 documentation . 04(06,tefraiaino ciiist ecarried be to activities of formalization the (2006), 1074 ait fdfeetatvte r are u hna when out carried are activities different of variety A The rjc b ihtersucsta en norganiza- an define that resources the with during (b) performed project be a to activities of sequence ecutable process cycle process Software – 12 Definition process: software the called is development during Std. out IEEE with accordance In developed. is software document artifacts process Software 2.1.3 h otaepoeslk u reports. bug like of process not activities software are different the and the software, of the outputs of planned form necessarily final neces- the not of are part sarily they meaning medium, intermediate machine-readable possibly or in are human- tangible a are in not, exist or they either that base are knowledge they project that the in of part atomic are They superseded. be is a predecessor Instead, its created. supersedes be. that cannot revision artifact artifact an new changed, although the be that is can Note activity. file file the a modified by the output is of that revision artifact new should The software the behave. how information the knowledge project replacing piece the by a updating to thereby change code a source by with of desired reacts developer not The is that users. way its be- in behaves request software change the incoming an cause of re- form developer in a input example, ceives For whole project. the the of in information knowledge process of software pieces more contain or that one artifacts is into output input The of form output. some an transform tools their devel- and software opers activities, these of each During activities. project. the development by the followed of approach implementation specific the is process software The environment. project software tion’s otaepoesatfcsaeucagal u can but unchangeable are artifacts process Software different of dozens spans usually project software A otaeProcess Software rae knowledge created steagetto f()acoe ex- chosen a (a) of augmentation the is ) sotfor (short otaepoetlife project software 17

II — PROBLEM SCOPE:ONTHE QUALITYOF SOFTWARE PROCESS ARTIFACTS , ex- RTIFACTS requirements process A ROCESS P During the The major strength of wikis is as all- OFTWARE S Given the manifoldness of software processes, their management, software design, reference andformation, configurations, setup specifications in- and installation guidelines (Majchrzak et al., 2006).or A article single is wiki considered page a software process artifact. When is impossible to compile ansoftware exhaustive process list artifacts. of Instead, kinds this of three section major picks types out of softwareples process that artifacts reappear as exam- throughout this work: requirements, research is more on the latter ones. Requirements pectations about a future software system are collected dardized way of how requirements are managed. The for documenting a requirement, itsnale origin, and (i.e. the ratio- a justification)Robertson, that 1999). led Figure to 2.2 it depictsment a captured (Robertson on Volere and require- paper. Itin is a a human-readable software medium. process artifact Wiki articles round documentation tools forery software developer projects. can Ev- easily contribute to documentation engineering (e.g. Decker etapplication al., 2007). include issue Other tracking, areas of quality & process Figure 2.2: Volere requirement software process artifact good practice (Richardson and Gwaltney, 2005).ever, a How- source code fragmenttween that two is developers, is sent a viaCoordination software email and process be- communication artifact, activities too. continuously create software process artifacts thatpieces of contain information. valuable different outputs, and the diversity of available media, it (IEEE Std. 1074, 2006). Projects typically define a stan- (Louridas, 2006). Wikis are also used for requirements wiki articles, and source code. However, the focus of Volere requirements template, for instance, is a standard pri- UALITYOF ones Q NTHE . secondary :O extracts structure . 2 viewer COPE of the knowledge base, S views are containers for the infor- viewer ROBLEM II — P ). For example, Doxygen www.doxygen.org/ views software process artifacts), or are See 2 Often one activity produces more than one kind of Software process artifacts are documentation, with If not stated otherwise, this work deals with primary software process artifact. Thecutable output Code”, of for “Create example,cutable Exe- can binaries be and source databases. code,code exe- can And take still various even forms. source a Storing revision all control source system code testifies in professionalism and is activities in 17 activity groups thatform can the be software combined process. to diverse The activities spectrum like “Perform covers Estimations”, such “Conduct Feasibility Studies”, “Prioritize and IntegrateRequirements”, Software or “Validate the Training Program”. medium. An artifact isstored in a a piece supporting of medium. project knowledge, 2.1.3.1 Types of Software Process Artifacts IEEE Std. 1074 (2006) identifies a total of 69 distinct and comments from object-oriented sourcejor code programming of languages ma- to create documentationoff-line like reference manuals. It is a each artifact being a document.this According means to that ISO an artifact 9000, is information in its supporting information to the knowledge base.tomatic In transformation some might cases, make au- informationeasily consumable more for humans. A tool that achieves this but it still only createsfacts secondary ( software process arti- software process artifacts onlyfacts because are regarded all as other redundant.the An arti- machine-executable example software would obtained be fromcode source through the process oftransformation compilation. of The existing automatic information does not add new and not necessarily planned. Softwareare process either artifacts created manually bymary software developers ( derived automatically from primary onesondary by artifacts a are tool. also Sec- called Definition 13 – Software process artifact mation that builds the projectunchangeable, knowledge atomic, base. tangible, They possibly are intermediate, Software process artifacts would be considered as a 18

II — PROBLEM SCOPE:ONTHE QUALITYOF SOFTWARE PROCESS ARTIFACTS etrudrtn httecmue ildo. will computer the what understand better — ehmn ihorcgiieaiiisbiglimited being abilities cognitive our with — humans we h pca oeaddaimo orecd utf a justify code source of dualism and role special The ... orecd w ae fteultimate the of faces two — code Source 2.1.3.2 documentation. is code Source between communication machine. the and than man more is purpose The code source 2011). of Starke, and (Hruschka alone enough code not source is software compilable that, a said of Having project. artifacts most process the software constitutes suffice code fundamental Source not compilers. do to they input but as important, are source Concepts with code. deal explicitly will project software Every code Source further research. for this subject in as investigation them in qualify applicability process all-round software the wikis’ The software another artifact. is revision process new its changes, article an htaeinrdb h oplrwe tbid h software. the builds it when compiler the by ignored are that Markup Text ( Hyper like Language ones descriptive or Golog, hybrids like Haskell, like languages object- functional to Java, C oriented imperative from languages: programming of time a at information of amount small a only handling to way This 2010). (Grier, workings too inner know their to about having much without problems to algorith- solutions model mic to help languages Programming (Spinellis, 2010). details code implementation source unneeded from masking clearly by more that appear abstractions design just the are make names respective their macros, and comments, etc. objects, types, source data of Functions, needs lines code. human-readable computer the the with that intertwined instructions are methods The and functions named. how care are parser a one does language nor programming uses, what computer a compile for will program matter binary that same code infinitely exactly the source still into but writing less, of are ways there many And com- same-purpose program. a writing puter of ways of number infinite code source of documentation Inherent artifacts. process software ultimate these at look closer optrsinehsdvlpda nrdbenumber incredible an developed has science Computer 3 sapof osdrabtaycmet de otesuc code source the to added comments arbitrary consider proof, a As otaepoesartifact process software HTML orecode Source ;aleitjs omk tese o us for easier it make to just exist all ); stebupito software. of blueprint the is I1—Itouto oSfwr rcs Artifacts Process Software to Introduction — II-1 3 ete osit does Neither . hr san is There aegoslre.Suc oe—oiial medium a originally — code Source larger. grows ware Gm n omr 00,o e ie hti Java a in that times ten or 2000), Sommer, and (Gumm Dbce,20;Pas ta. 2010b). al., et Prause 2009; (Dubochet, ytci sugar Syntactic neetdocumentation Inherent eoeamdu fcmuiainaoghumans among communication has of — medium machine a and become man between communication of soft- a when especially prefer, people it what length, obvious its is Despite formatting. standard with program string byte 10 a ( guage, bytes two add takes to numbers computer two a Telling 1992). (Knuth, fellows their 1964). (Landin, code comprehensible source and in palatable expression more an makes that etc.) constants, sugar Syntactic – 14 Definition 2010b). al., et (Prause documentation mostly is IDE an embedded other without or Even comments computers. to talk to humans as akrud ainl n otx nomto are information context and be- rationale limited background, are cause documentation insufficient inherent and and code Source limited is documentation Inherent 2005). (Raskin, reality in fulfilled partly be code only still source can And self-explanatory free. completely for for come desire not the does code source of ability an in it do to understandable skills soft hard functionality, requires implement that to art skills an is code source understandable the make to code documentation source further need not does that understandable. better it make to structuring, order in clear etc. names, variable an sugar, of syntactic use like features good inherent making language’s programming by appropriate code source documents It documentation Inherent – 15 Definition 1996): Klein, and (Didrich is documented code inherently source called Such documentation. further be requiring should code source that orecd switnt era ydvlpr and developers by read be to written is code Source osyi prn,i spsil ocet orecode source create to possible is it upfront, it say To development software in desire general a is There itself ssm oain(..acmet named comment, a (e.g. notation some is a,admtvto eas understand- because motivation and way, aiyudrtnal.Btwriting But understandable. easily ytci sugar syntactic spr ftesuc oeitself. code source the of part is D X AX BX, ADD self-explanatory 3D8 03 orecd enin seen code source , nisonlan- own its in ) nAssembler in ..not i.e. , 19

II — PROBLEM SCOPE:ONTHE QUALITYOF SOFTWARE PROCESS ARTIFACTS is software process ) software devel- XP software process artifacts RTIFACTS internal documentation A — as opposed to internal docu- — as opposed to external docu- ROCESS P . OFTWARE S Internal documentation, instead, targets developers of ment in a software projectever, (de the Souza et Extreme al., Programming 2006).opment ( How- methodology demands that sourcenot code require should other documentation so that more time can ondary software process artifactsa whenever they didactic have advantage overat others. software And process the artifactsdidactic that term goal were hints only. created In situations foris where the to the be didactic emphasized, value theused term instead of the more technicalartifacts term Definition 17 – Internal documentation mentation — is internal toIt the encompasses organization all and information project. thatdevelopment, originated and from is the bound forers developers and (Hunt maintain- and Thomas, 2003).the It sum is of largely equivalent all to softwarerelevance to process developers artifacts and with maintainers. didactic 2.1.4.1 Internal documentation controversy Currently, there is no agreement about howkind much of or documentation what is useful.feel Software maintainers source code is the most important thing to docu- targets an audience outside oftion, the i.e. developing organiza- the users. It is not relevant in this research. Definition 16 – External documentation mentation — is published forsupports users them of in the learning software. howand to It Thomas, use 2003). the software (Hunt the software. The definitiondifferent given connotation here from has the acause common of slightly definition its relation be- to thethat term was defined earlier. Allcan software be process considered artifacts as documentationinformation because in they carry the project’s knowledgeinformation base. content From point an of view, bothor terms less are equivalent. more However, documentation emphasizes didactic value. Internal documentation may prefer sec- Internal documentation External documentation in- UALITYOF Q and NTHE external :O COPE S ROBLEM II — P of a few minutes can already cost the de- 4 tion, and how to best make it documentation. External documentation primarily The time between the execution of tasks that build on top of each 4 Documentation is often divided into Instead, such mental models need to be captured in 20 eventually need documentation (Raskin, 2005; Hruschka and Starke, 2011; Tilley, 1993). ternal other can be regarded as an interruption in one joint bigger task. Source code is thesystem. Its precise inherent description documentation explains ofrationale, facts it a but has no details software but notone-person, the throw-away-prototype big picture. project, In itsufficient a may to small, write be source codeunderstood that by will someone never again. need But to any be other project will modified (Raskin, 2005). 2.1.4 Source code and other documenta- edge does not get lost.a Through developer further communicates documentation, (possibly tobackground, his context, future trial-and-error self) and side-effectformation in- while it is presentcomments in embedded his in mind. source For code, example, sign or architecture documents de- belong intomation this is category. extremely valuable This if infor- ever the code has to be else, but a collection ofSo organic the and above social mental documents. representationsdevelopers are in not the available to first placechanges if themselves. they have not made the the project’s knowledge base promptly so that knowl- him in re-activating the mental representationscame that inactive but be- that are neededmost for representations a have task. faded, After too. weeks, representations Once the have mental faded, reading themcode from is source hard to impossible.sacred tome Also, carved source into code is stone not and a touched by no one making the change startsmind to and fade will from be permanently theterruptions lost developers if not preserved: In- 2004, p. 229). After several days of interruption, details memories are still available to the developer that help missing. A study by Parnincesses (2010) of about developers cognitive while pro- they arethe problem: programming details Unfortunately, shortly after bringingcode source into being, the valuable information required for veloper his ensemble of well-crafted thoughts (Graham, (e.g. names of identifiers) have faded. Yet traces of

II — PROBLEM SCOPE:ONTHE QUALITYOF SOFTWARE PROCESS ARTIFACTS ileetal ahrpluetepoet ihrbecause either project; the pollute rather eventually will knowledge impart to is documentation of purpose the As hr sn nvra ouetto ehd n,in- and, method, documentation universal no is There utse l 18)d o ovyardcini required in reduction a convey not do (1988) al. Yet et Curtis projects. distributed communi- in reduce especially overhead; to cation and track, on development keep necessary to as documentation consider They neccessary. of object position strictly the (2003) against Rosenberg and Stephens documen- code tation. source owe brilliant developers colleagues that (future) says their (2010) op- Spinellis As this, undesirable. up to them cover posed makes to which there code, often source are they bad that but actu- useful are very comments ally that adds information (1999) duplicated in Fowler keep results consistent”. to it trying as of foolish”, “nightmare [...] “documenting a level calls code (2009) the at Selic software again, Then implemen- speeding sees to tation. way instead, a (2005), as Raskin documentation internal developing. on spent be nta,i netgtstepolmo h nweg is knowledge why of problem knowledge. the investigates its it document Instead, should organization an how (Papadi- way right the in 2010). moulis, things necessary right is the it Therefore document infor- find. to of to piece difficult needed too the is because mation or it, in information contained wrong is and obsolete up-to-date, inaccurate, documentation because keeping or for costs high the of which documentation of amounts unmaintainable to lead and 2002). (Forward Lethbridge, maintainers later conveyed and is developers information other necessary to the appreci- as be long still ability. as and this ated, incomplete on be based instance, judged for be may, to It has it audience, way. its only in and one the be cannot there that seems it deed, method? documentation optimal The 2.1.4.2 document knowledge. to their manage how and each on software of decide discretion to the project in or should is is organization software there it Instead, how Therefore, of documented. be way like. accepted look commonly should no it what and documentation is, what of understandings are different There many different. very are documentation regarding documentation. of because communication hsrsac osntwn opoli rprescribe or proclaim to want not does research This standards documentation overly this, to opposed As opinions and experiences the this, by evidenced As XP htdcmnainwsun- was documentation that I1—Itouto oSfwr rcs Artifacts Process Software to Introduction — II-1 ast piiedcmnain o eas hr sa is there because Not documentation. optimize to ways documenta- setting But documentation. appropriate with htaon fitra ouetto snee and needed is documentation internal of amount what The time. same the at knowledge that expand they while liv- developed nature like much very prototypes, with engineer- to than research scientific to akin more is which h prefix The .. umr fsfwr development software of Summary 2.1.5 it. they do because to but motivation it, a do have to them forces that prescription for look themselves better would much developers be if would It efficient time. and extra little only him consequently, costs artifacts, Doc- process software mind. his the in umenting already present he knowledge change, important a all makes has developer are a they When because way rigid. wrong too the be may standards tion it accompanying by and invest- documentation by inherent understand: in to ing easier code source making by code addressed be way. can agile cause an root in the how and managed, not iedvlpr oiaint netteronprecious own their invest to motivation a developers give this what at. not solving is aims it research And decided be fashion. never across-the-board can an this in Perhaps project. and the what for about good consensus no is there Yet code needed. source is beyond documentation More insufficient. is alone documentation inherent However, inherent documentation. with sugared contains commands it machine-readable because the is artifact code process source software them, ultimate Among code, the more. source many or are there articles but wiki requirements, are artifacts process software the Typical from base. artifacts knowledge process project’s software using them is to changes taught making for need developers that knowledge project the of knowledge the con- who relearn developers, and by learn not made tinuously do are They evolution free. by for required come changes But Darwinian through code evolution. genetic their and organisms ing experimentation through being to comes Software ing. production) its not (and development its in products. lies industrial cost that traditional Its technology with new common a in few is has It knowledge, complex. encoded condensed and is interlinked, Software not. is it But htmtesi h ifrnebtenwiigsource writing between difference the is matters What nta,aieapoce odcmnainne to need documentation to approaches agile Instead, uc n dirty and quick skoldecreation knowledge as soft mle htsfwr ses ochange. to easy is software that implies n en iet elwdevelopers fellow to nice being and 21

II — PROBLEM SCOPE:ONTHE QUALITYOF SOFTWARE PROCESS ARTIFACTS RTIFACTS A ROCESS P and the cost of maintenance OFTWARE S Almost anything can be built with software. It is only In other engineering disciplines, detailed construction For example, Bruegge and Dutoit (2000) compare a virtual construct that rarelyhardware. hits the limits But of a modern large program is too complex for a to show general feasibility,started and from then scratch a to build differentdetailed the team initial industrial construction version. plans The are refinedIn and the re-used. biological metaphor, nature alsoopment continues from devel- existing designs. Theby genetic individuals code representing carried their design mustotherwise have evolution value, would not worksense. in the Darwinian plans are only totallypletely discarded, failed. In when software a engineering, the design detailed plans com- product is successful. Spolsky (2004)ing considers source discard- code as onenization of can the make. biggest mistakes an orga- 2.2.2 Protecting the value in source code, software company that had anthat internal redevelopment statistic would that said happen wheneration the of third developers took gen- over development.then Why that is organizations still it sometimes decide for rede- continuously refine a product aftertype using a the successful detailed proto- documentation.ing In disciplines all except engineer- software engineering,tion documenta- has been accepted asnecessity — (Selic, perhaps 2009). unloved — basic software development to the engineering ofsenger a aircraft. large pas- The first Boing 747 was rolledIt out underwent steady in improvements ofing its design several includ- major revisions.sible Such with success good is management only of pos- decisions construction (Bruegge plans and and Dutoit, 2000;Bruegge Drommert, and 2011). Dutoit stresssoftware that development the for same software process holds artifactsrequirements, true like designs in documents, source code,on. and so Similarly, blueprints and constructionprototype plans of of the the largest passenger airlinerbus ever A380) built were (Air- re-used inthat a the group later of product. researchers first It built is the prototype, not just velopment if it is such a bad decision? (i.e. source code) are sometimes discarded although the A colleague of mine had once worked for an internet 1968 and was since in production for more than 40 years. UALITYOF Q NTHE :O . The connection with COPE 5 S http://c2.com/cgi/wiki? , retrieved 2012-08-15 ROBLEM of software covers quality aspects II — P valuable output internal quality C2Wiki ¨ osch et al., 2007). 5 The difference between crafting and engineering is 22 that engineers take goodquired care during the of engineering their process.laborers knowledge Craftsmen use ac- the and engineer’s blueprints toaccording build to products their plans. At the same time, engineers do InternalSoftwareQuality Source code encodes the project’s essential knowledge. code. It is the mosthavior, precise and the description most of valuable software output. be- 2.2.1 Source code as the project’s most Definition 18 – Internal quality that are primarily visibleing to organization, e.g. members architects, of developers the and testers develop- even end in a total economic loss. This section discusses makes software more reliable, simpler, easierunderstand, to maintain, and extend, change developer knowledge makes internal documentation the most essential element of internal quality. 2.2 The Case for Internal Quality Source code contains a lotthe of project. detailed knowledge It from is perhapssoftware the project most generates. valuable But asset source that code a is burdened tial for organizations engaged in the development of soft- later in this chapter.cusses Before what that, happens if the software nextaged. knowledge section is dis- not man- time into carefully editing documentation fordevelopers. their fellow The reason why documentationstill exist standards is that developerstion do because not of like this to discrepancy. do This documenta- will be discussed The (Pl All that is needed to produce the software is its source ware. If maintenance costs are too much, the project may why poor internal quality is a problem. Internal quality with maintenance costs. This reduces economic poten-

II — PROBLEM SCOPE:ONTHE QUALITYOF SOFTWARE PROCESS ARTIFACTS aemitnnecnit fcretv n enhancing and corrective of consists maintenance ware has so and constantly, change requirements Additionally, maintenance. de- the at starts that processes cycle the life of primary one as five maintenance software defines 14764 ISO maintenance software of definition A 2.2.2.1 so discarded cost. still maintenance is (Spolsky, is code it often source write why to reason than The code 2004). source read it to And harder engineers. is by have re-understood will continuously blueprints be software’s to The becomes end. and the ages in quickly useless it Otherwise, software. the phase. research initial the for contracted if especially, only team; is team original this an build to available to be time will more than takes and mind, developer’s single hssmyb i uz.Nwdy,dvlpetof development Nowadays, fuzzy. bit be may maintenance phases and development between demarcation the maintenance. perfective and adaptive of enhancements form from in result cases. costs some maintenance in the of 90% Most over for account maintenance can system evolution to and devoted lifecycle software costs then, total Since the of costs. 75% maintenance to literature 49% on from spent in efforts ranged surveyed the 1970-1990’s (2007) the in al. that et Kozlov projects. sub- major four of its areas into divides further that maintenance Soft- capability. modified or new unforeseen introduces for it need when the or opera- errors, an detects when environment needed tional is Maintenance includ- code. base, source knowledge ing project the is of maintenance modification of the process The while integrity. product its software preserving existing Its an retirement. modify its to with is objective ends and software, the of livery n esnfricesdmitnneefrsi that is efforts maintenance increased for reason One software of factor cost considerable a is Maintenance • • • • a fetra aspects external of internal of enhancements implement to — perfective chang- a in and usable environment, software ing the keep to — adaptive operation, in that failed faults yet latent not correct have and detect to — preventive problems, discovered correct to — corrective I2—TeCs o nenlQuality Internal for Case The — II-2 ehia debt Technical ae aneac a led aepaedrn devel- during place take already may Maintenance ware. eomn n eiey aneac shnedefined hence is Maintenance delivery. and velopment petieain naiedvlpetmethodologies. development agile in iterations opment soft- the of integrity preserving while software artifacts further process with base knowledge project existing an Software Maintenance – 19 Definition work: this in broader de- between demarcation the complicate there- further methodologies fore These finished. officially yet is development not although reduced delivered be with may prototype functionality functional A development classical methodologies. to comparison in are ground gaining that costs and methodologies agile maintenance development of rise in prototype-based the increase with coincides The chronologically also software. release first the the after of even on goes software long-lived et(e iue23frhsilsrto ftetodimen- two the of examples): illustration including his sions for 2.3 Figure (see debt both between gap the closing states. for cost the measures Technical quality. debt internal best their of and state artifacts, ideal hypothetical process software software’s a of state debt Technical – 20 Definition is debt technical (2010), al. on defined: et dependence Brown is by In It proposal quality. the process. internal software been of the counterpart since of the has areas but other code to source extended to only debt original referred The term tests). unit doing creating example, or (for documentation functionality new that activities implement development not on do down cutting by debt go into can developers speed, development improve porarily concept debt the introduced (1992) Cunningham redevelopment make can debt Technical 2.2.2.2 olr(09 dnie w iesoso technical of dimensions two identifies (2009) Fowler sawyt pe eeomn.I re otem- to order In development. speed to way a as necessary maintenance hrceie h a ewe h current the between gap the characterizes smdfiainadetninof extension and modification is technical 23

II — PROBLEM SCOPE:ONTHE QUALITYOF SOFTWARE PROCESS ARTIFACTS stay RTIFACTS A ROCESS P source code OFTWARE wants to buy a dinosaur. They want to have a S 7 except for John Hammond from Jurassic Park may be... 7 All the innovation and effort that nature put into di- Spolsky (2004) calls it a delusion to think that rewrit- into the first softwarecode, are recorded and only are in lost the when source the source code is discarded. is attributed to their failure to adapt to ation and changing innovation enin behavioral strategies. When their tion to new foods (Lloydseemingly et unbreakable al., predominance 2008). abruptly ended, Themaking dinosaurs’ room for mammals thatMammals had instantly waited filled in the a gap,plethora niche. and of diversified species. into a Today, they embossas the successful planet’s predators face in mosthad ecosystems. not If failed dinosaurs to adapt, they would have remainedextinction, software in- must evolve (Prause, 2011). nosaurs is lost.software-encoded knowledge Whoever should wants be to awarebody that sell no- research and mouse with a bright future,great which products. is easy to evolve into ing the software from scratchredevelopment. would be The any team better working than onhas the probably new no version advantage inold experience, mistakes instead will many be repeated but enriched with new ones. then the remaining knowledge isand unsalvageable. McClure Martin (1983) observed that itsoftware is professionals consensus that among “Inadequate documentation is a major contributor totenance the high and cost to of software the main- distaste for softwaretainers maintenance can only be effective whensary they knowledge have available. the Otherwise neces- theythan are any no others better (Tryggeseth, 1997). 2.2.2.3 Redevelopment is a total economic loss of Once more back to thequality of biological software metaphor: is External itsbeing fitness, used i.e. by its users. ability Internal todoes quality survive, software is evolvability, adapt e.g. toliked? new Without user fitness, needs? there isfor no Does evolvability. evolution But it and given no fitness, need onlyensure evolvability long-term can fitness. The extinction of dinosaurs vironment. Their races showed only few diversifica- vincible competitors for any other species. To avoid The detailed decisions and precise knowledge that went world changed, only two species attempted an adapta- work.” Indeed, research showed that even the best main- 6

UALITYOF Prudent Q NTHE :O COPE S how we should have done it.” and deal with the consequences.” “Now we know “We must ship now http://c2.com/cgi/wiki? , retrieved 2012-08-15 Deliberate ROBLEM Inadvertent II — P (on technical debt) is any effort that does not

incurred intentionally. reckless vs. prudent —curred whether because the the developers debt did wasand not in- know better, deliberate vs. inadvertent — whether the debt was time for design.” “What’s Layering?” “We don’t have Reckless C2Wiki 6 Sometimes, however, redevelopment seems to be with- Technical debt, however, leads to a danger when that • • 24 out alternative to developers when technicalresulting debt interest and payment the is too high.edge If is too much lost, knowl- important rationale and links are missing, InternalSoftwareQuality Definition 21 – Interest change the amount of technicalhave debt, had and to that be would spent, not if there was no technical debt. ization of developers and bringsstand-still. entire development A to developer describes a the following situation: but had been maintained, used,fixed, bug and fixed, fixed, and until fixed, the and low. defect Just count don’t was try astonishingly to add any new features though....” debt is not repaid promptly:because software All process the artifacts time areregarded that not-quite-right, as is is spent interest ontechnical the debt debt. in a If project, there it is leads too to much extreme special- Figure 2.3: The four2009) kinds of technical debt (Fowler, Interest “Conversely I have seen decades old code that was Bad,

II — PROBLEM SCOPE:ONTHE QUALITYOF SOFTWARE PROCESS ARTIFACTS odfrtetes(rshaadSak,21) Other 2011). Starke, and (Hruschka trees the for wood Lws 05.I oe eintrsotntt work to not out turns design novel a If 2005). (Lewis, hscue ot iia otedvlpetcsso the of costs development the to similar costs causes This sapeatoaymaueaantattleconomic total only a not against matters measure quality precautionary internal a Still, as lost. these of are quality parts or internal revised the to into have Investments may design dumped. the of parts expected, as prototype due the by discarded revealed be problems insurmountable to to have will prototypes Sometimes mat- quality internal where and Why 2.2.3 market. to and time software; of earlier loss the huge in a put of effort loss and economic knowledge total the a is Redevelopment version. first nsuc oei n tpcoe ottleooi loss. economic total to closer step one is code source encoded knowledge in valuable the that source Every means the degradation 1997). of (Tryggeseth, quality degrade then inevitably not, will is code it If is them. knowledge to necessary available the if changes quality high make respectively. disadvantage competitive a or in advantage competitive resulting a the not in reduces resulting this interest market, end, to the avoiding time In by 2005). (Raskin, implementation payments documen- speeds use thorough tation The development: for needed quality. effectiveness, product cost and to speed, key development the is quality quality internal produce knowledge through to of re-use way enabling cheapest Consequently, arti- and products. existing best of the re-use as considers debt. facts (1993) technical Griss on fact, payments In interest and are again These re-learned be again. to have designs failed or prob- known lems, to solutions requirements, like the knowledge see to not may one code, to source needs from architecture pro- understood software costly be the a if in example, recovered For be cess. to has is or it forever, minds, lost developers either the from fades it (adequately) When not is preserved. knowledge that pro- means software artifacts of cess non-existence or quality internal Poor dura- cost, on quality internal of effects The 2.2.3.1 process. negatively development debt the technical affect of amounts small Already loss. ial,go ananr a nyuetersiland skill their use only can maintainers good Finally, time reduces also reuse benefit, cost the Besides ters inadquality and tion I2—TeCs o nenlQuality Internal for Case The — II-2 hl-e emta a oeo h nweg fthe of knowledge the of none has that team whole-new into development from changes system software a when knowledge without enter developers new and them, with e de msheltni uznf Nutzen in schnellsten am Ideen wer be- area an in operate organizations research Applied to transferred actually is knowledge that ensure to way hscs saseilzto fteaoehn-vrof hand-over above the of specialization a is case This eta omncto sntcpue.S h receivers the So captured. not technical is of communication amount as the debt reduce communica- rarely verbal does the process that tion Note be. process must verbal extensive more the the the But is, for there lost. documentation be is fewer will there knowledge time more less the hand-over, the the bandwidth, higher pro- a verbal has the cess Although word-of-mouth documentation. original by through down the and handed of is work Knowledge the over team. take will team, a original case, this In delivered. devel- gets was and contractor software a the by oped when or phase, maintenance situation the problematic more Even software. the haveof they knowledge take leave that Developers person- in fluctuation nel. to subject are teams Development hand- the on quality internal of effects The 2.2.3.2 otaepooye,adrcre nsfwr process of software form artifacts. in in recorded is and domain prototypes, software software the in from research resulting knowledge applied the funda- to of opposed much As research, for mental time.] use shortest into the ideas in turns customers who his he is winner the [german: ist, “Sieger way: to this knowledge it transfer put to (2010) is Bullinger business industry. core Their prod- into prototypes ucts. transform to coop- closely industry hand with other erating the on hand, but one research, the out On carrying industry. and research academic tween more. some and problems same the all with knowledge prototypes on quality internal of effects The 2.2.3.3 Internal for. research. pays contract he to results indispensable the is reuse quality to able that be customer will the he to assure plausibly research must contracting organization any Therefore, effective 1997). no (Bailin, have him does He getting. never is customer he the that what is knows problem the contract, on oped debt. technical less no with up end diinly ntecs htsfwr a devel- was software that case the in Additionally, vro knowledge of over napidresearch applied in rKne umsetzt.” Kunden ur ¨ 25

II — PROBLEM SCOPE:ONTHE QUALITYOF SOFTWARE PROCESS ARTIFACTS RTIFACTS A ROCESS (Casey and Richardson, 2009; Zopf, P search teamness OFTWARE S Herbsleb and Moitra (2001) find that updating and The purpose of documentation is to communicate in- engineers are working hard, whichnovation makes important continuous for in- Europe. Researchtransferred results to must be industry and bequickly transformed (Bullinger, into 2010). products settings due to thecommunication higher (Liukkunen et cost al., and 2010).that lower Information is richness basically of there, ismaking fragmented it to difficult different to sites, et reunite al., it 2009). into one place (Boden revising documentation are as importantthe as first writing place. it in Written documentation,the instead, need lessens to have co-locatedberg, teams 2003). (Stephens and Furthermore, Rosen- Cataldo andshowed Nambiar that (2009) high software processmeans maturity to (which carefully also manage artifacts)ative mitigates the effects neg- of spatialteams, distribution. the interest So, rate in is distributed higher than in co-located ones. 2.2.3.5 The effects of internal quality in EU re- dustry with their cheap products.2008 only The made market this crisis situation of worse.had China, an for expected instance, industrial growth ofthe 8.5% crisis. even during At theparable moment in their quality products to are western not counterparts. com- But Asian Besides interpersonal problems, the exchangemation of (communication) infor- is generally more complicated coordination problems (Herbsleb andaggravated Grinter, by 1999a), distance-induced impairments oftion motiva- and 2009; Kiel, 2003). This leads to the effect that distributed formation to individuals involved in development (For- of information between developers byinto standardized channelizing forms it of communication.ity High software qual- process artifacts preservefrom more information getting lost, andsequently, are fewer easier one-on-one communication to issary, understand. neces- strongly Con- simplifying communicationand overhead. topologies While documentation iscommunication always among a project form members, of (Curtis et al., (Carmel and Agarwal, 2001). The result is extraordinary Asian states are putting a high pressure on western inwork takes much longer (Herbsleb et al., 2001). ward and Lethbridge, 2002). It facilitates the exchange 1988) gaps in it are more costly to fill in distributed UALITYOF Q NTHE :O COPE S ROBLEM II — P tributed development projects The plan of building one prototype first to later throw A prototype project generates a prototype (i.e. some- It is a common misconception in research that inter- 26 Poor internal quality negatively impactsdistributed collaboration projects. in So, technicalrelevant debt here. becomes In more such projects,nication there opportunities, are and fewer commu- the costis of higher. communication Vital informalleagues communication can between even col- fully break down (Grinter et al., 1999). away the whole program is dangerously2004). foolish (Spolsky, 2.2.3.4 The effects of internal quality on dis- it away, is not a good idea insource large-scale software code de- would bemental lost. code and It throw it isfine away to fine a refactor to week individual classes. later. writeexperiments These experi- It are prototypes is limited and in also time and scope. But throwing the customer paid for whileunderstanding not the getting source it code delivered. isredoing If similarly the expensive prototype as project, then thea original total project economic is loss. Dueof to knowledge the through high the probably hand-over, of aloss a total for loss economic the customer can occur earlier. that develops a new material forsupposed a to customer show is that not suchmake only material more exists of but it. how A to not customer only of wants contracted to development knowto that make something it works work. but how Knowledgethe that project was but generated which during is not preserved is something that competition. thing tangible that actually works)how and to the build it. knowledge Thefailed knowledge consists and of successful requirements, designs, andplans precise in construction form of source code. A research organization ble and improvable. For thismust to happen, be the preserved knowledge andengineers. be Only understandable then to are thesign they incrementally, later able and to to optimize eventually build theHaving the to de- product. re-create large portionsthey of blueprints are because incomprehensible, wastesand valuable time; resources both of which are important in today’s global nal quality has no placetypes. in the In world order of to softwaretions, proto- allow blueprints for of further prototypes development must itera- be reusable, extensi- velopment because most of the knowledge built into the

II — PROBLEM SCOPE:ONTHE QUALITYOF SOFTWARE PROCESS ARTIFACTS ad fpetg,wiepiaeetrrsswn opro- to want enterprises private while prestige, of wards important particularly is quality internal on focus a with teeeuieo h uoenUin ( Union) European the of executive (the udmna eerhadidsra engineering. between industrial as gap and research works the fundamental bridge and thereby realizable Prototypes is design expected. demon- certain to a pilots that run and strate prototypes create applied to of re- is role Fundamental research The concepts. proven cycle. deliver innovation must search the the of in phase well-usable are next that results deliver competitiveness, must European research and in German science, achieve of to domain order the matter No money make knowledge.] again from also must We knowledge. into money machen.” m Wir machen. etteritleta rpry(ou n ei,2001). Metiu, and (Kogut property intellectual their tect re- with results of their norms publish to The researchers urge code). science source (including secrecy inventions the of regarding opinions different industrial have Pro- and Work partners academic that (ICT are economy.” Reasons wider 2012) gramme, the by up taken which are services) and processes in- products, into (new them novations translate to and generate technologies to important breakthrough pro- critically is future it boost growth, To and chains. ductivity weak- value the European is in markets link to est and production from to arising research ideas basic from is transition technologies the enabling strong, key generally other and ICT in R&D pean goal. this by guided be project. to industrial has an or management into Quality projects project research research further a for turning be source for it like if results matter no engineering code, of re-use development permits in flexibility and grants This projects. such for quality of level medium reached cheaply A whole. fail a rarely as projects research always large is research, failure in of involved risk the Although long-lasting projects. comparatively research these for flexibility important This very project changes. is market a to then direction reusable, its adapt are can results partial if (2009) fail- Tolido that of But notes risk misconceptions. high theoretical a to as due well ure as projects research of nature the by wanted re- is re- project sults distributed of exploitation in Subsequent partners consortia. search industrial and academic cooperation the of by signified is Programmes Framework oee,teeaemjrpolm:“hltEuro- “Whilst problems: major are there However, Commission European the by co-funded Research Wissen Geld aus nur nicht kann Gesellschaft “Eine 8 Hans-J r ulne nHri (2009) Horeis in Bullinger org ¨ 8 gra:Asceymyntol convert only not may society A [german: se u isnac idrGeld wieder auch Wissen aus ussen ¨ EC edfrfedmle nthe in lies freedom for need A . I3—A si nlsso nenlQuality Internal of Analysis As-is An — II-3 EC ne the under ) diinly eerhfcsso oet vrexcel- over novelty on focuses research Additionally, hl e eeomn em orh ndistributed in Fourth, team. development new whole hsscinmd h aefritra ult.I empha- It quality. internal for case the made section This hsntol assmntr ot u lodly the delays also but costs monetary causes only not This eomn n erdssuc oeqaiy Second, quality. code source degrades and velopment neeticessdvlpetcs,last lwrde- slower to leads cost, First, development interest: increases is interest problem the advantages, debt short-term technical has deliberate While market. economic tive software. of life the in factor cost main most, the matters today debt are technical main- which Software in value. processes, not its tenance does has case still worst quality a this internal to if occur, lead even But can debt loss. technical economic total case, extreme technical the with burdened In be debt. can software value in its value that primary but the projects is code source that sized inter- of importance the of Summary 2.2.4 to development. adds distributed further of This problems 2004). the (Graham, design a of lence td n esnlexperiences. personal and study ( TechnologicalDevelopment and Research for software Programme Framework with in surveys and experts on popular engineering reports and literature culture, for science developer be looks computer section to in This tends evidence projects. quality many by in internal easily problematic However, the more for learned developers. be advantages, may long-term knowledge project’s has quality Internal Internal of Analysis As-is An 2.3 global in critical be can which competition. launch, market a of date economically. reused be cannot results research quality that artifact are process software the poor Fifth, of effects negative complicated. be- more higher is are communication debt cause technical on a rates to interest transferred settings, be to have may day one early that the feature-reduced, a product, to is contractor prototype a a Third, from handed customer. is or software developers, the between when lost over ar- get process to software prone is in tifacts captured not is that knowledge nenlqaiyi hrfr motn ntecompeti- the in important therefore is quality Internal Quality a quality nal FP6 ,adcnldswt rjc case project a with concludes and ), EU rjcso h h Sixth the the of projects 27

II — PROBLEM SCOPE:ONTHE QUALITYOF SOFTWARE PROCESS ARTIFACTS RTIFACTS A ROCESS P for the decay of source code due to not Time pressure has a negative effect on docu- OFTWARE S Nordberg (2003) considers poor documentation as a representation is difficult, especially ifand e.g. its source code evolution is notis often properly the documented, case”. “which Also,time many to projects document do the not knowledge take gained the in the project problems in documentation. 2.3.1.3 Reasons for documentation problems Zopf (2009) made the observationcipline that “engineering is dis- not yetopment basic organizations”. knowledge Singer in andreport Schneider software anecdotally (2012a) devel- of industrial projectsers where omit develop- writing test cases,by or emailing perform project revision directories. control Casey and Richardson change knowledge are missing. Butalso Boden found et that al. in (2009) distributed development, different de- each others’ contributions to documentationeach but accuse other of notprojects, doing Kiel it. (2003) observed Particularly thatdards not in can following distributed almost stan- be liketwo a reasons point found of in pride. literature Butand seem the economic to major rationality be of time developers. pressure Time pressure as externalproblems source of documentation of Mud pattern cleaning up. Bertram et al.developers (2010) to report contribute of to reluctance of bug databaseskind although of this documentation is considerededge pivotal management. in Seibel knowl- (2009) conducted interviews ported to think that peopleRead who the say code.” “What are does unprofessional itshows because what do? source it code does, notrecites what Edsger it Dijkstra’s is words supposed that toat one do; English who will is and never no be good a good programmer. Jamie Za- more”. And Douglas Crockfordreads reports source code that written when by he by other “cleaning developers he it. starts I’ll throwfixing it it.” in a text editor and I’ll start typical symptom in software development, whenare there no clear assignments of ownership. Pigoski and velopment teams sometimes do not honor or recognize (Casey and Richardson, 2009).(1983) attribute Martin the typical and high McClure cost of maintenance to (2009) argue that incentives that encourage staff to ex- with several famous developers: Joe Armstrong is re- winsky said that “I always wish people would comment April (2004) report that comprehension in text-oriented UALITYOF Ball Q ” stage, NTHE :O Slow Death COPE S ROBLEM Bob’s Theory of Software Redevel- is a term that describes the phe- II — P tion problems problems ”, culminating in the eighth “ software crisis 28 maintenance is only poorlymedium understood; and small moreover, projects, in there istation. little to Laramee no (2010) documen- reportsstages from model coined his “ experience a opment ment anew. Foote and Yoder (1997) derived the Research literature is fullproblem in of different opinions levels of describinginternal detail: the Raskin documentation (2005) “one calls of the most overlooked and Lethbridge (2002) recognize thatumentation while internal is doc- abundant in large projects, its role and and find that oneactually of gotten the worse, is biggest the problems, quality of which documentation. has 2.3.1.2 Individual statements about documenta- Dekleva (1992) lists the lack oron quality rank of three documentation of major problems.observes Kajko-Mattsson that (2001) documentation requirements are ratherin low companies, implying that withoutlations, appropriate documentation regu- activities are carriedKrogstie out and rarely. Sølvberg (1994) repeateral the earlier results empirical of studies sev- on software maintenance, In a study covering almost 500zations, data Lientz processing and organi- Swanson (1981)mentation find is that considered poor as docu- one of the biggest problems. internal quality are soment pervasive that in the fact software itself develop- rarely requires scientific proof. inhibit internal quality are developermanagement personalities lapses. and 2.3.1.1 Empirical studies about documentation scope. While no singlesponsible reason for the can crisis, be a major madelack reason solely is of the re- communication omnipresent and documentationdevelopment in (Kraut software and Streeter, 1995).analyzes the This status section quo ofquality communication in and literature. internal In fact, it seems that problems with 2.3.1 The software crisis in literature nomenon in software development to not deliver soft- The analysis of literature finds that the two factors that The where generations of developers quit or restart develop- ways” to increase efficiency in development. Forward ware in time, on budget, or in the expected quality and

II — PROBLEM SCOPE:ONTHE QUALITYOF SOFTWARE PROCESS ARTIFACTS ihu aigtomc bu hi em ei (2009) Selic team. their about much too caring without 20)rao htifrainecag hog “a through information-exchange that reason (2007) put developers that true is it that down narrow (2003) the of outside created “information if that find (1988) in documentation insufficient for reason one see (2011) on eeoestn ocet ouetto prob- documentation create to tend developers young h troyia esnlt f“ of personality stereotypical the source problems internal documentation as of analysis cost-benefit Individual (1988). al. et Curtis is by pressures mentioned project also to vulnerability This Leth- 2002). and bridge, (Forward work from their them documenting prevent adequately constraints scheduling timing, and often documentation, budget of importance the be- in doc- developers lieve poor while Even for reason 1999). one (Connell, be umentation can of this inside then the software, with their than deadlines with concerned short-sightedly more is management When (Kajko- 2001). standards low Mattsson, are there if particularly mentation, ofim hsve fyugpoesoaswod not do who professionals (1997) young Humphrey of view this developers. confirms young of “cowboy model” the documenting role not calls (2002) Boehm lems: team. whole the for to benefits experience the the see lack to they individuals if for especially rational contribute; only not is It benefit individual”. no the provided for and costs induced database Cress shared and Kimmerle road. the down it way long because too argument a convincing is a not is better maintenance that bequeaths (1987) personnel Leonard-Barton process advocate. software and and timely managers as as completely documentation updating into effort less al. et Lethbridge developers, After with emphasis.” interviews no conducting needs and known well among is documentation developers to “resistance (2001) the Moitra that and ascertain and Herbsleb tools, lost.” the be will using information around ways find entered, will manually developers be must environment tools software al. et Curtis tasks. other with code on source pressing when low-quality behind leaving code-and-fix do then too, programming, experts, inexperi- but time, to programmers time they’re enced from know that in witnessed we persist and we when poor?” do has practices “Why documentation programming asks poor (2009) writing Wray them, value. To few doing avoid altogether. to trying their it often for documentation, famous of are dislike developers software that notices problems implementation through way their fight them ti hrfr o upiigt e hti particular in that see to surprising not therefore is It rshaadStarke and Hruschka oeheroes code I3—A si nlsso nenlQuality Internal of Analysis As-is An — II-3 htlets that ” ihtersuc code. explanations source helpful their include with to start developers when iw ihsfwr niern xet oiqieon inquire to experts engineering software with views 2.5). Figure (see years nwtevleo ouetn.Siels(00 sees (2010) Spinellis documenting. of value the know from code source http://abstrusegoose.com/432 understanding of Problems 2.4: Figure rueadDri 21)cnutdsrcue inter- structured conducted (2012) Durdik and Prause engineer- Software with Interviews 2.3.3 since habits bad programmers’ of cartoons featured that is culture developer A of resource software. famous documented poorly or undocumented in lost inaccurate”. be or to non-existent known is documentation organi- internal most your “like zations, Papadi- that, presumes need. openly in (2010) when moulis have to wants everybody to contribute that to but wants nobody that funding government Internet. the on culture developer scientific problems less quality in internal and evi- documentation recites of section this dence section, previous the to Similar popular in problems the of Evidence 2.3.2 professionalism, early and maturing of indicator an as it iue24boce h su fdvlpr getting developers of issue the broaches 2.4 Figure to documentation (external) compares (2007) Oram n experts ing culture developer ekadPoke and Geek 29

II — PROBLEM SCOPE:ONTHE QUALITYOF SOFTWARE PROCESS ARTIFACTS RTIFACTS A ROCESS P OFTWARE cartoons (Widder, 2012) S UALITYOF Q Geek and Poke NTHE :O COPE S Figure 2.5: Excerpts of ROBLEM II — P 30

II — PROBLEM SCOPE:ONTHE QUALITYOF SOFTWARE PROCESS ARTIFACTS eeeprsitrse ndvlpetadmanagement. and development in interested experts were of quality internal the about concerned less even were hyswtems rbeswt ouetto and documentation with problems most the saw They documentation seeing experts of Percentage 2.1: Table tcaesmaie nTbe2.1. Table in summarized problem- are as atic documentation and experts code of source considering percentages The problems. documentation mn itiue eeoes ih emr important more be communication might of developers, medium distributed a among internal as The code source problems. of quality code source to sensitive more code. source of quality internal the spectrum the of end the On documentation. prob- with of awareness lems broad was and there design, Yet requirements, implementation. phases like process early software the the in of interested were experts code by source noted the mostly with Problems (17%). code source and documentation, with problems few saw engineering alike general. academia in and documentation with industry problems both noted from Experts problem- as recognized atic. still but concern, pressing most 2012) Durdik, and (Prause background by problems xrm Programming Extreme dev. source Open Scrum > 10 5 < Theory Management Development SDLC Late cycle life development SDLC software Early in interest of area by Industry Academia All noe oresfwr eeomn,eprswere experts development, software source open In software of theory the in interested are who Experts the as seen not is code source of quality internal The − 20 5 − 10 years 20 years years years yaiepoesexperience process agile by ywr perspective work by ywr experience work by orecode Source I3—A si nlsso nenlQuality Internal of Analysis As-is An — II-3 50% 56% 50% 43% 33% 60% 43% 17% 57% 56% 47% 67% 40% 48% 47% 100% Doc. 40% 71% 82% 88% 66% 78% 74% 76% 70% 68% 74% 57% 72% 65% ihdcmnainpolm.A xlnto could explanation An problems. documentation with 21) aao spce paani eto 7.1.6.1. Section in again up picked is Karakos (2012). icdta h rbesaentseict gl projects. agile to specific not are problems the that vinced endqaiygas(5) io esn o poor for reasons Minor (35%). goals lacking quality and (41%), defined benefit personal limited each), only (46% having documentation time, considering lacking properly, explicitly it not do or to how knowing not (49% agree), documentation about caring not developers from therefore and problems. jobs, less their perceive in advancing implementation when from away activities farther move experts that be problems to insensitivity and experience work a between was there experience, ( on moderate based results the down Break- ing problems. reveal documentation which towards opinions, sensitivity experts’ low the in reflected clearly which is design, architectural above implementation rapid programming. pair different by e.g. spread means, is knowledge because documenta- problems less tion experience might methodology ment the following developers Similarly, here. eto ... o ecito fkycharacteristics. key of description a for 7.1.6.1 Section a as and research. analysis the problem of context for development serves it section, this In Zimmermann and Prause see study the For on context. details important more most the developed was the Karakos on concept, influence additional Although had 2010). projects (Erdogmus, other inception context its initial to is leads specific that it very as a idea with engineering intertwined software origi- always new the a know of to context important nal and therefore objectives is It their in execution. greatly research. vary this projects of Software context primary the was project Karakos study Karakos case project EU Karakos The 2.3.4 con- were (88%) experts interviewed the of most ment, requirements. in of changes importance frequent the and professional of documentation, of awareness the lacks of oneself), and management express well, write (inability to dissonance cognitive and prototyping linguistic good fast approach, writing a of follow projects goal (19%), the documentation with conflict goals are other design that architectural neglected and documentation h xet esndta h rbesmgtcome might problems the that reasoned experts The hl h td a bu gl otaedevelop- software agile about was study the While 9 suoy o otaepoetue nteeauto.See evaluation. the in used project software a for pseudonym A 9 a rjc odce ne the under conducted project a was r 0 = . 34 infiatat significant , XP < p sfmu o putting for famous is 0 . 05 XP correlation ) FP6 develop- The . 31

II — PROBLEM SCOPE:ONTHE QUALITYOF SOFTWARE PROCESS ARTIFACTS ¨ ost, 2009). . Informal EC RTIFACTS A ROCESS P ¨ ost, 2009). In fact, a case study does not need OFTWARE S Experimentation in software engineering has shown The researchers were involved with the project for its The main means of collecting data were interviews, from a single case is possible (Runeson and H to randomly select the casehow (Flyvbjerg, Karakos 2006). was selected This as is study subject. Still, per- distributed processes were observed during projectings, meet- individual work situations andas cooperative well tasks, as projectinterviews were reviews conducted held repeatedly by withticipants various the par- and stakeholders. Moreover, researchersintegrated were into the executioncesses of without various interruption, software and continuously pro- email, monitored phone conferences, chats anddocumentation, protocols, project and internal documentsconfidential (work papers, reports, white board sketches). The soft- documented over the full durationten of notes the with project supporting in material writ- the attached. collected Afterwards, material wasaddition clustered to into this, lessons findings. learnedpartners were In each gathered year. from the that there are many factorssoftware that engineering impact activity the (Shull outcome et of al.,son a 2002; and Rune- H a strict boundary betweenronment studied but object studies and their interaction, itskey which envi- to may understanding. be the Case studyqualitative research in is nature, inherently not providingtistical conclusions with significance. sta- But still sometimes generalizing entire duration as employees ofconsortium. a Researchers major and partner their in directcupied colleagues the key oc- positions in the project and were generally in- and especially in the softwareages. engineering work This pack- allowed forthat broad was going insights on into in the everything and project. close Participant contact observations with thesary day-to-day to reveal work the are actual neces- informaltices and that situated the work collaborative prac- work insoftware a development consortium-based project follows.research towards Orienting ethnographic methods, the offeredadvantages many for the analysis ofcomplex differentiated relations environments in (Boden et al., 2007; Suchman, on-site observations of project participantsanalysis. and artifact During the projectners duration, were visited respective repeatedly part- for several days. Local and volved with some resources in all of the work packages, A way to achieve a certain degree of generalizability is ware process was deeply analyzed, and findings were 1995). UALITYOF Q ¨ ost, 2009). NTHE . Actual work ¨ ost, 2009). They :O FP6 ¨ ost (2009), the study COPE S ROBLEM ¨ ost, 2009): II — P neous consortium can agree? How does the consortium charactersoftware affect processes? planned What is a common denominatorcesses of to tools preserve and knowledge pro- on which a heteroge- What does the typical EU project look like? According to Runeson and H • The research method that is best suited to answer the • • 32 of being observed by theseen subjects. as The normal researchers participants. were 2 The approach advantage is of that a itthe Case provides phenomenon, a and deep that understanding itof can of the see “official” behind view the of scenes substantial matters. amount However, it of produces data a and is very time consuming happened due to these events (Yin,the 2003). optimal Consequently, research method was a case study. can further be categorized asa a classical “Case 2” ethnographic approach, studyinteraction i.e. by with the a researcher but high with degree a low of awareness and are recognized as aet formal al., research 2009) method with (Verner aof high experimental degree control of (Runeson realism and at H are the appropriate expense when the researchor questions “why”-type are questions, “how” when thecontrol events of are the out researcher, of and the to understand “what” has is the original research methodrative for questions, investigating i.e. explo- finding outing what new is insights, happening, and seek- generatingfor ideas future and hypotheses research. Furthermore,a it situation is or used phenomenon to (RunesonCase and portray studies H explore and explainengineering contemporary software phenomena within their real-life contexts, above questions is a case study. In fact, a case study questions guided the research process: tium projects in the environmentpractices of are often different from theoreticalought “how things to be”-perspectives. Studying the collaborative ing perspective. It can showplex how the collaboration necessities affect of best-practice com- opment software processes devel- (Boden et al., 2007). The following 2.3.4.1 Research method vivid descriptions of difficulties that can arise in consor- The goal of the case study was to capture and preserve (Runeson and H work in such a project can therefore offer a complement-

II — PROBLEM SCOPE:ONTHE QUALITYOF SOFTWARE PROCESS ARTIFACTS eescesul rsne titrainltaefairs. trade international at presented successfully were distinct Fifteen Euro. million ten over of budget a with ( h taei betvso the of objectives strategic The years four than more was project the of duration The rjcsadgie ol-ieatnintruhtalks through attention world-wide gained and projects research universities. large several a and of institute, divisions co-located not some nies, ( Small Enterprises of Medium couple a and of consisting consortium, project formed the countries European different seven from partners overview Project 2.3.4.2 extraor- being while successful. example, dinary typical quite a that is teach Karakos projects similar several with experiences sonal ainaditgain srapiain,validation, applications, user integration, and implementation technical specification, engineering, ments 14 over of start the from goal major project. a the the was reference of of in complexity frame maturity its and process software size high the Comparably project. for adequate established are be to that need processes software pub- suitable be fore, There- to commercially. exploited need and that distributed, knowledge lished, and assets huge of a to amount leading projects problems technical research and such scientific tackle Naturally, or bodies. corporations, public even large small enterprises, institutes, medium-sized research and universities, involved: need this be organizations in to academic funding and receive industrial to both project area, a (FP6, for policies order EU In of 2011). support in activities research to ing promot- and while industry, competitiveness international of encourage bases technological and scientific together. nations European different three least at from other among i.e. kind, projects. its FP6 of as project considered successful be very can are a Karakos that code source. C# open produced and as Java published developers of lines software source 70 200,000 about than excellent More an cluster with review European rating. final important its passed three and of activities part is demonstrations Karakos software Its speeches. keynote and IP h ao ciiiso h rjc eedistributed were project the of activities major The the by co-funded Projects European ten top the among selected was Karakos ne the under ) okpackages work FP6 r eurdt rn utpepartners multiple bring to required are rjc aaeet require- management, project : SME EC FP6 ) e ag compa- large few a s), I3—A si nlsso nenlQuality Internal of Analysis As-is An — II-3 sItgae Project Integrated as r osrnte the strengthen to are P4wsddctdt eiwadassmn executed assessment and review to dedicated was WP14 eepandt oio h okpors.Several progress. work the monitor to planned were with start usually They sub-projects. to comparable what (i.e. h oeseilzdrl fa of role specialized more The aiae h rjc’ ult betvsrgrigde- regarding objectives quality project’s the validated n loec ako okpcaehsats leader task a has package work a of task each also (i.e. and partner is project package one work by Each the lead into packages. brought work be various to in needs project expertise this where and expertise excels, of it area certain a consor- possesses Every member work. tium necessary the to to package contribute work jointly one on collaborate partners consortium milestones defined of set objec- a and of tasks, number accordant small and project. tives a the of exhibits end package the work until Each last often and start project the project. Karakos the shows of 2.6 organization Figure and progress. structure project the monitor to required EU was the which year, by per once reviewers external by package Work exploitation. and training demonstration, ls novmn feduesi h rjc,Karakos project, a the and in changes end-users to of response involvement agile close an guarantee to order In Engi- Software User-Oriented and Iterative 2.3.4.3 later. introduced but planned project originally the not was in role deliver- This software code). of (source quality ables the of care took specifically risks. and issues technical of management identi- and the oversaw fication also role This documents. liverable research. all of in areas and involved aspects the scien- all of the in as objectives well technological as and vision, tific project’s the with in developed project solutions the technical ensure the to of had compliance board the package technical work The technical appointed. technical a been the of have leaders chaired all also which to manager board, tech- This all to regard work. with nical project the of progress overall the The Board. Project manager the technical of project authority Karakos the the under of manager responsibility the was of project dis- coordination the overall partner The each representative. which one patched to Board, Project overseeing okpcae omlreuiso okadaesome- are and work of units large form packages Work aao loipeetda implemented also Karakos an comprised Karakos of organization project The akleaders task ern Process neering ). a h epniiiyfrmanaging for responsibility the had otaeqaiymanager quality software ult manager quality okpcaeleader package work who 33 )

II — PROBLEM SCOPE:ONTHE QUALITYOF SOFTWARE PROCESS ARTIFACTS Dissemination Demonstration Domain Business WP5 work package WP9 WP13 and Exploitation Manager Advisory Board Advisory Manager Knowledge and Financial Administrative Business and RTIFACTS A Training Domain System WP4 work package WP12 ROCESS WP8 integration P Project Project Review Review Project WP1 Management Assessment WP14 OFTWARE Demonstration System S Domain Work package and task leaders package Work WP3 architecture WP7 work package WP11 Project Board Project Liaison: Project Manager Liaison: Project Manager Liaison: Project UALITYOF Chaired by Project Manager by Project Chaired Chaired by Project Manager by Project Chaired Q (Project Officer) (Project Project Management Office Project Validation & Validation Domain Requirements European Commission European NTHE work package WP6 WP2 engineering Business Framework WP10 :O COPE Figure 2.6: Organizational structure of Karakos S Training Manager Domain Domain System Quality Manager ROBLEM Technical Manager Technical WP4 work package work package WP6 WP8 Integration Technical II — P Advisory Board Advisory Technical Board Technical Chaired by Technical Manager by Technical Chaired Architecture Domain Domain Technical work package leaders work package Technical WP5 work package WP3 specification WP7 work package 34

II — PROBLEM SCOPE:ONTHE QUALITYOF SOFTWARE PROCESS ARTIFACTS lo eain ewe eurmns(..contradiction (e.g. requirements between relations Also, software. envisioned the from expect would they what (like Stakeholders each. year one of duration a with u oamxmmo he fe h hr trto)to iteration) third the after three of maximum a to (up hr r eti hrceitc of characteristics certain are There appli- three the of each for applications prototype The oe h emo aao osse fpsychologists, usability-engineers, of and industrial- consisted scientists, Karakos computer of team There- the process. fore, engineering the enhances multi-disciplinary greatly in a team on from early expertise from Further, users project. human the of inclusion the is design process human-centered a for in- Essential computer-based systems. of teractive cycle life design the human-centered throughout activities on guidance 13407. gives ISO 13407 norm of ISO principles the followed project the domain-independence. demonstrate domain application proto- additional the one iteration, address each would of type end the At approach. development software user-oriented and iterative an applied rsrbdb h rmwr rgams o instance For Programmes. Framework the by prescribed project EU an Organizing 2.3.4.4 of project. the phases different of the iteration shows each 2.7 Figure were far. that so results achieved the integrate to consoli- and to development valuable date are iterations develop- Also, user-oriented processes. of ment feature key a Iterative is needs. development end-users’ the of simul- consideration with taneous gradual goals implementation a the ensured to process approximation iterative This augmented continued, evolved. been and also have domains cation system. the of parts (integrated) of validation evalu- and the ation and implementation, and architecture software resolved. necessary, if and, identified were inclusion) or values. dissatisfaction and satisfaction customer rat- overall and an ing using prioritized be could Requirements identify to them asked and users potential re- of groups and consisted focus of The refinements requirements. existing new, of to prioritization lead iter- earlier would the this from ations, feedback them. and from groups focus scenarios Using technical and detailed scenarios vision more of derived refinement Every the project. with the started to cycle project) prior the scenarios for vision idea defined the had gave who owners” “vision the others. and designers, h olwn tp eete h vlto fthe of evolution the then were steps following The cycles development full four through ran Karakos of process development software user-oriented The I3—A si nlsso nenlQuality Internal of Analysis As-is An — II-3 EU rjcsta are that projects hl en ieyatnmu n self-dependent. and autonomous widely being while be. will re- (like activities process key them, in much how work au btat hsgvsE rjcstenecessary the projects EU gives This abstract. vague uain euse udn,coc fprnr,number partners, project of e.g. choice consortium, funding, the requested of duration, discretion the at is istics three least at from different come to have partners consortium that project of packages Karakos work the and phases Development 2.7: Figure lndie eeomn a o tte eywell. very them fit not may development Plan-driven natures, consortium and research their from agreement and results. collaborative direction, expects fundamental its sees course, progress, Of project’s scope and parameters. quality fixed, variable are are project the of funding duration results While and research 2009). (Tolido, new experiments to failed direction and its adapt to freedom a than more much of that consist at barely and deliverables be measurable, can time, Efforts hardly estimate. are rough milestones a shifted, only is this How- of packages. all work ever, how the of among plan distributed rough are a partners makes consortium pro- the the phase, During posal self-organization: on much very un- in relies Work of lot plans. a to adds that predictability component research strong a in have different This design, projects. by stable is, and situation understood well for only tion there iterations development what and if source, open will be code source developed if engineering), quirements will who and packages work goals, of ambitiousness and nsmay rjcsfc o fucranya low at uncertainty of lot a face projects summary, In distribu- consider to recommend (2006) al. et Lings EU onre.Bttecoc fms character- most of choice the But countries. EU EU rjcstherefore projects EC rjcs which projects, oiosthe monitors 35

II — PROBLEM SCOPE:ONTHE QUALITYOF SOFTWARE PROCESS ARTIFACTS , Word ; except for Word X and Subversion, and E RTIFACTS T A A was necessary to fulfill quality ROCESS Word ). Furthermore, BSCW offers version- P - and email-based process for their deliver- . In a folder-like structure, the users conjointly WebDAV OFTWARE EC S Word So when developers needed knowledge about another The original approach to documentation envisioned to tion, and that it wasstandards a did problem not that those develop themselves. whoalso set The did quality developers not accept Jirareported for them documenting bugs informally but to rather the respective developers. on a ables, or deliverables for whichSoon, they performed all reviews. reviews and corrections were done in Eventually, all writing wastwo done partners in that managedof to writing keep deliverables with their L ownanother process that promoted Google Docs. From then on, the lect feedback from others. Fewcaptured. project Instead knowledge was of keeping theeasily documentation editable in place one (wiki), it became scattered across partner’s software component, theyfor would support rather then be beg able to first consult documentation. about this practice that “gettingtation the is necessary like documen- trying toing get a blood management out initiative, of some stone”.tions installation and Follow- instruc- training materials werethe added partners’ to processes the differed wiki. in But standard what was documentation. considered For example,provide when Javadoc asked for to private methods inpartner their complained code, that one this was excessive documenta- times when documentation (e.g. areport) software had architecture to becould delivered, be the exported respective to wikisubmitted, a pages while Word the document actual that documentationremained was in a then the living wiki document. Athe BSCW platform server for served internal as sharingproject of documents artifacts and like other pictures,ments work-in-progress and reports, docu- but alsoto for the submitting deliverables organized, processed and exchanged documents in their sioning ( ing of files and allows configuration of access rights. generate documentation deliverables from documentation filed in the wiki. Onpost-processing the in quality manager’s request, a standards, since the layout of generatedthe text standardized did project not look. have Additionally,ners some had part- never worked with a wiki before, and insisted various media. while changes were not replicated into the wiki anymore. wiki was merely used for meeting planning and to col- A representative of one partner once resigningly noted workspace using Web Distributed Authoring and Ver- UALITYOF Q and the con- NTHE ) documents. At :O EC Word COPE S The main means of com- Following the “Don’t re- ROBLEM II — P and send a link instead. Skype-based phone problems 10 www.bscw.de/ 10 36 peat yourself!” metaphor — dotation, not instead duplicate generate documen- it (Spinellis,employed 2010) — a Karakos wiki fortively letting write living several documents. people The collabora- ported wiki export of to Karakos Microsoft sup- Word ( people in a call soSkype that chats. the While consortium this later simplified switcheding the to minutes, writing it of also meet- ledinstead to of extensive carefully chat distilled logs meeting being minutes. sent Software Documentation partners worked good. Developers reportedtimes that it some- was necessary to fall back to normal phoneportant calls documents were sent viahave email been while better it to would BSCW archive them inconferences a turned repository out to like be cumbersome with too many and work package meetings heldthat several there times was a a year high so at likelihood of least two twice persons a meeting yearemails as for well a as few about days),calls ten emails and mailing chat lists), (personal meetings, Skype-based andgeneral, normal communication telephone and calls. collaboration In between the posed to avoid the pitfallsever, from it earlier turned projects. out that How- hassle preserving in knowledge a can heterogeneous be consortium of a several partners. Communication artifacts munication were project meetings (general assemblies detailed definition of such processesbe needs a to compromise aspire between to theof different developing partners’ software, ways and theirdisciplines respective engineering and habits. Manyhad partners extensive and experience individuals in EUthe projects software and processes designed and tools in a way that was sup- process of the distributed consortium, and tothe capture proposal stage, the developmentproject processes are of usually an not EU yetthough fully elaborated the in project detail. contract Al- sortium between provides a high-level overview, itproject is work part itself of to the define its software process. The 2.3.4.5 The process in practice and internal quality Several tools were designated to handle the software valuable knowledge in software process artifacts. In when emails would remain unanswered. Sometimes im-

II — PROBLEM SCOPE:ONTHE QUALITYOF SOFTWARE PROCESS ARTIFACTS a e pi omo omtmiigls.Missent Mails list. mailing commit a of form in up set was h igs rbesaecd ulcto n violation and duplication code are problems biggest The hesi- are developers that observations the confirms This omntradps h opinet hs ue was rules these up. to set compliance not the push structure and enforcement monitor An to as rules. coding served the wiki for container The and developers environments. Karakos development the their to tailored cod- were These rules ing sources. of as maintainability such up. and properties set achieve readability to was defined repository were Subversion rules Coding a code, source ing Subversion and code Source reducing time influence. same hierarchical the at while and documentation, supply for between need disparity the increases nature Karakos consortium of The documentation. create to tating mi htsc esgswr sls n hudbe should and useless were messages via such and that personally email committers reminded manager ity mes- “ commit or many “a” like in rejected sages that resulted repository This the to messages. added empty we filter response a In this, 2010a). to Weimer, and Buse Hruschka 2011; (e.g. Starke, important and very are these although sages 1996). (Grinter, awareness unsuitable development is behavior supporting This for time. a vast partners at then changes but some of year) amounts filter per Furthermore, twice client (only rarely email trash. very committed their the had to or emails out list, mailing removed be the to requested from They messages. the read not did files’ changed a the message, and log names, the contained list this to support awareness Basic distributed 1996). in (Grinter, essential collaboration is which awareness, creating for released. source eventually when is remain code should staff” mem- x] any [organization to of footprint ber trace-able community, “no source that open requested and the in create publicity might viral release source adverse open an that feared Sonar partner calcu- on and based analysis This is lation Note: rules. analysis static of about code of source debt the technical end, a the accrued In ignored. generally were tory 11 lo nyfwdvlpr rt sflcmi mes- commit useful wrote developers few only Also, point starting good a is system control revision The reposi- or code source the regarding rules coding The http://www.sonarsource.org/ diff .Idvdasadtesfwr qual- software the and Individuals ”. fcags u aydevelopers many But changes. of 11 ihdfutstig.One settings. default with o aaigadshar- and managing For 30% I3—A si nlsso nenlQuality Internal of Analysis As-is An — II-3 fisttlvalue. total its of okn ietre.So h eoioycnandde- contained repository the Soon directories. working ihteerrmsae“or,yuaentalwdto allowed not are you “Sorry, message error the with eoesi aao sdt hc neeyhn,entire everything, in check to used Karakos in velopers requirement a to commit a linking of use made velopers orcmoet”–“e,sr,wi iue ’ljust I’ll minute, a wait sure, “Yes, – component?” your u n o ls nemdaeadfia compilation final and intermediate files, log and bug de- The 2005). Gwaltney, noth- and but (Richardson product, more the creating ing for necessary is that thing 7.10). Section (cf. developers of majority the by Subversion behavior poor their to attributed developers The concept. icity atom- commit Subversion’s contradicts belonged which together, actually changes up, those this that bringing stated When developers message. same the with cession report. bug or de- the of None helpful 60%. of to number 52% the from while increased each, messages 5% messages to up helpful by minimally dropped or number useless the completely short time a year’s of a in within much However, very frame. change time not did that Yet avoided. ue h nierpstr codn obs rcie — practices best to according repository entire the restruc- tured and initiative into took days developer Some one development, technology. part- new some a For was Subversion repository. ners, Subversion the for defined you.” for it compile of version fre- newer a heard me be send could please following you “Could the like quently: dialog twice a meetings year, When integration a on Skype. together via executables came just partners or code exchang- were source partners ing It that situation again. common repository a the became use actively to could several convinced partners be months, the not support, after of Even offerings and it. reminders using anymore. stopped used they be the not So that could assumed and silently “broken” had was first. partners repository at few work a to fact, seemed in This But administra- in. to them libraries checking for send tors should they that email by the if compiled tmp allowed” a not example, is for name was, its file because XY file in check check-in a reject would that installed reminders, were unsuccessful filters several commit After server. machines developers’ the the on on as well as repository size, mis- the in and rapidly version results, grew the conflicts, were Commit chaos and matches like. the and results, 12 eiincnrlrpstr hudcnanevery- contain should repository control revision A suc- short in commits 10 to up were there Sometimes ttebgnigo h rjc,n la tutr was structure clear no project, the of beginning the At http://subversion.tigris.org/ or obj ietr.Tedvlpr eeinformed were developers The directory. 12 upr nElpe hc a used was which Eclipse, in support .class l,o na in or file, 37

II — PROBLEM SCOPE:ONTHE QUALITYOF SOFTWARE PROCESS ARTIFACTS RTIFACTS A in the form that partners and dein the form that the project consor- ROCESS P project survey OFTWARE S open rejection tium is unable to agree upon installing it,hidden and rejection developers mostly avoided writingsages, commit split commits, mes- or rarely committed twice a the structure of thedissolved public and Subversion its repository contents gotessary polluted files, with unnec- much source code or evenexchanged pre-built between binaries partners, were and no bug reports were filed,nostic information. wasting valuable diag- velopers (inadvertently) do not use it. year, inhibiting effective coordination of work, ’s Framework Programmes. To establish a frame of While dealing with open rejection is a management • A major reason may be that hierarchical power is • • • 2. 1. Karakos from the above caseEU study was a project inreference the that shows Karakos wasof not its a kind, rare a specimen survey with EU projects was conducted. ment faces two kinds of problems: decision, consortium partners will usuallytechnologies not that object do to not obtrudeon new ways them. of A working new technologysurvive although must there be are robust deniers, enoughniers and to their it freedom must leave to de- deny.cessfully Next, avoids a this technology first that pitfall, suc- hidden still rejection. needs to Instead cope of with makingtechnology hard should prescriptions, constantly a advertises behaviorbeneficial that to is the project. Itagement could of relieve some technical of man- itsing constant obligations. advertising For and instance, remind- itto could increase remind contributions developers to documentationthrough and direct motivate and visible personal benefit. 2.3.5 Karakos is not a black swan – EU lacking in a consortiumeverything: project. Leonard-Barton But hierarchy (1987)organization-wide is summarizes mandate not may that increase theinnovative switch techniques to but they rarely meetceptance. immediate ac- Organizational members who havein not to bought a new technologytage may it. under-utilize or A even tool sabo- that is to be successful in such an environ- UALITYOF Q faced some NTHE was introduced :O COPE S ROBLEM In reaction to the above problems, software quality manager II — P software quality manager coding rules were definedtherefore ignored, but which not is considered enforcednot worse and than codifying rules at all (Deimel and Pozefsky, 1979), • The case study showed an example of the organization To increase the effect of the software quality man- 38 them correctly. Problems regarding theknowledge preservation were of that Many of its details were therefore presented here. of a typical EU projectlems that occurred, was, one of although the manykind. most prob- Although successful it projects was of possible its tools, to it agree on was some difficult specific to get project participants to use discipline partners. 2.3.4.6 Case study summary and created much of its conceptual problem context. ager’s statements, a reportdeliverable to was the established EC. The asnesses report in a would the capture new project, weak- andadvice provide to quality the improvement partners. Buthierarchical even pressure that that did might not have create been the necessary to business “in a reasonable way”. A partner noted that it not produce components themselves. Soered it unfair. was In consid- general, thethe recommendations software made quality by manager wereconsortium. often ignored in the the role of to coordinate software quality effortstheir and to impact. emphasize The open reception problems in theas beginning. a He bureaucratic was instance seen thating hindered their partners work. in do- Instead, everyone should proceed with cleaned up. Severalunderstand developers what reported is problems where to (“youto just check out don’t or know where what to commit your changes to”),the or repository clean did not show an effect. Lack of hierarchy i.e. having trunk, branches, androot tags directory directories — in and the wrotetive an email explanatory to and all defini- repository’s (see chaotic Figure structure returned. 2.8). This time But no soon one after, the The Karakos project had much influence on this research, was a problem that those who formulated the rules did which version is the trunk. General reminders to keep

II — PROBLEM SCOPE:ONTHE QUALITYOF SOFTWARE PROCESS ARTIFACTS lhuhteewr o fseigpoesdeficien- process seeming of lot a were there Although Dmn,20) fteKrkscnotu achieved consortium Karakos the If 2000). (Deming, oprbyhg raiainlmtrt,te tis it then maturity, organizational high comparably a a components goal organization’s the the give with in-line to is that task aim common management’s system. is aim, the destroy common It and a selfish without become will organization components an unequal In such on with impact projects partners. an EU has in processes course, software of the This, self-organization 3.5.3.1.) of Section discussion in the to con- refer heterogeneous (Also the sortia. of because and disagreement character, is research there the of because results un- in is is certainty there quality Additionally, software 2009). (Grechanik, them secondary at To publications through venues. and high-reputation success cost its in reduce defines success academia to whereas quality, through needs their satisfaction customer Industry define increase — consortium ways. projects both different EU — industry in and partners Academia again: ubiquitous. illustrated are already problems review quality internal literature that The all. after bad the in from ratings excellent successful receiving EC most also the kind, of its one of was projects still it Karakos, in cies survey project the of Background 2.3.5.1 e srvsttepcla iuto in situation peculiar the revisit us Let hnalaemvdt sn rn l ei eoigsufi brances. in stuff removing begin ill trunk using to moved are all When x nm removed] [name xxx aetkntelbrlt omv h lsaon i.S laealmk rs hcot and checkout. best fresh not a do code make the all wrote that please person So the with out bit. it work a conflict, [...] around a is files there the merge. if force move often. update” to trunk. “svn into liberality always work as the your merge taken and checkout have fresh I a make mind. Source my We Open of etc). a sake etc in the it”, svn of dir i read use please sub (yes we to So three how way svn exists on the of there standard not use svn facto thats common Karakos de wrong, [...] files the some the are 12 on to use you checked created lets point no only as so “no to are project project. it”, have there Source do I svn. Open to that the a use clear of developing we structure is are way the It the at not broken. look are a already “this know, had is just structure I the but and breaker, party ind a be to hate I all Dear fcas hscnrdcinmgtete eafailing a be either might contradiction This officials. EC sassmn rcs,o aao a o so not was Karakos or process, assessment ’s iue28 mi ocnotu n uvrinlgmsaeo eiin27 revision of message log Subversion and consortium to Email 2.8: Figure I3—A si nlsso nenlQuality Internal of Analysis As-is An — II-3 EU projects ihbodto upr.Priiat ftesuywere study the of Participants support. tool broad with that sense common the to according successful, was hycudete epn ihaseicto,“don’t tool, specific a with respond either could They ebrsafdfrti oea doae ycommon by advocated as non-team role a this have for partners staffed of member 7% Only develop- (33%). the of team members ment quality often have are partners they such If then let project. managers, least the at by do fulfilled 14% be “every- quality). role on of rely care partners take the to of body” 11% man- while quality all, no at have agement (38% dedicated roles missing are management partners quality the of 49% exist. managers rarely quality independent organizationally depart- or quality ments separate EU because to presumably managers projects; quality assign rarely partners Project results survey Project 2.3.5.2 “none”. or know”, project-wide. used are tools which indicate to supposed processes basic for tools development software of consortium-wide maturity, use organizational for proxy and, a personnel as assurance quality of availability for asked 50 over of with survey a projects conducted (2010a) al. et Prause 2010). CMMI, (cf. success to related is maturity actually project Karakos the that assume to reasonable nodrt sals rm frfrnefrKarakos, for reference of frame a establish to order In ≈ 100 atcpns h questionnaire The participants. FP6 39

II — PROBLEM SCOPE:ONTHE QUALITYOF SOFTWARE PROCESS ARTIFACTS RTIFACTS project was to implement A 14 ROCESS P OFTWARE S The name is a pseudonym for a software project. While Karakos had all the tools in principle, difficul- The problems reported in the Karakos case study in The survey also revealed that reuse of source code 14 alization. The platform integrated acomputer tutoring instruction system, management a system and a set of planned 47% of reuse ofcurrent source project. code resulting While from there the areplanations, also this other can possible indicate ex- thatreuse there that are could problems be with aParticipants result of of internal such quality projects problems. the reuse source only code they about were halfevidence planning of an to. experience These from numbers theKarakos: open source All release its of components werefor originally reuse intended but only ahave few sufficient quality components for seem reuse. tosource actually team Meanwhile has the discarded open most other components. 2.3.6 Diverse Own Experiences In addition to the Karakosand case the study survey (see of Section EU 2.3.4) sonal projects experiences from in Section several 2.3.5, other projects per- firm further the con- image portrayed in literature. 2.3.6.1 Bora a web-based learning environment. Itthoring contained environment an as au- well as adaptation and person- ties occurred with the processes and tools intools practice. are not usedestablished adequately. software A engineering reason could toolsspecialized be are and that require too too highly muchimprovised learning collaboration for the of short, consortia.quire They too might much re- discipline, whichthe is developers’ in desire stark for contrast autonomy to and independence Section 2.3.4 are not atypical. Theon survey the did official not view inquire ofeveryman processes of but asked what the and project how they perceivedalike. the Karakos tools. is not aInstead, black the swan, encountered a issues misleading are sample. group representative for of a widely spread problems. among the participants was about 27% compared to a The objective of the Bora (Beecham et al., 2008; Singer and Schneider, 2012a). A lot of problems are based on the fact that available With respect to this, evoked images are to some degree UALITYOF Q had to 13 NTHE :O COPE S ROBLEM II — P -integrated static analysis alone did not qualify IDE http://checkstyle.sourceforge.net/ Not shown in the table are communication processes Participants of the interview were also asked to rate Results derived from the project survey regarding the 13 archival systems accessible through the web (top 32%) ity manager and a dedicated software quality manager. from its start and establishedport consortium-wide for tool more sup- processes thanexample, many Karakos other projects. had requirements For engineering (top testing (top 20%). Itprojects was to also among recognize the thecommunication, first power or 20% document of of sharing wikis using for dedicated internal documentation, projects also rely moretypes on like suboptimal emails and official deliverables. 2.3.5.3 Karakos in the frame of reference Karakos aimed at a high software process maturity right much on volatile media likeconferences, meetings, instant telephone, messaging video and personalless email, on and media more suitable formanagement archiving like and knowledge emailing lists or wikis. For sharing of lutely important). Interestingly, only aprocesses few are project-wide considered actually importanteleven). (four out Source of code revisionimportant control one among being them. the Due most tools, to other this process lack areas of certainly shared collaboration. experience even less code reviews, also “Meetings” intool. person counted as a as static analysis. Extensions likebe Checkstyle used. the importance of formally conductinga the Likert processes on scale from 1 (not at all important) to 5 (abso- 29% have one quality managereven in have the more project than and one. 11% use of tools are summarizedknow” in and Table 2.2. “none” Both, are “don’t takennot as being indicators executed of in the a process formalized and mature way. For standards. The situation is similar on the project-level cluding 13% that do not knowand their 7% quality that management, see it as a task for everybody. However, This higher maturity could be the reason why Karakos (Prause et al., 2010a). Karakos also had a dedicated qual- (due to a different format of question) which rely very 40 was one of the most successful projects. with 63% of projects having no quality managers — in- 17%), continuous integration (top 19%), or automated

II — PROBLEM SCOPE:ONTHE QUALITYOF SOFTWARE PROCESS ARTIFACTS a rae na tep orcmietedistributed the recombine to attempt an in created was De- Technological and Research for Programme work The er.Oe 5prnr rmtncutiscollaborated countries ten from partners 25 Over years. ( velopment versity. yposo eprcue o e netet in investments few too cause: documentation. deeper a of symptoms problem, documentation a of user of overall high acceptance a Despite interconnected documentation. of coherent piece and one into information of pieces In- ( Information terchange for Code from Standard information most American reconstruct scattered to had develop- still original they the ers, from knowledge gathered Although had they scarce. was and documentation version that first complained the built the not on had working however, developers refactoring, The 2.0 project. version original to the refactoring after major a underwent it projects, Flail project in this re-use during found created and were courses that numerous contents The e-learning goals. project’s the reach to uni- virtual comprehensive a and with experiment domain, to engineering served an in base build knowledge to large used a was environment This tools. cooperative 15 hntesfwr a bu ob eue nlater in re-used be to about was software the When Frame- Fifth the in co-funded was project Bora The h aei suoy o otaeproject. software a for pseudonym a is name The Dendro˘c ASCII FP5 ouetto ol(ruee l,2007a) al., et (Prause tool documentation Dendro˘c ,satdi 00 n a vrfour over ran and 2000, in started ), ls misadsrd fJavadoc. of shreds and emails files, ) al .:Mtrt n ecie motneo rcse nE projects EU in processes of importance perceived and Maturity 2.2: Table 15 Scin2.3.6.2). (Section hc uprsteexistence the supports which , rhtcua design Architectural management Project plan Test tracking Bug reviews Code metrics Software control revision code Source analysis Static testing Unit integration Continuous engineering Requirements area process Software Dendro˘c I3—A si nlsso nenlQuality Internal of Analysis As-is An — II-3 a rigt fix to trying was aepoesatfcsrmie h oeresponsibility sole the remained artifacts process ware h olo h li rjc a ouiyadenable and unify to was project Flail the of goal The yia otnsicue uhdvremteslk pho- like matters diverse such included contents Typical eeu n nlge eoioistruhu Europe. throughout hetero- repositories unaligned across and scattered geneous materials electronic to access Flail 2.3.6.2 ueadcuigotae rhtcuedocumentation. architecture outdated causing and architecture original the degrading layers, sometimes service developers by-passed times, part- waiting a the of for Because waiting ner. by down of- slowed and was impossible, development almost ten testing was Local machine components. developers other a the on to because access change no a had make he to capabilities limited very only partner. respective the of soft- the of most Consequently, con- services. were web software via the nected of components part- the the The Instead between servers. shared ners. own not its was on code software source the developed of part its developed ran partner and Each consortium. project the formed few a SME and institutes research some universities, Mostly contents. to to repositories access existing uniform of provide top on built was super-repository infrastructure virtual a com- Instead, or background. professional mercial educational, repositories, their other incorporate of to irrespective in able overlay being of infrastructure sense flexible the and open an aimed Flail being projects, at disparate reposito- from several originated of that foundation ries a on Building informa- sheets. material tion or regulations, experts, local domain questionnaires, famous about texts prints, blue tographs, h osqec a hta niiuldvlprhad developer individual an that was consequence The the by co-funded was Flail Mature rgntn rmfiedfeetErpa countries European different five from originating s 71% 68% 47% 35% 29% 23% 21% 20% 20% 19% 17% Importance 3.41 3.23 2.98 3.03 2.58 2.45 3.81 2.89 2.64 2.31 2.64 EC n a o he years. three for ran and 41

II — PROBLEM SCOPE:ONTHE QUALITYOF SOFTWARE PROCESS ARTIFACTS RTIFACTS A ROCESS confirm the results in a similar study P 16 OFTWARE S A wiki used by a research institute’s work group to share informa- The Moknowpedia is such a corporate knowledge The success stories of Wikipedia or wikis used by Numerous companies, research institutes and private A major problem is that users of public and corporate 16 out that Moknowpedia had some troubles taking off. tion. More information in Section 7.1.6.4 precious time in creatingis and often improving far more content usefularticle’s that to author others already than has themselves:about. the an To knowledge him, heprovides the writes only article few benefit. is Rational of beings less like humans value becauseit. it But a wiki withto few support contents knowledge and management poor asto articles there fails is consume. nothing Thecontributors wiki are is further not demotivatednobody as attractive, else they and is feel the contributing. that few In aeveryone closed is community, a specialistsmall with percentage of specialist volunteers knowledge, isactive not a participation enough. of all Instead, isin necessary. Moknowpedia The observations of knowledge sharing tools andand processes Veith by (2005) Reichling (Dencheva et al., 2011). management system. It is onein of the the evaluation environments of used thismation research. will More be detailed given in infor- Section 7.1.6.4. open source software projects clearlya voted for promising wikis approach. as In practice, however, it turned become temporarily unavailable, or if theydifferent change position to (McAdam a and McCreedy, 2000).namic corporate Dy- information systems likedeployed wikis to can support be knowledge managementet (Dencheva al., 2011). persons rely on wikis toknowledge manage their (Danowski business-related and Voss,However, most 2004; research Rohs, focuses on 2007). Wikipediaso and not much onand wikis Cheshire, in 2010; Kittur corporate andnumbers environments Kraut, 2010). for (Antin Resilient work environmentsBut are problems hard with to contributions come tobe by. wikis an seem exception not (see to Kraut also et Hoisl al. et (2010) find al., that 2007). success stories Indeed, like thatdiaWiki of software, not even every secondusers one or has more eight (Dencheva et al., 2011). will clearly prefer consuming content over producing where the number of potential authors is limited and wikis have to make personal efforts and invest their own Wikipedia are rare. Out of 6000 installations of the Me- UALITYOF Q NTHE :O COPE S ROBLEM II — P When the last developer’s contract finally ended, he Flail also relied on a five year old installation of Bora knowledge, insights and experiences aboutzation’s the field organi- of business. Preservation andsharing continuous of such knowledge amongin workers is knowledge-intensive businesses. essential Knowledge management helps to avoid redundant work,training to times, reduce and employee to adapt tosaves changing intellectual environments. capital It when employees leave, if they Organizational knowledge comprises highly specialized have to maintain that system.new Shortly problems after his occurred. departure, could There take was care nobody of left thethe who problems system as worked, nobody and knewSo there how a was decision no was documentation. form, made and to to decommission extract Bora. allnot Then data be the changed in data anymore a could butcontract static would duties suffice for in fulfilling Flail. 2.3.6.3 Corporate Wiki ment utilities. Fortunately the onestill development existed machine and the developer couldbackup do was the created changes. immediately and A the a computer post-it saying glued that to nobody should touch it. gave the advice in private to fight tooth and nail to never existent. Only one developer whooriginal had developers learned how from to the maintain Boralocker was there left. existed the In last his development machine.changes When to Bora became necessaryturned because out of that Flail, no one it knewenvironment how including to several set required up legacy a develop- development alone. Everyone was dependent on the others. decommissioned but it had notthree been used years. anymore since None ofployed the anymore, original and developers documentation was was em- almost non- this information was not enough:and telephone email conferences discussions were heldjointly frequently reconstruct in knowledge order of to thethe actual system workings when of a change was needed.commercial exploitation Meanwhile, since opportunities increased, reim- plementation plans were discussed onof the the quiet. partners None could do something with his component Most documentation was the solepartner. responsibility One of notable each exception wasbriefly a wiki described page all which web serviceduring the interfaces. later However, periods of the project, it turned out that (see Section 2.3.6.1) for data. Bora was never officially 42

II — PROBLEM SCOPE:ONTHE QUALITYOF SOFTWARE PROCESS ARTIFACTS htt o eticmeet n aeohrimportant other have and incompetent, felt do, to what 2011). al., et (Dencheva so do to like really would Especially domain. certain a in articles to tend and write aiu rbes ayitriwe ae mostly named interviewees Many problems. various afo h neveesmnindta hr hudbe should there structuring. that clear mentioned a interviewees missing the was of it Half them Also, stopped This it. learn. using and from use to complex too was tax (Dencheva use 2011): for al., a enough et that reward said be and would tool problems, good as wiki the of shortcomings and reported interviewees situation The this contribute. improve to to them done motivate be could contributing, what actively not and possibly themselves was they else why someone asked or It questionnaire. filled for short group a work reasons out the the of into members investigate all To contribution, low hoped. and as many good as as not were Moknowpedia to Contributions ta. 2011). al., (Dencheva contribution et increasing to dis- key this is of the proportion extent on the ride free Reducing to achievements. would tempted group’s whole is a person as single every group profit, the While acces- easily information. from sible profit them- others making while even replaceable, case extreme selves the in them time, costs own articles their Writing exists: for disproportion value clear beings, enough a thinking see rational, not For do contributing. they in that themselves is — do to them things tell to one no their have have tools, structure, other a to create drawn — to attention time syntax, the wiki have the not by learn do to hurdles they the that reason overcome The to themselves: mo- try not to are sufficiently they that tivated is explanation alternate an plaints, they although articles, write to time enough have not they did that stated interviewees quality most Finally, and professional contributions. make were to they enough that afraid competent also not write were to Students them article. told certain or a contribute, to regu- them that reminded instructor larly an missing were employees student to ones only were the persons were few they A felt too. they wiki, because the frustrated promote should to opportunity out No left wiki. be the help structure would better it to because instead, them wiki, semantic preferred a they have that to said Others con- additionally Moknowpedia. not to could tribute requir- they tools that group attention other their many ing so were interviewees there Some that then. stated easier much contri- be own would Adding butions structure. the creates who someone lhuhteei rt nmn fteues com- users’ the of many in truth is there Although syn- wiki’s the that answered interviewee fifth Every I3—A si nlsso nenlQuality Internal of Analysis As-is An — II-3 olau nesn ea mi ihtefollowing the with email an me sent once colleague A a o lne.As,dcmnaino h rtpro- first the of documentation Also, planned. not was e h oe si ih apnta h rjc st be to is project the company... that a happen to might sold it as could code, else the someone see neither that He is embarrassed testable. downright It mention feels to shit! not that maintainable, nor produced extensible for has what reason idiot himself: some complete to for thinks a is developer The it moreover because tested. and be time to all, long after the used after is again is it he code Now his at anyways. looking prototype a only hoping was time, everything long very a the for that code the at looked and hasn’t something programmed Somebody problem: esting a on working when project: English) into (translated content hacks quick of life long The 2.3.6.4 h otaewr ewitnfo cac Pas tal., et (Prause scratch 2011). from of re-written parts were large software project, the second for the responsibility for origi- took Silicate team was preparing new as a deadline When accumulated the intended. the after nally However, repaid to never debt time. was technical on debt of finish amounts to trade vast able a up be before took time They of period fair. short a the developed in had prototype developers first of force pres- task project small extreme a to sures, Due projects. been new had to others assigned were left, project had had first Someones project the anymore. of first available developers the the of after None year finished. one than more project the of half second project. the first during demanded not was totype project second a conducted, was When project partner. Silicate one first only by the built was part- software few the a ners, of both consortium industrial Although an involved projects. projects subsequent two of consisted Silicate Silicate 2.3.6.5 :-)” it of use make to like would I for Then responsibility code? of the out completely himself sneak could 17 olntyudvlpaslto o hsdeveloper this how solution a develop you Couldn’t inter- another have I topic? research your that “Isn’t h eodpoetfloe sapltadevaluation and pilot a as followed project second The h aei suoy o otaeproject. software a for pseudonym a is name The 17 uc dirty & quick a nidsra eerhpoetta actually that project research industrial an was ouinwudnvrb sd and used, be never would solution 43

II — PROBLEM SCOPE:ONTHE QUALITYOF SOFTWARE PROCESS ARTIFACTS (see Section 2.4.3). RTIFACTS A ROCESS P ogy affect quality tragedy of the commons OFTWARE S This section further investigates into social back- Both factors are important. Either of them is enough Regulations are not very helpful in such situation, es- People, processes and technologythree are main regarded factors that as influence the quality and productivity. Internal quality deficits are a common problem in soft- found on management and individual developer levels. documentation through unrealistic project plans,sibilities its of pos- preventing poor documentation areenforcing limited to regulations and appealing totivation. developers’ mo- The root cause isis that not valued writing very documentation much bydeveloper developers. himself The has individual few personalfrom his and efforts. direct benefit grounds and causes ofthe the iron problem, triangle of starting quality out2.4.1), and production with (see and Section broad resistance againstattributed new to motivation, processes resistance againstand innovation, the of sociology andsoftware economy, and internal it quality (see is Section then 2.4.4). transferred to 2.4.1 How people, processes and technol- to spoil internal quality. If managementmore is concerned shortsightedly with functionality ornot deadlines, express or a does desire formuch internal that quality, then individual there developers is canare not do. not If motivated to developers investnot in do quality, it. they will simply pecially if the role of managementis in consortium naturally projects weaker dueConsortium to partners less are pronounced on hierarchy. New equal technologies hierarchical that support levels. proving management internal with quality im- withoutmotivation coercion of but individual through developers areever, needed. such How- technologies shouldsusceptibility be to designed open to and reduce hiddentium. rejection in the consor- 2.4 Analysis of Problem Causes (see Section 2.4.2). The actual problem, however, is ware projects (see Section 2.3). The reasons can be After presenting the phenomenon in its original contexts While project management is capable of causing poor UALITYOF Q sweet spot NTHE :O COPE S ROBLEM II — P indicates that the Karakos case study is not FP6 management that neglects the importancenal of quality, inter- and fails to giveto priority, it, time and and effort the individual developer who isseeing not no motivated direct by personal benefitternal for quality. himself in in- • The descriptions of the situation already revealed two • After all, this experience is nothing new, as discussed later (see Section 3.2.1.4), improvements are of cost-optimal development. factors of internal quality problems: a problem. Next, a caseby study a of multi-national software consortium development framedcontext the of development this thesis. Aof representative survey the in projects a singular case. Finally,experiences several were examples of presented. personal Itcrisis of is internal clear quality that in there software projects. is As a will be problems are ubiquitous in softwareimage development. is The solidified by examples ofture popular which developer broaches cul- the issuecomics. in Also various ways software engineering including expertsterviewed recently about in- this issue confirm that documentation is techniques which support the creation of internal quality. 2.3.7 Summary quality. It started out with a review of research literature, need for qualityfurther but functionality will enriching likely cause theschedule massive costs slips.” scaffolding and Prototypes, which with shouldfor be the subsequent basis industrialquality. exploitation, need Industry internal market-oriented and applied organizations research specializing demand on tools and this strategy, butarchitecture it is more also mature creates thanthis it the practice actually is. is illusion appropriate, At that times forshow example, prototypes. an for trade However, fair suchproduction demos quality. are rarely of necessary Either before significant the refactoringthey real is must development be can thrown begin away. or Not communicating the Buschmann (2010) notes:ize some “It’s fundamental functionality oftenquality if aspects. easy you ignore Progress to the is real- immediately visible with very likely to move development closer to a This section drew a picture of the status quo of internal 44 which showed that documentation and internal quality

II — PROBLEM SCOPE:ONTHE QUALITYOF SOFTWARE PROCESS ARTIFACTS cmo”staino o hnsogtt be. to ought things how or situation “common” ihltl xeinei otaedvlpet perhaps development, software in experience little with the called often is This way. hr sadfiieheacyi httid ihpeople with triad, that in hierarchy definite a is There factors three the how investigate to seeks section This iinaditletaewa ae epeindispensable people makes what are intellect and vision eie rmJlt 20)adKc (2005a) Koch and productivity (2005) and Jalote quality from of derived Triangle Iron The 2.9: Figure vnwtotaykoldei olbrtv software collaborative in knowledge any without even people young assistants, research are universities of from portion large a However, intentionally. universities and partners research involve projects Research People 2.4.1.2 2006). this (Koch, achieve rejected to be fails should that both support tool for Any role processes. supporting and a people in tools and supreme, being effective. and efficient them make to their ought and It 2005b). people (Koch, processes. support it to to there is adhere technology itself. who Finally, in those problem to a invisible poses is never It process good solve A to necessary task. are a that problem. actions this the mitigate guide to Processes pri- is processes the of Therefore purpose mary 2005a). (Koch, tasks mistakes forget make imprecise, are and people But project. every in productivity suitable a in (technology) tools having good by use achieved people skilled are productivity and quality High Triangle Iron The 2.4.1.1 the to compared as reality research in situation different the if any and is environment, the by influenced are Process seFgr .,Jlt,20) Creativity, 2005). Jalote, 2.9, Figure (see Productivity Quality People and rnTinl fqaiyand quality of Triangle Iron Technology I4—Aayi fPolmCauses Problem of Analysis — II-4 people h r xet nterrsac oan hc snot is which domain, research their in experts are who 451712,00.html 13&s=1&zv=VBL&z=100 rnsfeun oancagsadancsiyt adopt to necessity a and changes domain frequent research Finally, brings needs. company’s the necessarily with not congruent are course, that of goals career This, personal emphasizes contracts. fixed-term closing perspec- long-term by offer tives to fail software. institutes maintainable research creating Many of aim possibly the a with yet salary, do conflict less to for opportunity compensate the could doctorate and Freedom average. below that notes interesting (2002) particularly Boehm is as which buy, can develop- money best that the buy ers necessarily not might in less than Paying academia and service (38kEUR public industry in less 15% were paid Germany in professionals young Additionally, prac- development tice. software not promoted research, got doing and through assistants, research as sci- careers their entific started They engineering. software necessarily researchers senior only architects, quality software developers, or senior managers few are There engineering. rechner/tvoed/bund?id=tvoed-bund-2009&g=E_ choosing by problem personnel. this address of consortia fluctuation Research high through hampered fur- is ther Specialization jobs. unsta- specialized process-related the or for and specialization room teams few small is of there result environment, at a ble universities As Eastern) 2. level Far at (mostly least ten than more no example, catalog For ( Model Maturity 3.5.3.1). Capability Section the also hostile (see naturally processes is which to self-organization of area an in per- 2002). commonly Jain, (Turner and development is agile it with but incompatible as industry ceived in role important an ad-hoc and tools, processes improvised using necessary to The leads matured. flexibility fully not and themselves ment Many Processes & Technology 2.4.1.3 technologies. applied the a of than knowledge important deeper more is Flexibility technologies. new 18 20 19 u otentr frsac,tashv ooperate to have teams research, of nature the to Due http://oeffentlicher-dienst.info/c/t/ http://sas.sei.cmu.edu/pars/ http://www.spiegel.de/jahreschronik/0,1518, technologies 20 otaePoesIpoeet( Improvement Process Software . it lot30 opne rdvsos but divisions, or companies 3000 almost lists 18 n ol sdaeudrdevelop- under are used tools and scmae o45kEUR to compared as 49 . CMM 99% fdvlpr are developers of eree 2010-02-06 retrieved pria results appraisal ) 19 SPI n2009. in ) plays ) 45

II — PROBLEM SCOPE:ONTHE QUALITYOF SOFTWARE PROCESS ARTIFACTS . excludability Laggards 16% and 34% Late majority RTIFACTS A rivalrousness 34% Early majority Time until adoption ROCESS P Early 13.5% adopters commons OFTWARE S 2.5% Usually people do not want to change the ways how While Rogers’ diffusion of new approaches is consid- Innovators In economics, four categories ofaccording goods are to distinguished the criteria they work. Even if apeople, new it technique takes or its technology time helps untilbook, it Rogers is (2003) taken classifies up. fiveadopters, kinds In of his signified famous technology by how muchadopt time a new it technology: takes innovators, them earlymajority, adopters, to late early majority and laggards.bell The curve typical is Rogers’ shown in Figure 2.10. ered on a voluntary basis,does even not organizational guarantee mandate that developers accept themor any at faster all. If theyas do immediately not improving regard their an performance, approachare then as highly they useful likely and to reject it. Many approaches inlong-term soft- benefits for the wholechances organization. of But success their are feware because incurred the by immediate individual costs developers (Leonard-Barton, exploit other motivations of developers couldto have increase value the adaptation ofand processes Schneider, and 2012a). tools (Singer 2.4.3 Economic goods and the tragedy of Figure 2.10: Adopter categorization onnovativeness the (Rogers, basis 2003) of in- earlier that young professionals havecannot fewer experience, yet value documentation andbe are cowboy more coders, likely and to have differentsortia motivations. have to Con- agree on usingnication the and same documentation media and for working commu- to processes be have harmonized, e.g. when and what to document. ware development that aim at internal quality have huge 1987; Riemenschneider et al., 2002). Approaches that UALITYOF Q NTHE :O COPE S project there is neither time nor ROBLEM EU II — P and technologies The above sections showed that differences from how People are the primary resourcecesses of and a tools project, are andthey only do. pro- there However, to it may assist beto them inevitable processes in to and adapt what tools. people For example, it was described internal quality, then freedom should notFrom be the a existence problem. ofseems the that software there crisis, is however,amplified a it by deeper the cause weakness that of could processes and only tools. get 2.4.2 Resistance against new processes of weak processes is thatquality more is of burdened the on responsibility individuals. for of The projects long-term rests health on individualexperienced, developers underpaid, who and are who less onlyperspectives. have short-term Yet they aredegree supposed of to freedom to use create their good high internal quality. If de- standards. But when processes arepends weaker, on success heroic de- deeds of individuals (Humphrey, 1988). things ought to be areever, present the for problems all in three one factors. factorsify How- may problems reciprocally with inten- other factors. In particular, the effect 2.4.1.4 Agility increases the focus on people fluence the situation of quality and productivity in soft- projects and projects close tomaturity research level may that not is have comparable a to advanced industrial existent. This reflects in theof way process-supporting how tools, projects although make usage use ate of tools appropri- is even more important in distributed projects sortium partners. Yet this were aright complex even task if in its only own two2009). parties A proof were of involved high (Sechser, neither process from maturity the is project not consortium required, members, as although a an whole increase nor inlevel maturity its has by been just shown one to bringof an up efficiency improvement to 10% (Clark, 2000). High market pressure is not skilled partners. But then company,language country, culture barriers and run in thethe midst beginning of of project an tasks.resources At for process harmonization between new con- velopers are intrinsically motivated to take good care for The three factors of people, process and technology in- (Carmel and Agarwal, 2001). 46 ware projects. Due to their inherent characteristics, agile

II — PROBLEM SCOPE:ONTHE QUALITYOF SOFTWARE PROCESS ARTIFACTS lhuhuigtepbi odde o erd t still it, degrade not does good public the using Although public of overgrazing or fishery seas high the are world fih,tecs fdgaaini hrdb l n more (no all by shared is degradation of cost the (fish), h iigo aubersuc ihalwquality low a with resource valuable a of mixing The aeo iaru odlk s,hwvr h society the however, fish, like the In good rivalrous individual. a the free-to-air of for case cost watching low or a used, only seas gains. has are high television the they only from but much fish cost how additional Removing matter no no almost are is they there available, i.e. are all, they for as cannot free long goods As the of controlled. use and be to access definition, by that, 2.11. Figure in given are kind each Examples the 2010). of Melvin, and an (Boyes using good from excludable excluded effectively market be i.e. can use, good’s participants the from excluded can efficiently are party be them a using whether determines by Excludability up rivalrous. used get that its goods reduces good availability; a using whether decides Rivalrousness oen a opyfri obcm vial.Teterm The available. become to it for pay to has applies. someone commons of tragedy the to similar nomenon resource. valuable a of destruction and the resource, to combined possibly quality lesser a to leads material landscape. the air, in the signs advertising into unpleasant fumes placing dangerous or or toxic water releasing into commons: a chemicals case, to this added In is too. bad like, something the and pollution like cases inverse over-exploitation. from them goods private protect into to turned been have societies, goods modern common In most 2005). (Diamond, extinction cultures the entire to led of even possibly has Overuse real pastures. the in examples Typical run. all long although the others, in of lose expense may the individual at selfish use the maximize for to This rational and degradation. profitable the it for makes way pay no individual is an there making non-excludable, of is resource the As fish). individual an by gained the is between usage in of repaired profit de- not the is While is it it uses. if when left), point fish degrades the (no fish) stroyed to even a possibly (removing resource, use the Every rational- humans. the of from ity originates that phenomenon logical else. someone by taken is it that does find it only if no may because it fish has not, a individual no take An is not to individual incentive one. one economic another by to taken available fish more the cost: the bears onnecual,nnrvlosgosaphe- a goods non-rivalrous non-excludable, To to applies principle same the that notes (1968) Hardin The problem the share goods non-excludable of kinds Both rgd fcommons of tragedy Hri,16)i socio- a is 1968) (Hardin, I4—Aayi fPolmCauses Problem of Analysis — II-4 reriding Free h s ulco omngo,pyls hnrequired than less pay good, common or public a use who the for paying not thus ticket, the for paying without hnalmmesdcddt ihodinformation.” withhold to decided members all when off. worse 20)seagnrcsca iem ninformation- in dilemma social generic a see (2007) h rgd fcmosi o iie otedomain the to limited not is commons of tragedy The The aube omnyondrsuc oe au through value loses resource owned commonly valuable, ento 3–Taeyo commons of Tragedy – 23 Definition individual. selfish an a by is made riding decision Free rational but maintenance. its for nothing or riding Free – 22 Definition 2007). Hoisl, 2000; of Huberman, tragedy as the identified of been cause has the networks riding sharing free file articles, Wikipedia like and resources digital With to rivalrousness. of respect distinction the making not goods, term the Often riding resource. free the to of way, loss this and to in deterioration act leads the parties This enough if and, resource. under-payment, transport the of maintenance riding free o h odo oit c.G,1949). it GG, (cf. use society to of obligation good the the it for with bring also consider to legislations far ownership some grants owner, its ownership to While rights reaching developer. a process and software artifact a between relation a defines Ownership ownership Code 2.4.4.1 suffered group entire “the while individual the ben- no for but efit costs has only contributing because exchange literature in used is term the Another and it carry. projects particular, they software information In of physi- goods or humans. digital virtual to the value includes any have to that applies goods It cal goods. physical of internal for commons of tragedy The 2.4.4 is group the that so gains individual of value sum more the loses resource than The individuals. of riding free nomto-xhnedilemma information-exchange rgd fcommons of tragedy quality tm rmteiaeyo rvln nbus on traveling of imagery the from stems sapidt ohknso non-excludable of kinds both to applied is ecie iuto hr oeo those of some where situation a describes sasca iem nwiha which in dilemma social a is digital imreadCress and Kimmerle . omn Aa and (Adar commons 47

II — PROBLEM SCOPE:ONTHE QUALITYOF SOFTWARE PROCESS ARTIFACTS s IDE RTIFACTS A restricts the number of develop- ROCESS P Non-excludable e.g. fish, water, air e.g. national defense, free-to-air television Public goods Common goods OFTWARE S The advantage of strong ownership is that responsi- But changes that cross module boundaries are diffi- The privilege to modify a software process artifact can do and thatcode often all affect over differently the code ownedtions base, source are and easier cross-module to modifica- implementcan because change a whatever single is developer necessary. to a few individuals or even a single one only. bilities are clear: Grossbart (2011)ownership notes mentality that “normally is great.feels like When they own every the developer productfuels it the drives passion quality up that and leads to inspiration.” Nordberg a direct benefit from increasesincome, in pride the in value achievements or and generated appearance,bility responsi- for problems or routineto maintenance, long-term commitment involvement, and potential for reuse. cult and time consuming todevelopers implement must because be several involved. Only theers respective of own- affected softwareto process change artifacts them. are Weak code allowed ownership deals with this does not restrict which artifactsmay an individual modify, developer and insteadall allows artifacts; everybody however, to it is change stillchanges considered through polite with to their talk ownersparticular, (Fowler, weak 2006). code ownership In has therefactorings advantage (like that renaming methods) that modern can be regulated through policies.ownership. This It is is the strong traditional code opment. model of Some software open devel- sourceMozilla software project projects follow like strong the code ownership models Definition 25 – Strong code ownership ers that can apply changes to certain parts of a software Strong code ownership (2003) continues that for developers, ownership implies (Mockus et al., 2002). weakness, allowing higher agility in development. It ) no UALITYOF Q (or NTHE :O collective and COPE S weak , e.g. food, clothing, toys, furniture, pizza e.g. cable television, private golf courses Excludable Private goods Club goods ROBLEM Figure 2.11: Four classes of economic goods (Boyes and Melvin, 2010) strong describes a relationship between a soft- II — P Rivalrous Non-rivalrous the obligation to use them for the good of the team. the privilege to change them, and Notice that code ownership also applies to other soft- • • In analogy to public laws, ownership of software pro- Fowler (2006) identifies three broadownership categories models: of code code ownership. implies. That means, when aform project is of said code to ownership, have not some other only source software code process but artifacts also are owned in the same are described below. 2.4.4.2 Three different code ownership models plies the privilege to changethe it and obligation be to called invest effort owner,tion. and to keep it in good condi- to invest effort in itresponsible to keep for it its in maintenance good soencoded condition. that in He the it is is knowledge easily accessible to the team. Definition 24 – Code ownership tification of knowledge. The developermost who about knows an the artifact is consideredthe to one be its who owner. is He probably is because best he prepared knows to the make most acall about change himself it. owner But of while the he artifact, may he has the obligation cess artifacts in a software team implies This view is — from a practical perspective — congruent Code ownership 48 ware process artifacts; not only source code, as the name way. The different forms of code ownership in a project ware process artifact and a person. Code ownership im- with Hattori et al. (2010) who see ownership as a quan-

II — PROBLEM SCOPE:ONTHE QUALITYOF SOFTWARE PROCESS ARTIFACTS ekcd ownership code Weak hl the While take easily cannot others leave, developers key When ievra pcaiaincue oeonrhpa de- as ownership code causes specialization versa, Vice knowledge. of distribution is ownership code weak with olciecd ownership code Collective The iulownership. vidual they artifact process software on more focus velopers vrterrsetv at.Ti a naaiaethe incapacitate 2005): Gwaltney, can and This (Richardson project parts. respective their over them. by owned not are that much parts have the not about do knowledge often developers other models these In and artifacts is of pieces there developers. between Still relation artifact. ownership process an software any change to ownership code Weak – 26 Definition uedabc hti slcigcerresponsibilities. clear lacking is it the that has drawback it huge team, the within and exchange knowledge collaboration higher closer to leads and flexibility, greatest ownership code collective with Problems 2.4.4.3 ownership code Collective – 28 Definition ownership: code projects collective Such have instead 2011). en- Grossbart, the 2006; by (Fowler, “owned” team is tire base make code can the as Anybody anywhere changes ownership. code sign- explicit any of ban posts Subversion like projects software and ship, more documentation. knowledge with of compensate to lack difficult a making by ownership strengthens therefore code debt Technical with. familiar are software. the of parts other about much learn not will they be- incapacitated. project gets the it leave fore to have that developers of number factor Bus – 27 Definition o hsreason, this For developers’ favors ownership Code also but ownership code strong with problem Another u factor bus olciecd ownership code collective o u/rc ubr steminimum the is number) bus/truck (or XP en htdvlpr r allowed are developers that means doae olciecd owner- code collective advocates bnosaynto findi- of notion any abandons specialization I4—Aayi fPolmCauses Problem of Analysis — II-4 oe offers model as hssbotmlt a emdtcncldb rfrto (refer debt technical termed was sub-optimality This o-xldbegos f o ntne h individual the instance, for If, of goods. commons. problems non-excludable above of prevent tragedy can ownership the en- code and is Strong riding that arti- free projects ownership, software by code in dangered no good or a weak constitute of facts case ownership) the code In no good. or (weak non-excludable owner- or code ship) (strong excludable an are artifacts process ownership. code research of This model this models. treats ownership primarily code weaker the build on often Es- methodologies development 2003). agile (Nordberg, pecially habits communication, work poor undisciplined teams, and in turnover clear high without schedules causes, slipping fixing, slow returning and repeatedly bugs designs, in inconsistencies source Typicalcode, unreadable documentation, poor knowledge. are communicating symptoms for and internal for quality accountable or responsible minimally only as also to ship refer therefore people Some oe nte sbr yeeyn.Abdatfc repre- artifact bad A everyone. by born and bor- is debt to them technical on belong but rowed everyone They to accessible projects. easily are software in good mon This good. team. rivalrous entire a the artifacts by makes incurred is contributor artifacts one on by borrowed debt Technical 2.2.2.2). Section read. to not difficult is and artifact crowded the becomes if it or well, there words structured When many too characters. instance, text for are, of number limited hosts a artifact been optimal only have The only contributor. can one by passages authored text individual it, informa- to of tion pieces contributed that may authors artifact multiple an have While other all artifacts. time; secondary a are at artifacts information certain artifact a one represent Only may base. knowledge of the pieces from unique information those cover and represent informa- Artifacts of tion. pieces legally duplicate information or of contradicting piece displaces One base. knowledge constraint the minimality of the to artifacts due But exclusive first. mutually at are good them non-rivalrousness makes a This as replicated. appear easily usually are they dia, projects. software agile proto- in for of or undesirable tardiness types it the makes But ownership code maintain. strong to motivation simple of them deal keep good to arti- a his have would with work he forever, to facts have would he that knew developer eedn ntecd wesi oe,software model, ownership code the on Depending osqety otaepoesatfcsaeacom- a are artifacts process software Consequently, me- digital in stored are artifacts process software As no oeonrhp vroei feeling is Everyone ownership. code collective oeowner- code 49

II — PROBLEM SCOPE:ONTHE QUALITYOF SOFTWARE PROCESS ARTIFACTS RTIFACTS A ROCESS P OFTWARE S A different solution is to privatize common goods. If Problems with internal quality were found to be ubiq- In many projects, non-excludability of software pro- Source code encodes detailed knowledge that was cre- to establish. the common good is broken up and its shares are given debt. It can bring antotal entire economic project loss. to Internal a quality halt,reduce therefore or costs. matters cause It to a improves developmentity, speed enables and a qual- better hand-over ofdistributed development knowledge, practical. and makes uitous in software development. Thereof is research a literature, large it volume isobserved a by topic software in engineering developer experts, culture, a evidenced in case study and acorroborated survey in of about personal 50 experiences. EU projects, and cess artifacts (i.e. weak codeit ownership) lowers is bureaucratic desirable barriers as for modifying software. commons result. The reason isadverse a personal market cost-benefit failure proportion rooted for in contributing to the knowledge base. Afailures classical is response regulation, to market i.e.or a federation, higher which instance, restricts authority example, access no to source the code is good.the allowed to revision For repository be without checked into priorBut checks like the reviews. bureaucratic approachmonitoring has and two enforcement disadvantages: have theirregulations are own rigid cost, and and inflexible, and possibly difficult 2.5 Summary of the Problem Software development is more likeneering. research than Most engi- of results createdpure during knowledge development are that istotype discovered through by several evolving iterations. aevolution proof A software key are factor developers, whoand in need relearn the to knowledge learn to be ableMost to of modify their the knowledge comes software. fromartifacts software that process serve as a personal learning environment. of the project. But othercode documentation is beyond no source less important. ated during the project. Losingthe a biggest project’s source risk code that is softwareface. developing organizations Still discarding source codenative can if be too without much alter- backgrounddifficult knowledge is to lost recover. or too Unnecessarypoor effort internal spent quality through was termed interest on technical Among the artifacts is source code as the prime output Yet free riding and social dilemmas like the tragedy of UALITYOF Q NTHE :O COPE S ROBLEM II — P the associated phenomenainformation-exchange like dilemma or free the riding, tragedycommons. of the developers’ reluctance to taketechnologies that up incur processes an adaptation and cost on them and while only benefiting the organization as a whole, • The problem with non-excludable goods is that ratio- • 50 down the road and dividedrienced among all developers, developers. elaborate Expe- processes andtechnologies advanced can suppress the effectscommons. of the Environments that tragedy do of not haveconditions, such have favorable few to putsolution that up directly against addresses the the tragedy.opers motivation of is A devel- necessary here. all are destroyed, leading to a net loss for the community. in software projects.own Developers precious will time not into investcondition their preserving because artifacts the in benefits a of good doing so are too far nal, selfish humans tend to overuse them instead of in- situation were identified, it wasintensify reasoned a that deeper they cause. only The deeper cause was found in problems started out with thecess three factors and people, technology. pro- While differences from an ideal it: everybody’s property is nobody’s property (Gordon, people, no one of whombeing like really a owns common-room. it, It itabandoned, will will and tend accumulate to end feel cruft.” up bleak and 2.4.5 Summary of origins even more expensive as discovering theedge relevant in knowl- the firstnot place. persisted leading to The costly knowledgeLayzell rediscovery. (1998) is Lynex openly and effectively call thisbehavior. unprofessional anti-team But as thebad consequences developer for does spoiling not thesubject knowledge have base, to to it the fear is tragedy of commons. Or as Gordon put sents the information in athe bad knowledge way, and base. therefore Any spoils developerthe wanting to information pick covered up bythrough the a bad costly artifact process has that to may go be as expensive or vesting in their sustainability. Goods that are valuable to The same principle applies to software process artifacts This investigation into the root causes of internal quality 1954). Indeed, Graham (2004) has observed that when “a piece of code is being hacked by three or four different

II — PROBLEM SCOPE:ONTHE QUALITYOF SOFTWARE PROCESS ARTIFACTS hr vr eeoe sfe ocag otaepro- software change to free is developer every where apino ulnatoiisotaiigpolluters ostracizing shows authorities 2.12 Dublin Figure of place. campaign in a occur is not control does rigid failure no market although a where but artifacts, cess property. developers others’ as modify rigidity easily is cannot prob- ownership The code property. strong their with to lem tied is well-being know their own- developers that code and clear strong are of Responsibilities negative advantage ership. against the it is protect This to and influences. good the interest of value an the has in now individual every then individuals, to iuto ihwa rn oeonrhpi needed, is ownership code no or weak with situation A I5—Smayo h Problem the of Summary — II-5 ihi n h rbeso rerdn hudgta get should riding free of problems the and it with nsceisa fidvdaswr uddb ninvisible 1776). an Smith, by 2004; guided all (Graham, were of hand individuals benefit if the as improve to societies way in a has as selfishness known individuals’ been internal long the of level to efficient Appealing an quality. to closer step one project dealing But commons. process software of tragedy the their by public seen when litter peers. reputation to negative a free risk remain but Dubliners good, While as idea. pressure social an introducing thereby space, public of hr smr osfwr ult hndaigwith dealing than quality software to more is There 51

II — PROBLEM SCOPE:ONTHE QUALITYOF SOFTWARE PROCESS ARTIFACTS RTIFACTS A ROCESS P OFTWARE S UALITYOF Q NTHE :O COPE S Figure 2.12: Litter is disgusting. So are those responsible. ROBLEM II — P 52

II — PROBLEM SCOPE:ONTHE QUALITYOF SOFTWARE PROCESS ARTIFACTS S h omrypatclyukonterm unknown practically formerly The the in research as classified be broadly can research This preexisting to reference of frame a defines chapter This tt fteArt the of State 3 Chapter lcto fegneigt otae 2 h td of study The (2) software. to ap- engineering the of is, that plication software; of maintenance and development, operation, the systematic, to a of approach application quantifiable The disciplined, “(1) as Std. it IEEE defines 610.12 thereof. study the and development software 1969). Randell, engineering and established (Naur other disciplines the practi- all and as theoretical foundations software cal same let the to on necessity build was the development It clearer event. deliberately make the a to to run-up was intended the in it provocation that a as is chosen wisdom Common develop- software of ment. problems common with dealt that neering Introduction 3.1.1 engineering. software of field Engineering Software 3.1 field. the area. in the knowledge of complexity the to add further motivation otaeengineering Software OFTWARE rcse.Cnet n oeslk ult or quality like models and and Concepts projects its processes. for methodologies and nologies on ieuetruhacneec n1968 in conference a through use wide found eeomn eiso ltoao tech- of plethora a on relies development iu Torvalds Linus — is f,Idsgetpitn u oyo h N oigstandards, coding GNU the of copy a out printing suggest I’d off, First n O edi.Br hm tsagetsmoi gesture. symbolic great a it’s them, Burn it. read NOT and a enclrzda emfor term a as vernacularized has otaeengi- software BugeadDti,20)adat iepitn,writ- painting, like arts and 2000) Dutoit, and (Bruegge hswr ucntytreats succinctly work This vrcnb ieayo h other the of or any is like development be software can if ever But (1).” in as approaches software-enginering/archive/SEorigin style?].” management Kahn organi- Gengis style a hoards [and mongolian sation a to in paradigm results projects programming large a extending Simply reuse. and/or maintenance over development green-fields sise empha- to tends also It software-production-in-the-small. Hamer ter ming 2010). al., et dance So, (Bertram intricate partners an equal is that under often top coordinated it the Instead, at work. leader all one devises with is, That way. erarchical term. the of meaning exact might the this be not although development, software of science engineering. and science craft, art, involving process development a Seibel, software is (cf. that craft argues a (2004) as Clark seen 2009). is it (Gra- Sometimes not sometimes 2004). and ham, 1997) Bailin, mathematics (e.g. to science, compared or is (Gra- it buildings) Sometimes (of 2004). architecture ham, or composing, or ing discussion: of subject still is otaedvlpeti ifiutt raiei hi- a in organize to difficult is development Software cooking to compared been has development Software 1 http://www.unl.csi.cuny.edu/faqs/ otaeengineering software h itnto a mrsieydsrbdb Pe- by described impressively was distinction The . 1 Pormigtnst eaevr uhto much very relate to tends “Programming : osbyn plain beyond goes otaeengineering software engineering disciplines program- sthe as 53

III — STATE OF THE ART . pro- 4 ), can . 3 WebDAV LAN s or tools for ), or IDE . Other kinds of ar- NFS 2 ) CVS Document sharing tools allow de- tools. For instance, source code is ) with their web browsers but without are easy to use hypertext systems that ) sticks or emails. Shared drives that are CASE ), Network File System ( USB RT Wikis articles A FTP http://www.cvshome.org/ http://www.mediawiki.org/ http://germany.emc.com/products/detail/ 2 3 4 software2/eroom.htm 3.1.2.2 Software process artifact repositories repositories like email archives. File-sharing drives platforms are independent of thebe actual kinds shared, of and files every to or project the comes up other withphisticated to one ways share way are documents. copyingrial files Bus with ( Arguably Universal less Se- accessed so- through the Local Areabe Network realized ( in many waystocol including ( File Transfer Pro- More sophisticated solutions like BSCW or eRoom lost. However, sharedproprietary files file are formats too making genericmodify, them and and cumbersome have impractical to totherefore analyze. restricted to the This more research specific types is below. Wikis allow developers to directly read and edit documents explicit editorial control. Theyriers have and low their entrance structure bar- can evolve freely and all kinds tools come to attention, forgenerating example, documentation. But most importantly, soft- managed by stored in revision controlConcurrent systems Versions System like ( Subversiontifacts or are stored in otherin repositories, a e.g. Jira requirements issue trackerdocumentation (Prause texts et in al., a 2008), wiki or like free-form MediaWiki Software process artifact repositories storethe and artifacts archive that preserve theteam. knowledge of There the are project repositories forthe artifacts activities from of most the of softwaretools development process. vary in The their specializationand for some specific have activities, also beentheir applied to original other one. activities than treat some The kinds following of paragraphs repositories though briefly there are other velopers to access the same documents and files. Such vide histories for files so that older versions do not get (called ware process artifacts are stored in repositories that are TATE OF THE ) is a sub- III — S CASE lists as types of . Any computer tool tools are omnipresent. CASE CASE CASE tools include diverse applications such ing CASE s, compilers, debuggers, bug trackers, revision IDE Particularly important for this work on a technical The Wikipedia.org article on 54 or re-engineering. But thisimpression arbitrary of list the only multitude gives of an CASE tools. level are tools thatage project help knowledge, developers and that accesstheir are development and already environments. present Documentation-related man- in management, documentation, system software, quality assurance, database management, software configuration management, analysis and design tools,and interface development, design prototyping, programming, web de- namic analysis, test management, client/server testing, as control systems, wikis and manypractically more. countless. Their number is tools, tools for businessmodeling process and engineering, management, project process planning, risk anal- Software engineering without the aidlonger of possible computers to is imagine. no collaboration, knowledge management andcesses. other pro- to software design, requirements tracing,tion, code testing, produc- document generation, andengineering other activities.” software (IEEE Std. 610.12) 3.1.2.1 Software process artifact repositories classified as a CASE tool. Definition 29 – CASE process. May include the application of software tools discipline of software engineering that isthe concerned aspects with of technical support forIt software addresses engineering. the use ofFor tools example, during bug software reports engineering. maynotes be on maintained an with office sticky wall, or be entered into athat bug supports tracker. the software engineering process has to be 3.1.2 Computer-aided software engineer- Computer-aided Software Engineering ( ysis, project management, requirement tracing, metrics velopment, integration and testing, static analysis, dy- They directly help the individual developer, or support The latter is an example of “The use of computers to aid in the software engineering

III — STATE OF THE ART iiei a eea huadrgsee sr n an and users registered thousand several has Wikipedia ii lo taheto rpitr files. proprietary of attachment allow wikis visitors for crafted carefully not is it web-based, is wiki 2011) al., and et (Danowski Dencheva way 2004; this Voss, in volunteers by authored were completely add or articles existing edit users The world. Slaeci 09.I diint h aae articles, managed the to addition In 2009). (Salvaneschi, otiuigt h rils n nlnigtepee in pieces the of linking process in and the articles, in the readers to involve contributing to a wants While instead but functions. layout simple with artifacts text created plain The are artifacts. process software these languages different in articles of Millions texts. new the over all from users anonymous of number unknown Wikipedia. probably is wiki a of The example colleagues. prominent equal most and leaders that group knowl- between of space exchange edge work horizontal a and provides vertical the It enables possible. are articles of confluence/ GForge like Jira or 2010). knowledge in al., et pivotal a (Bertram as are repository teams trackers software Issue of web. communication the the through database e.g. a to it, in interfaces them management storing sophisticated providing by and issues of track- track Issue keep issues. ers other to information, links origin and summary, persons, short involved a be the can of These properties different issue. highlight that fields consisting several structure of pronounced requirements. a or have typically requests Issues feature is- reports, Typical bug development. are of sues work of unit one covering trackers Issue research. this in role major a play Wikis management. com- Confluence and encyclopedia, mercial Wikipedia famous that MediaWiki the free runs the also are ones major Two software. documentation internal generic managing very them for makes suitable lightness ex- and towards flexibility Their content ternals. representing at not collecting collaborative knowledge, the of in internals engaging Wikis at 2001). aim Cunningham, and (Leuf ways meaningful eondeape fisetakr r Bugzilla are trackers issue of examples Renowned server wiki of implementations numerous are There of history revision the of track keeps also wiki The 9 8 7 6 5 http://trac.edgewall.org/ http://gforge.com/gf/ http://www.atlassian.com/software/jira/ http://www.bugzilla.org/ http://www.atlassian.com/software/ 7 u loitgae eeomn infrastructures development integrated also But . 8 rTrac or An 5 issue 9 htam tcroaeknowledge corporate at aims that roln eeomn n col- and development online or , sasfwr rcs artifact process software a is I- otaeEngineering Software — III-1 6 uoai ouetto eeaoscno vrpro- ever cannot generators documentation Automatic authoritative an be to have not does there that is work BugeadDti,20) nsfwr engineering, software In 2000). Dutoit, and (Bruegge i ewr.I nue htn l rrvso thereof revision or file no that ensures It network. via ienwifraina hycno eeaewa is what generate cannot they as information new vide esrSbeso,adcmeca rdcslk Vi- like products commercial and Subversion, cessor lost. ever gets in, checked was that developers to system updates control distributes and revision code A source stores systems. control revision artifacts. control process software all encompasses it product the that of items configuration all one of design i.e. the items, influence configuration of entirety in 2004). the engineering Hunt, and software (Thomas and particular general, in engineering ( Management systems control Revision constraints. time to implemen- due the evaluation in and addressed tation this been for not relevant have are they trackers research, issue While trackers. sue SourceForge.net like platforms laboration e o hyetatdcmnainfo orecode source from Doxy- documentation or extract They 1995) do. (Friendly, gen Javadoc 1996), Klein, and from extracted is there. and documentation is code the source in the integrated i.e. code, source and documentation example, For programming formats. more understandable in under- easily information support existing can refurbishing they by But standing 2005). (Raskin, there not documentation Automatic 3.1.2.3 reposi- non-accessible in hiding, in tories. lie can lot history a the Consequently, history. of evolution full the of source this regarding control version distributed The with control. problem version centralized This with area. deals source only have open research They the in evolved. ground have gained server particularly centralized a on rely Git like systems control version tributed SourceSafe sual 12 11 10 aoseape r pnsource open are examples Famous hsi iia owa ol like tools what to similar is This http://git-scm.com/ http://www.microsoft.com/ http://sourceforge.net/ is SCM sdsrbdb nt 19)interweaves (1992) Knuth by described as SCM 11 ple osuc oeb h s of use the by code source to applied c.Pas,20) eety lodis- also Recently, 2006). Prause, (cf. sasbtnilcmoetof component substantial a is ) otaeConfiguration Software D CVS OSF 12 SCM htn longer no that 10 O n t suc- its and P nld is- include addresses Revision (Didrich literate 55

III — STATE OF THE ART V- by High- does not perforce , as it does not con- ) that do not directly (the “rugby” approach) agile!manifesto waste waste model (Boehm, 1986). Scrum open source Spiral or wide-spread RT A XP model (Royce, 1987). Other models include the ¨ ohl, 1995) or the Reduction of overheads is indeed one of the central Today, agile methodologies are mainstream with Plan-driven development is not the focus of this re- “Agile” means an incremental approach to develop- be prototyping-based development (e.g. Naumann and imply a development methodology, itan is agile considered methodology as due to its similarities in its prime ment overhead and forwards a good team atmosphere Prause and Durdik, 2012).cized However, cynically it as was having many ﬁrsta hidden criti- dangers lame and excuse as for notBoehm, 2002; doing Rakitin, the 2001). necessary things (e.g. ideas of agile development: Agileactivities development (commonly avoids called contribute to project goals.sometimes Internal regarded documentation as is such tribute a direct valueis to important the for end-product. project However,tenance success, it and dissemination, evolution, and main- hasopment. its place Agile development in cannot agile serve devel- for as not an doing excuse important butmenting unpopular (Tryggeseth, tasks 1997; like Abrahamsson docu- et al.,Rakitin, 2010; 2001; Prause and Durdik, 2012). about 40% of projectset being al., 2009). classiﬁed Agile as methodologiesample agile include (Toth the prime ex- 2012). One of the earliest agile methodologies may fall search. It may be thatapplicable the to concepts such developed projects here as are suitable well study but due environments to this asearched. cannot lack be of further re- 3.1.3.2 Agile development industry and is also a popular research topic in academia Durdik, 2012). Agile methodologiesage entered with a the golden publication ofsmith the and Cockburn (2001) and others. ment with a strong focuscustomer on involvement. project It goals has and lower intensive risks, few manage- Jenkins, 1982). Although (Dingsøyr et al., 2006; Salo and Abrahamsson, 2008; (Takeuchi and Nonaka, 1986; Prause and Durdik, (Br (Salo and Abrahamsson, 2008; Babar, 2009; Prause and Agile software development is increasingly applied in s TATE OF THE IDE Water- Dendro˘c III — S before. methodologies are views is the approach to orga- agile and plan-driven development methodology Figure 3.1: Class reference generated by Doxygen 56 Plan-driven methodologies aim atrather large stable projects requirements, with where therein is the high results certainty to bethe obtained stakeholders and (Boehm, high 1986). agreement among Thesentative prototypical of repre- the plan-driven methodologies is the Characteristics of the different homein grounds are Table shown 3.1.driven Note and that agile the methodologies distinctionmost is between methodologies not rather plan- sharp, dwell and inthe a that two gray extremes. area between 3.1.3.1 Plan-driven development nizing the software developmentgroups process. of Typically, distinguished. Boehm (2002)of notes methodologies that have their both respective groups home grounds. documentation (Prause et al., 2007a).evaluate Many Javadoc Java comments to assist the developerspot on with the tool tips thatactivity. assist Such his tools current were implementation called 3.1.3 Development methodologies comments inheriting the structurefrom and the concept source names code, whiletion organizing in the a documenta- more palatableautomatically way interconnects (see disjunct Figure bits 3.1). and pieces of A

III — STATE OF THE ART aepoessi aiu as n fiskypurposes key its of one ways, various in processes ware con- knowledge human of borders the pushing while adap- rapid enables and value generates quickly that ware iomnssinicvlehst egnrtdquickly, generated be to has value scientific vironments individuals, motivated customers, satisfying like values esit eeomn ehdlge.Teemethod- These methodologies. development into cess it. to interest of Also them lost. from get documentation not obtaining does to information repositories valuable in that artifacts ensure process software manage to is While development. software into discipline engineering bringing tools, computer By of disciplines. use engineering making other of and virtues draw perspectives, the arts towards to and term crafting a from away as attention emerged has engineering Software Summary 3.1.4 needed. are problems tailored specific tools these and to Processes 2.2.3.4.) Section dis- in were cussed development distributed of (Problems research consortium-based projects. large, in problematic teams be collocated can and characteris- small Other requiring like however, tics, requirements. new reveals stantly en- research In environments. research in software development of characteristics iterative, self-organizing the and emergent for alternative viable a makes development property agile This requirements. emerging to tation soft- lower-quality developing for suitable are ologies 2005). Gabriel, and (Goldman others self-organizationand software, working of delivery frequent otaeegneighsfraie h otaepro- software the formalized has engineering Software method- development agile that states (2002) Boehm rmr objective Primary Size Refactoring Architecture Requirements Customers Developers area Home-ground CASE eeal uprsalknso soft- of kinds all supports generally al .:Dvlpetmtoooyhm-rud Bem 2002) (Boehm, home-grounds methodology Development 3.1: Table cest nwegal,collabora- knowledgeable, to Access lndie methodologies Plan-driven ihassurance High products and teams Larger Expensive requirements foreseeable and current for Designed stable largely early; Knowable tomers cus- empowered and representative, tive, knowledge external to access skills; adequate Plan-oriented; CASE scnendwith concerned is I- otaeQuality Software — III-2 Quality . otaeQuality Software 3.2 self- with research. well like fits projects organizing and Ag- mainstream projects. is of development kinds agile different ile for and suitable plan-driven are into that ones split generally are ologies h otadrl fsfwr ult ntetinl of improving triangle quality. of the ways in traditional quality surveys software and production, of section quality, role software This and modeling cost of the it. area improving the start into introduces the and grasp quality, better then of can cost One context. its to according 2006; (Prause, context specific 2009). that a Lincke, in term regarded abstract be an to is needs such, as of Quality, absence the deficiencies. [...] 2005), characteristics 9000, inherent (ISO of requirements” is set fulfills Quality a it. which without to exist “degree mea- not the be does can but essence, it, its through service, sured or object inherent an an is of quality property itself; in ob- excellence an means of and excellence ject, of char- achievement specific the to and refers acteristics it philosophy; like contexts or musics its physics, on dependent and property, characteristic, essential distinguishing an a accu- is and to Quality universal definition. person a give rate from to vary difficult it definitions making Its produc- person, all in servicing. but or industry software tion the in only not service) role a (or product a is a there is there Whenever situations. nodrt elwt ult,i ed ob modeled be to needs it quality, with deal to order In safeunl sdtr ncuteseeyday every countless in term used frequently a is quality gl,koldebe olctd and collocated, knowledgeable, Agile, methodologies Agile ai value Rapid products and teams Smaller Inexpensive requirements current for Designed change rapid emergent; Largely empowered and representative, collaborative, collocated, knowledgeable, Dedicated, collaborative soitdwt t tpasa important an plays It it. with associated 57

III — STATE OF THE ART the as part depends is what users understandable analyzability is an internal quality external quality ), maintainability ). This relationship is shown in Figure 3.3. RT external quality A “Internal quality” Another source of problems are the developers who Another interesting aspect is that quality of the de- It is obvious that external quality has a direct effect quality in use change the software. Any change toto the increase software its is technical likely debt. Inprudently the and best deliberately case, take developers the decrease in quality on internal quality. This relationshipbefore: has been the discussed better developers canthe modify better the it software, can adapt to the users’ needs. quality. This aspect is traditionallyimprovement important initiatives. in quality 3.2.1.3 Adversaries of quality Reaching high levels of qualityacteristics in is all (usually) of desirable the but qualityis achieving char- this often goal not possible orreason at is least that difficult high in quality practice.opposing can The forces only from be various attained sources. against Interestinglyever, sometimes how- high quality in a sub-characteristicsits (and superordinate characteristic) stands inhigh conflict quality with in another (sub-)characteristic. Foradding example, source code to improve the usability of a program etc.) will make the sourcetherefore code be larger and impossible slower. to It reach may characteristics maximum due quality to in inner all conflicts in the quality model 3.2.1.2 Quality characteristics, and internal and Sub-characteristics of the six quality characteristicsISO from 9126-1 can be associatedquality. with As internal and opposed external to internal quality (see Definition see of the software. Forof example, characteristic attribute. Users do not seeis, how i.e. analyzable the how good software developersproblems, can diagnose deficiencies and and identify parts to be changed.software Con- is for users, i.e.to how use easy the they software understand for how a particular task. on how users perceive the software in the context of use ences external quality, and that external quality velopment process is supposed to lead to better internal versely, developers do not see how The standard also recognizes that internal quality influ- (automation features, special cases, fancy GUI elements, (Spinellis, 2006). ( 18 – , ) TATE OF THE ISO func- III — S maintainability manufacturing- , into quality charac- one, (iii) a subjective one that considers cost efficiency one, (ii) an economic and , quality value-based usability , product-based . These characteristics are then further transcendent one, (iv) a process-oriented reliability , would be unreachable in the near future. The one, and (v) a portability The ISO 9126-1 identifies six characteristics: In the software domain, wide-spread quality models Depending on the philosophical background of qual- 58 and broken down into sub-characteristics, and attributes.Figure See 3.2 for atics graphical and presentation their of associated sub-characteristics. characteris- has ISO recently 9126-1 been incorporated andSystems and superseded software by Quality the Requirementsation and (SQuaRE) Evalu- series of standards (ISO 25000, 2005). model integrates various aspects of qualityon models international based consensus. It issidered therefore as generally the con- most useful one (Al-Qutaish, 2010). tionality software quality is the onetakes by more McCall of et al. aon (1977). product quality It (Kitchenham view and instead Pfleeger,modified of 1996). quality a With his model, userat view Boehm automatically et assessing quality. al.sociates Dromey (1976) structural (1995) aimed elements as- of sourceerties code that with link prop- to higher level quality characteristics. decompose the vague term teristics and sub-characteristics to makestandable, it and more eventually under- measurable. are those from McCall et al.Dromey (1977), (1995) Boehm or et al. ISO (1976), 9126-1. The oldest model for ingredient-centric user-based based and price (Garvin, 1984). ity, there are many different quality models. They try to quality root cause of the problemat becomes the visible elementary when differences looking of theexist five about perspectives quality: that there isphilosophical (i) an uncompromising and on its nature throughmodeled different and perspectives, its how cost. it is 3.2.1.1 Modeling quality Quality has been a researchtil topic today for there ages. is However, no un- broad agreement on the term. Geiger 3.2.1 The nature of quality Quality is an abstract andficult complex to concept grasp. that The is following dif- sections elaborate more The International Organization for Standardization ( (1988) even postulates that the aim of unifying the term

III — STATE OF THE ART stetaiinlve httetrefcosqaiy cost quality, factors three the that view traditional the is 3.4 Figure the project. shows software the originating of pressures management other the from to due normally is it erately, 2.2.2.2. was Section This in quality. discussed internal foster or to (inadvertent) (reckless) knowledge motivation many lack In they choice. however, other cases, no have they because account into management project of Triangle Golden The 3.4: Figure hndvlpr arfieqaiypuetyaddelib- and prudently quality sacrifice developers When Quality Functionality Interoperability .compliance F. Process Process quality quality odnTinl fpoetmanagement project of Triangle Golden Suitability Accuracy Security iue33 ult ntesfwr iecceo S 161adIO25000 ISO and 9126-1 ISO of cycle life software the in Quality 3.3: Figure iue32 ult hrceitc n u-hrceitc fIO9126-1 ISO of sub-characteristics and characteristics Quality 3.2: Figure eed on depends influences Cost Reliability al tolerance Fault Recoverability .compliance R. Maturity Internal quality Understandability Time .compliance U. Attractiveness I- otaeQuality Software — III-2 Usability Learnability Operability eed on depends influences It . Quality where 20)dsrbsi i aosbo htduln efforts doubling that book famous his in describes (2003) iw otadtm aeasvr mato software on impact severe a have time and cost view, xenlqaiya nteeape bv.Tepositive The above. examples the in as quality external The quality. of cost negative costs negative and positive so-called the pro- reduce The fact, 2005). quality in Schroeder, and can, (Harry but costs time duction or cost result increased necessarily not in does production in- in that quality learned creased recently have organizations Industrial quality of cost The 3.2.1.4 duration. project longer or cost inevitably increased quality to increasing leads that misconception a Likewise, is duration. it project shorten necessarily not does Brooks factors. is three there the that between model dependency this linear from a is assume it However, to quality. mistake and typical cost a as both quickly for bad as is project possible a finishing example, For this quality. In 2009). (Ika, constraints opposing are time and eoreutilization Resource C total Efficiency iebehavior Time .compliance E. C scmol xrse nteformula the in expressed commonly is = total External quality C fail C stettlcs fqaiy tcnit of consists It quality. of cost total the is fail + Maintainability C eut rmpolm fitra and internal of problems from results .compliance M. Changeability Analyzability prevent Testability eed on depends Stability influences + C appraise Portability Replaceability .compliance P. Co-existence Installability Adaptability Quality nuse in + C tolerance otof cost 59

III — STATE OF THE ART archi- When a soft- RT internal quality become necessary. A good architecture means A that is no longer easily handable. Any successful soft- structuring that organize it intotecture an understandable and when prototypes evolve into products.ity costs Positive qual- analogously cancel outquality. failure costs This in research internal focusesbecause on external internal quality quality depends cost onresearch it, environment and has because negligible the externalinternal — — but still failure costs. 3.2.2.1 Considerations on avoiding internal failure costs are diverse. It cannot befashion said what in that an practically across-the-board means.topic In is many what cases research this inor software engineering indirectly is concerned directly with.summarized in It a therefore fewbullet cannot paragraphs. for the be There problem.Prause, is The 2011) no following present silver paragraphs a (cf. failures, few and perspectives making on and avoiding keepingBut a it software is modifiable. by norather means serves a as complete a list frame ofresearch of approaches. into reference the It for broader integrating body this of knowledge. Narrowness of necessary knowledge that source code is intendedit for in throwaway by a writing languagea or production with version. a tooltransition This that from prototype helps cannot to to be product used (Foote avoid and in accidental Yoder, must be expected to last,maintenance. become a Internal product, quality and therefore matters. require 3.2.2 Reducing internal failure cost with High and low quality both have aative cost fault determined cost by and neg- positive costs.directly depends External on failure the cost application domain of the soft- software causes when it fails.ure Positive costs costs and prevent balance fail- total cost towards the sweet spot. code is expensive to modify becausequality. of This a is lack of a internal problem in maintenance of products ware may eventually grow to that size. Abstraction and Approaches to avoiding internal (and external) failure ware grows larger, its complexity increases to a level ware and its expected maturity, i.e. the damage that the Analogously, internal failure cost arises when source 1997). Any other prototype is not a true prototype but TATE OF THE III — S (e.g. metrics, (dotted). The new prevent ) for the cost of quality C fail C function (dashed curve) uses recommends to make it clear sweet spot into the positive costs, where tol- total C tolerance C (e.g. review, testing, validation & verifica- The following advice bases on the same rationale: The Internal quality problems, however, make mainte- No one expects full-fledged, flawless and efficient This minimum is what Crosby refers to when he says Figure 3.5 shows the inverse relationship that exists appraise 60 costs (interest on technical debt).moves Though towards the lesser quality sweet in spot anexternal environment with failure low costs, maintainabilityquality is characteristic an in any important software. differently knowledgeable, skilled and motivated developers, or novel tools affect thequality various in positive the costs project. of nance of the source code more expensive (this was pre- is an internal failure and causes internal negative quality no risk of externalure costs 3.6 of the original software failures.a lower In failure Fig- costsweet function spot is at abut lower software not quality at cost zero andSimilarly, level quality the (cf. sweet spot Huang relocated and after Boehm, new processes, 2006). functioning from a prototype. Auct. prototype There is are not no a users prod- annoyed that through could e.g. be dissatisfied usability or problems get with the soft- ing a presentation or exhibition butare painful consequences not to be expected. Consequently, there is almost first time” (Crosby, 1980). Ifreduces the quality development is cost for of free, software,the and irrespective domain of it of software, thenalways this wants is to certainly be. where one Thislow is cost true of even software if failures there like in is research only prototypes: a failures decreases. As the costthere functions is are a non-linear, minimum (or that quality is free: “Qualityit’s is free. free. What It’s not costs a money gift, are but the un-quality things erance means that the softwaredeal is with enabled problems to occurring gracefully at run-time itself. between positive and negative costs:investment with in an increasing quality, the risk and fatality of software costs are the cost ofprocess prevention improvements, quality standards) andC appraisals tion) (Malik and Choudhary, 2008). Bruegge and Dutoit viously called technical debt). Unmaintainable software (Galin, 2004). (2000) add ware. It may be embarrassing if a prototype breaks dur- Throwaway Code pattern — all the actions that involve not doing jobs right the

III — STATE OF THE ART oacsl es ehia eti h result. the is debt Technical mess. architecture costly degrades a but to ﬁrst at This development up layers. back-end speeds access directly the and bypass layer, to middle tempted be might of developers components cumber- project, front-end Web-Service be multi-tier can a rules In architecture some. to developer, adhere individual to the having for un- But to everything. having without derstand parts change can developers that from adapted (2006) spot Boehm sweet and cost Huang failure Low 3.6: Figure Software Quality Cost

Low Low High Software Quality Cost

Low Low High o alr otsetspot sweet cost failure Low original new oto Quality of Cost Total Minimal iue35 otaeqaiylvladcs fqaiyaatdfo ai (2004) Galin from adapted quality of cost and level quality Software 3.5: Figure otaeQaiyLevel Quality Software C total 0 C oa alr Cost Failure Total total new iia Software Minimal C ult Level Quality fail 0 I- otaeQuality Software — III-2 High otaeQaiyLevel Quality Software we spot Sweet Cri ta. 98.Ltigapoetsacietg,will go, architect project’s a Letting 1988). al., et (Curtis es sadiscipline a is Reuse deaqiiincnb pdu yuigoe norms, open using by up sped be can Knowl- acquisition software. edge the modiﬁca- about knowledge of obtaining executor is productive tions, a becoming to block patterns and Standards-compliance problem. the ameliorate can architects new educate effort every to making and knowledge documenting quality. Only internal for consequences negative severe have invaluable is capabilities engineering knowl- system domain and deep edge of combination hun- The a million. costs dred knowledge architect’s software a building supervision Learning the then processes, the at look itself. technology to important reuse, more an enable To is prevent mess. it cannot a all into turning they from and architecture itself others, in the is to none reuse superior to inte- regard powerful With are platforms. programs gration small many its with concept Open services, ( web initiative example, but Gateway For used Services is used. technology is what it of how It matter rather a 1994). much (Wasmund, so discipline not a is is reuse instead, in; oa ult Cost Quality Total otcnlg a es built reuse has technology No vnwt h etarchitecture, best the with Even pria Costs Appraisal rvninplus Prevention OSGi rteUi toolbox Unix the or ) High ao road major A 61

III — STATE OF THE ART Java Code Convention – If information is presented – How well the documentation – If all relevant information is con- – How easy it is to change – How simple it is to find documen- – Absence of contradicting statements – Degree to which the documentation – Degree to which it conforms to the ) to emphasize its useless character. The rea- RT A tation for a certain fact Conformity conforms to the relevant standards and norms Understandability does convey knowledge Completeness tained in the documentation Consistency in the documentation Changeability Timeliness software Unambiguousness Identifiability without ambiguity Notwithstanding documentation’s primary purpose to • • • • • • • • shelfware is to ease the understanding of source code, and therefore However, determining the quality ofmostly documentation only is possible through humanhave reviewers. been Efforts made to assesstion the readability using of linguistic documenta- metrics likelength. sentence structure As or early asclear that in metrics the are 1960s, limitedtexts, however, with lists, it regard tables became to and structured thelayout like. and retrieval Metrics aids ignore like content, indexes.oped for They were children’s school devel- books, anddocumentation. not Instead, for only technical usability testingdocumentation applied brings to reliable results (Forward, 2002). 3.2.2.3 Code styles and smells son can be that thementation has effort been needed estimated to to createactual be good effort twice needed docu- as to much build asbut the the product organizations (Brooks, usually 2003) only investof about development 20 efforts to into 30% documentation2001), (Pressman, while it is atfor the developers. same time an unloved activity convey information, the following quality characteristics are defined for documentation (DGQ, 1995): The idea of coding rules and the ( TATE OF THE “Docu- III — S Automated tests are in- ). Documentation complements blueprints by However, Lethbridge et al. (2003) found that man- ISO 9000 states that the value of documentation is to 62 dated documentation is often unsuitableto for impart its knowledge. purpose It isdone frequently to out the of point date, of over- outweighingso its poorly benefits, and that written peopleful might content not in it. eventimes try And even to be to misleading. the find Lehner worst use- such (1994) of documentation remarks all, is that jokingly it termed can “Schrankware” some- and quality needs, training, repeatabilityprovision and of traceability, evidence, and the evaluationagement of (ISO quality 9000, man- 2005). The description concludes add value, and not be an end in itself. software maintainers reduces the timederstand they a need to system un- and learnhelps how them to to perform better changes. use It their skills. tion.” It contributes to satisfying customer requirements that stores information, and transmits thatreaders at information a to later point inmentation” time (cf. Definition 11 – providing information that is notsystem necessary but to adds build to the itsfound understanding. in Tryggeseth (1997) experiments that documentation available to is to read it frommentation its is written to form. communicate The knowledge, purposebe and of assessed it docu- by has its to abilityand to impart Lethbridge, knowledge 2002). (Forward Documentation is a medium they immediately see if theirside change effect. had It an has unintended onlycoverage a therefore low helps cost developers for by them.to allowing A experiment them high with test and explore changing the software. 3.2.2.2 Quality of documentation and Reiners, 2011). Playful experimentation tegrity checks. They allow developerscode to without modify having source to fearthey to can break run something. automated When tests after have made a change, standard libraries and off-the-shelf componentsdevelopers so can that use preexisting knowledge, andto have learn. less Patterns support developersto by reuse allowing structural them knowledge they already have (Prause with a reminder that generating documentation should After all, an important means of obtaining knowledge “enable communication of intent and consistency of ac-

III — STATE OF THE ART A oeudrtnal.Bta ftdy hr sn toolkit no is there today, of source as making But of part understandable. important code an also is identi- naming instance, fier For code. source rules-compliant about because decay. code styles; source let coding will strong developers otherwise with ownership possible code only no or are weak that argues Moreover, (2003) 2004). Nordberg Pugh, easier and (Hovemeyer code understand source to make to valuable coding a extremely Consistent has are noise.” styles theory signals: Information such for data. term no specific in carry signals that simply code are the because deviations [... style maintainability meaningless code’s the ...] of aspect “style key that a adds is (2011) Spinellis 2009). Seibel, Loveland, 1997; 2009; al., et (King maintainability its to contribute mlsteeaei ic fsuc oe h oe its lower the code, source of quality. piece internal and a violations in style are coding there smells more The code. source ity code. source in smell Code – 30 Definition 1999). (Fowler, is refactoring that for code need for source in scaffolding identify a to as learning serve when can developers They condi- code. troublesome source of in tions indicators such are few there a Yet many. and too guidelines are of method lines a many in how code or source many, many how too of are variables threshold aes- instance absolute with no do is to human There lot by a outperformed intuition. has often It are metrics define. while to refactor- thetics, easy for not time is right is, when ing the However, only. technical targets debt Refactoring effects. functional its ing toring 3.3.5.3. Section see processes toolkits such analysis static automating using For automatically. fact, do modern In that can something developer. is the styles for fixing effort especially and much checks, cause automatic by not achieved does easily be low-hanging can a it is styles fruit: step Fixing first approach. pragmatic optimal a an like than more therefore is Promoting rules 2010b). coding Weimer, and nat- (Buse from languages differs ural code source un- of to readability starting how researchers derstand are now iden- Only of goodness naming. the tifier assess could that available readily oesmell code oesel r hrfr nidctro orqual- poor of indicator an therefore are smells Code condition, suboptimal a in is code source When all are understandability nor maintainability Neither rnsi noabte odto,wientchang- not while condition, better a into it brings sa niao ftene o refactoring for need the of indicator an is I- otaeQuality Software — III-2 refac- IDE s ult management Quality management Quality ult control Quality planning Quality ento 1–Qaiymanagement Quality – 31 Definition products. of quality the improving at aim that activities management quality of Terms 3.2.3.1 quality processes. of organizational focus on the usually get, is can management they that best tools the buy and will people organization an that Assuming the improving process. and technology, improving people, improv- the threefold: ing are improvement oppor- quality indicated, for 2.9) tunities and Figure quality to of back Triangle (refer Iron productivity created the As are that organization. products the of by quality the improve organizational to an scale on conducted is management Quality management Quality 3.2.3 oto ilatal euti quality. in quality result and actually planning will quality control as done activities the that 9000). (ISO requirements” quality fulfilling on cused control Quality – 33 Definition met. are requirements en- quality to that testing) sure (e.g. processes production and products 9000). (ISO objectives” quality the re- fulfil related to and sources [...] specify- processes and operational necessary [...] ing objectives quality setting on focused planning Quality – 32 Definition them. achieve set, to to how goals planning quality then what and deciding with deals that ities surance of eas to regard with 9000). [...] (ISO organization quality” an control and direct to ial,tetidsto ciiisam tvalidating at aims activities of set third the Finally, of analysis actual the is activities of set second The ar- activity three the of consists management Quality ult planning quality . ult planning Quality s“ato ult aaeet[.]fo- [...] management quality of “part is s“ato ult aaeet[...] management quality of “part is en h codntdactivities “coordinated the means steubel emfrastof set a for term umbrella the is , ult control quality oe stefis e factiv- of set first the as comes and , ult asquality 63

III — STATE OF THE ART of the developing organization and of its processes are assessed. What tools The focus of ISO 9000 is on fulfillment of RT capabilities A maturity There are several frameworks for assessing the matu- Consequently, improving the maturity of the software rity of development organizations.provide These support frameworks in achieving higher process maturity. that they very much focusand on not the on production design of phases. products, mostly But a software design development process, is whiletion the is actual taken step over of by produc- process compilers. is difficult Success to in measure: theber how design of to failures measure there the num- areInstead per 3.4 million opportunities? the do they use and howthat is a personnel mature trained? organization The isthe idea more desired is probable product to at the deliver end of the project. process should lead to betterSoftware products process and improvement development. and assessment canget tar- all three factors ofcan people, aims processes, at and tools. empoweringskills developers It and motivation, with at knowledge, improving processesthem by explicit, or making commit toand development methodologies, at supporting work with more sophisticated tools. to software engineering. to quickly reach majorreach reductions fast in and defect sharp rates, payback.the and The number sigma of defects levels per describe numberating of at opportunities. a Oper- three- to four-sigmadefects level means makes that up the for cost about of 20 to 30 percent ofsix-sigma revenues. level, there is fewermillion than opportunities one (Harry defect and Schroeder, per 2005) 3.4 ISO 9000 expectations and requirements. The standarddefines therefore quality as “degreecharacteristics to [...] fulfils which requirements”. a Itperhaps set is the nowadays of most well-known inherent qualityexperienced standard very and wide has adoption inof industry. The the focus standard ismake work on repeatable documenting and reach processis consistency results in mostly goals. to criticized It for notnizations, and being specifically suitable not for so alldevelopment suitable and orga- for agile development software (Seddon, 1997) 3.2.3.3 Software quality management standards The problem with the volume production standards is This is where most companies operate. Upon reaching At this point, software quality management comes back TATE OF THE , the TQM III — S fitted better with that reductions in de- ) is a quality standard TQM σ 6 TQM (also ) is a management philosophy dedi- is “part of quality management [...] TQM Six Sigma ume production Going back to Deming (1994), Total Quality While standards in general promote quality in one way 64 that emphasizes the idea of fect rates actually lead toand cost control costs savings are because reduced. failure focus As is opposed less to on aterm broad, perspective business but philosophy on and statistically long- analyzing processes Still improved quality cancosts result as, in for fewer instance, production and quality scrapping control may may be reduced. befar omitted eastern philosophies and wasup therefore by first Japanese picked automobile industryreturned before back it to eventually the USA. Six Sigma cated to quality improvement. Itpant involves every of partici- the production processing (even organization) outside in the the produc- goal ofcus achieving is quality. The on fo- long-term perspectivesand within with the organization its suppliers. Perfectiongoal, of while the cost product savings is are the more of a secondary goal. organization, drive goal-oriented process improvement activities, and compare theorganizations capabilities (ISO 15504-1). of competing TQM Management ( or the other, there are standardsimproving quality. that specifically These aim quality at managementpromote standards organizational characteristics that areat effective achieving quality goals.risks Strengths, can weaknesses and then beneeds. identified and They allow compared to to characterize business current practice in an dardization. The purpose of standardsrect is functioning, to to ensure ease maintenance cor- andoperability to between extend products inter- and services.dards Today are stan- not specific tobut the are military common or to software industry. domain 3.2.3.2 Quality management standards from vol- Definition 34 – Quality assurance focused on providing confidencements that will be quality fulfilled” require- (ISO 9000). The military were early and longtime advocates of stan- Quality assurance

III — STATE OF THE ART Kepr 2007). (Kneuper, SPICE 2000). Dutoit, ever and having (Bruegge levels demands higher more with 1, least Level The is level organizations. maturity for services assessment independent provides and organizations, five for defines levels It maturity standard. management quality software CMM n enssvrlmtrt eesfrprocesses. for levels maturity several processes, defines own its and improve and assess understand, tion the to 9000. Similar ISO including approaches improvement process from ideas incorporate that Deter- ( Capability mination and Improvement Process Software work they how way the changing discipline fundamentally them from and requires It organizations. most for 1. 5. 4. 3. 2. eciglvl2i osdrdtems ifiutstep difficult most the considered is 2 level Reaching asadmas n edt meit adaptations immediate to lead and means, and ways Hmhe,1988) (Humphrey, ocagsadopruiisi h environment. the in opportunities and changes to new using fast and are comprehensive processes implemented Learning place. taking per- is process formance of improvement steady A Optimizing: standards. to without adherence project or new quality a sacrificing to processes adapt Management easily can 3. Level of in processes predictable to and comparison quantifiable well are 4 processes Level measures, of precise on Based Managed: context. respective all the are to These tailored tools. and stan- procedures in methods, orga- realized dards, the is in and well-understood processes is uniform nization of set A Defined: repeatable. become successes that earlier so methods that established ensures follows discipline development Process plans. time and cost controls management project Simple Repeatable: reproducible. rarely heroic are and the individuals on of depend deeds mostly is Successes environment the chaotic. ad-hoc, are Processes Initial: The h S 50- tnad as nw as known (also standards 15504-1 ISO The SPICE CMM CMM )i eiso nentoa standards international of series a is )) PC at ohl norganiza- an help to wants SPICE , spoal h otwell-known most the probably is CMM n te software other and I- otaeQuality Software — III-2 egtaiedvlpetfcsn ntctknowledge, tacit on focusing development agile weight with address standards management quality many what 1995). al., et (Curtis 3.7) Figure (see workforce ( ( h eea iwo gl eeomn methodologies development agile of view general The aiain h raiainsmtrt ee ste de- then is level maturity organization’s The validation. eoesmk itksbcueo hr-al ntheir in short-falls of because mistakes make velopers oiaigadmngn efrac,adsaigthe shaping culture, management: and and performance, managing people teams and motivating of building categories capabilities, developing process four the to from known levels The maturity efforts. five training continuous or em- strategies example, ployment for through, workforce corporate improve how policies assesses It management. resource human P-CMM 2010). (CMMI, levels capability capability overall individual compute the that from rules few a by termined or management, risk management, configuration areas process are of Examples levels. capability individual so-called a model with model maturity staged its CMMI nypasfrtenx e ak,ivle h customer the involves tasks, team few development next agile the for an plans this, only to inspec- code opposed source As (1976) Fagan tions. or plan, IEEE test documented 829 extensively Std. an to testing 9000, to ISO according documentation process speci- software requirements fications, extensive of generating the 2007), the or model Waterfall XT the model are processes such exam- Typical of quality. ples internal to lead processes mature knowledge. documenting on focusing development plan-driven heavy-weight light- and between opposites of clash a is possibly there as if considered be often may documentation development, agile in But documentation. is shortcoming This 2002). (Boehm, knowledge tacit de- if risk a be can knowledge the tacit on team’s documenta- only development resulting Relying the development. plan-driven on than value tion less put they that is develop- agile and management Quality 3.2.3.4 P-CMM CMMI ned ti h iwo S 161ta ev and heavy that 9126-1 ISO of view the is it Indeed, hr ifrn rcs ra ahhv their have each areas process different where , sacniuto fthe of continuation a is ) satase ftesae fthe of stages the of transfer a is ) ment aaiiyMtrt oe Integrated Model Maturity Capability ihisintegrated its with h epeCpblt auiyModel Maturity Capability People The CMM waste CMM rcse (Kneuper, processes CMM h usinis question The . htaugments that r applied are continuous CMM 65 to V-

III — STATE OF THE ART Shaping theforce work- Organizational Competency Man- agement Staffing Workforce Planning RT A Motivating and managing performance Organizationalformance Per- Alignment, Competency-Based Practices, Career Devel- opment Compensation, Perfor- mance Management, Team-based Practices Work Environment In a software project there are many ways to reduce Quality management is a set of activities performed With the rise of agile development, these standards Continuous Workforce Innovation face the accusation of notto being adequately be flexible too but heavy-weight.quality is Yet recognized their by experts. benefit There for is a internal need for light- of a product like software.aged, Before it quality needs can to be bemodels man- modeled. out There of exist which several ISO quality as 9126-1 broadly has applicable. established itself While highfeature quality of is a a product desirable becausecosts it can and reduce make development customers happy,saries it in has project reality, several ranging adver- fromthe inner model conflicts to in management, and motivation of developers. risks for internal quality failures,of i.e. quality. the An negative costs important waymunicative is features to strengthen of the software com- represent process documentation and artifacts source that code. to improve the quality oflevel. software on Quality an management organizational standards havetion a long in tradi- industrial production butapplying there the are conventional standards problems to in softwarement develop- because development is not aprocess. production but Instead, design specific standards werethe developed software for domain. once again seem to be at odds with development. They weight support of internal quality for agile environments ROCESS CATEGORIES P TATE OF THE III — S Building teams and culture Participatory Cul- ture Communication Team Building Figure 3.7: P-CMM key process areas and levels (Curtis et al., 1995) Developing capabilities Coaching, Personal Competency Develop- ment Mentoring Competency Develop- ment, Knowledge and Skills Analysis cation Training, Communi- ATURITY EVELS Mainstream development often uses a combination In a survey with software engineering experts, Turner M L Level 2: Repeatable Level 1: Initial Level 4: Managed Level 3: Defined Level 5: Optimizing 66 development foresees. 3.2.4 Summary Quality is a complex concept,ent often perspectives. viewed from It differ- is an inherent but important property to the requirements of thearea allows environment. developers But to easily this takefor gray refuge not in writing excuses documentationdo (Boehm, not do 2002). as much They by documentation plan-driven as development would but be at needed thethe same counter time measures avoid like pair programming that agile Prause and Durdik (2012) reportmalities that is neglecting not of regarded for- asment, the and essence that of a agile lack develop- a of problem documentation in is agile indeed environments. seen as of the light- and heavy-weight practices to better adapt pair programming for source code reviews. and Jain (2002) find thatopment both and concepts mature of agile processesperfectly devel- but are also far that they fromSpecifically are matching not investigating complete into opposites. the issue of formalities, to learn requirements as development goes, spreads soft- gramming, tests with automated unit tests, and relies on ware knowledge in frequent team-meetings and pair pro-

III — STATE OF THE ART ihtsigteqaiyo otaepoutaccording product software a of quality the testing with etn evsteproeo eiyn h nert of integrity the verifying of purpose the serves Testing .. a ftesting of Map 3.3.1 activities. of scope also therefore is ing control quality Another software for characteristics. name quality defined previously to concerned is It management. ac- quality the software of of part executive tivities the is control quality Software Testing 3.3 research in development particular. in software environments for and general, in raiainsbsns ol n customers. and goals development business the organization’s on depends This characteristics quality sub-characteristics. individual or on focus strat- also The evolvability. may its egy than critical the less of are malfunctions software if software of quality external over quality specific out pick strategy evaluation. the for aspects of part activities testing are Individual that others. over certain goals emphasize may quality but scope characteristics full quality the all cover of can strategy of testing quality The the evaluates software. and assesses generally more but 9126-1). ISO (cf. proper- characteristics its quality of or one ties just beyond is goes Functionality that functionality. software pure the of image an draw requirements together non-functional and require- functional its But in envisaged ments. were that functions correctly the does perform vali- actually of software process developed the that as dating seen be can testing Historically, testing of Scope 3.3.1.1 techniques. (2005) testing Lewis of overview applied. detailed is a that presents approach the they scope to the and to test, according classified be Testing can requirements. techniques its fulfill software the correct if in or results execution outputs, if together, fit implemen- that is parts example tation An base. knowledge project’s the h etn taeymy o xml,fvrinternal favor example, for may, strategy testing The requirements, of fulfillment validates only not Testing lhuhti empsil ugssatonarrow too a suggests possibly term this although , I- Testing — III-3 test- aultesting Manual sopsdt hs eeoe unn nttsst test to tests unit running developer a this, to opposed As Lws 2005). (Lewis, hrfr nutyadrsac u o fefr into effort of lot a put research and industry Therefore aiaeta tpoue h xetdotu n there- and output expected the produces it that validate n fbakbxadwiebxtsig nttsigis testing Unit testing. box white and box black of ing structure. internal tests expected but and input information use output not testing does box that White opposite the box. denotes black a considered, as workings not regarded inner are therefore output The and into input known. the is transform software that of piece a output expected of and input testing where box approaches Black separates describes that categories. two criterion into a techniques is testing testing box white or Black box white vs. box Black 3.3.1.2 that so ing In available. particular, are techniques definition testing by fewer are executable, artifacts not other as However, may tested. documentation be and requirements like artifacts cess ssiltepeoiatfr ftesting. of form predominant the testing still manual available, is (yet) not are tools automation with testing results. automating test consistent produce also will It automa- up. the set once is testing much tion manual is than it faster that and advantage cheaper great the has testing tomatic of example an is algorithm, same the testing. manual of example an is correctly, works fore to values input different implemented with an times running several developer algorithm A tester. human a automatic vs. Manual 3.3.1.3 techniques. testing structuring of for discussions used following not the therefore is that It testing next. static discussed of vs. are dimensions dynamic and two automatic, the vs. to manual criterion a as are orthogonal they categories, not two into techniques testing separate expected and 2005). (Lewis, data output input using while tests structure it because internal testing box gray considered sometimes rybxtsigi aetr htdsrbstemix- the describes that term rare a is testing box Gray pro- software other also code, source the to addition In aultsigi h rgnlwyo etn.Au- testing. of way original the is testing Manual to suitable are testing box white and box black While documentation correctness stsigta seecsdmnal by manually exercised is that testing is eoe adymeasurable. hardly becomes CASE pel ohmnunderstand- human to appeals ol.Bti ra where areas in But tools. uoai testing automatic 67

III — STATE OF THE ART continuous in- value for source code. Au- test coverage system so that automated tests get executed RT A Automatic dynamic testing is also only marginally Typical examples of automatic dynamic tests are unit In addition to the test plan, also the conduction and tions can be tested automatically. important here because execution behavior is related development. In larger apart scale, of manual the testing quality can controlthrough be processes before that being a shipped software to runs procedures a are customer, and precisely where defined asHowever, part manual of dynamic the testing test is plan. mentionedcompleteness. only It for is not further relevant. 3.3.3 Automatic dynamic testing software without the need for a developers intervention. report stating that the testsor were that completed there successfully were somedynamic problems detected. testing Automatic can betegration made part of a regularly or in response to some development event. tests. The ratio ofexercised by parts the of unit tests thenot, compared source result to code in those the that that were were tomatic dynamic testing tool kitstest allow to various automatically other aspects ofuser interactions a can software. be recorded For andgraphical example, played user back interfaces so of that applications or web applica- 3.3.1.6 Test documentation process. Before testing is done,tation it that is describes planned. the Documen- approachtest to plan. testing It is contains called information the of on testing when to and apply what when, kind results and obtained what is through to testing. be Aa done guideline test with for plan the writing document is given by IEEE Std. 829. results of tests are recorded in documentation.reports of At the the conducted end, testing areDutoit, created 2000). (Bruegge and 3.3.2 Manual dynamic testing that developers do when they developfix a a new bug. feature It or is also what Wray (2009) calls code-and-fix Testing is documented at various phases of the testing Automatic dynamic testing exerts its tests of the running An example of manual dynamic testing is debugging When the testing has completed, developer receive a TATE OF THE III — S . Integration Static testing builds. integration testing . tests the full system of software (or is testing in the classical sense, i.e. exe- at program unit level, i.e. class level in static analysis continuous integration Finally, end-user or validation testing ensures that the System testing The next larger scale is But dynamic testing does not always require the least The advantage of dynamic testing is that it can show Management also tends to underestimate the value of unit testing 68 tickets after successful payment. system does what customer and useruser expect testing from is it. often End- regardedbut as a is part listed of separately systemmodel testing here with to its reflect separate “quality the in ISO use”. 9126-1 and performs tests at that level.be A typical example would parts of it) in conjunctiontems with it hardware is and supposed other to sys- that work a with. ticket An vending example machine is is testing indeed issuing printed and Dutoit, 2000):is The smallest yardstickobject-oriented of languages. testing testing puts the different parts of the software together ing techniques can prove thatnamic there testing often are cannot no prove errors.it that can there Dy- only are increase no confidence errors, that there are none. 3.3.1.5 Testing scales a given input. Dynamicthat testing requires is less often effort in theexecute preparation. technique the It software means or to part just thereof. effort. Sometimes it may be more costly to run a soft- analyzes software without executing it (Lewis, 2005). static testing. Note that atesting more is common term for static that a system actually does what it is supposed to do for both necessary to detect weaknesses. 3.3.1.4 Dynamic vs. static cuting the software to analyze its behavior. manual testing and tend to give it to unexperienced de- because experience and knowledge of the software are velopers. Bruegge and Dutoit (2000) call this a mistake Testing can be conducted at various yardsticks (Bruegge The syntax checking done by compilers is an example of ware so that a static testing is cheaper. Some static test- Dynamic testing

III — STATE OF THE ART aewiete aecmet bu oeta prob- potential about comments make they while ware iuleaiaino eitosfo tnad.A standards. from deviations of examination visual htwr eetycetdb nte eeoe.The developer. another by created artifacts recently process were software the that at one review, look a developers In more or technique. testing major a are Reviews Reviews 3.3.4.1 represented foremost reviews. and through first is testing static Manual testing static Manual 3.3.4 noticing. without something break they that too fear without much code test source High change quality. developers internal lets of coverage unit expression an from also coverage is test tests However, quality. external to ih emse Rgye l,2008). al., et (Rigby missed be might ( first then-Review committing while development, down ( repository revision to the committed are A modifications before efforts. conducted additional review high too ap- cause might Different Formal inspections speed. development development. affect reviewing into to proaches integrated is it how artifacts. process software some at look invited quick, of is shoulder piece method the a informal rework An jointly 1976). the to (Fagan, time code at same meet the at people place several same which in review technical A 1028). Std. (IEEE lems soft- the through team the a of in members Finally, leads use. developer its for product the of review nical An party. third a by ducted where niques 1997). Tjahjono, and (Johnson cooperation and moderation preparation, with the associated of overhead because additional costly are reviews Tanenbaum, code However, 2010; 2001). al., secu- et into (Bernhart doors software trap relevant rity or bombs logic programmers introducing deter from or issues, style coding Furthermore, detect these reviews of 2006). cost Moonen, and and fatality (Boogerd the problems reduce to important arti- problems is Detecting these early 2003). with al., problems et (Ciolkowski of early 90% facts capable to are 50% they detecting that is of reviews of benefit and purpose h ido eiwn ehiu ple eed on depends applied technique reviewing of kind The tech- testing static similar of family a are Reviews a edneosa neial changes undesirable as dangerous be can ) sasseai vlaino h suitability the of evaluation systematic a is nwihaclegeo eeoe a a has developer a of colleague a which in audit sa needn xmnto con- examination independent an is Review-Then-Commit aa inspection Fagan inspection walk-through sathorough a is okn over looking saformal a is a slow can ) Commit- I- Testing — III-3 tech- the gl eeomn and development Agile significantly to messaging instant and interfaces web eicto fsfwr nwr h usinwehra whether question the answers software of Verification 20)so h fetvns fpi rgamn as programming pair of effectiveness the show (2007) h arprnrmksdvlpr epmr fthe of more keep developers makes partner pair The oe(ry 09.Advlprwowie low-quality writes who source developer A write 2009). will (Wray, they code how about made they promises 2000). Kessler, and increased (Williams for developers reason pair a by as quality named been has and gramming technique. review a Phongpaibul like Studies effects. review-like with nique programming pair programming Pair 3.3.4.2 reviews. of problems time and distance reduce like communication of means modern utilize like plug-in, tools However, time. same the Crucible at place same the in tnad Pogabl 07 ib ta. 2008). al., et Rigby 2007; quality (Phongpaibul, high standards very expensive meet to very allows a however, is that, technique it because and targets only, it quality because external here relevant software. not the is in verification non-existent Formal are to errors and certain specification, that its show to that according proving works at software aim a that checking) model sim- (e.g. or ulations built. approaches be mathematical to are intended verification was Formal what is actually software built verification Formal 3.3.4.3 of 2007). teams (Phongpaibul, with developers experienced especially pro- expensive, pair also However, is gramming 2009). (Wray, code developer source one the in than problems notice potential will and developers details two more And team. tacit the of in spreading knowledge enables off paired of un- are alteration developers own continuous how his The matter. deepens the it of derstanding what programming, about currently talks is developer a he When reviews. to ilarity he although reputation. colleague, his team risks a better, knows of front in code source 15 14 13 eiwn rdtoal novsa es w people two least at involves traditionally Reviewing arpormighsfrhrefcsbsdsissim- its besides effects further has programming Pair http://csdl.ics.hawaii.edu/Tools/Jupiter/ http://smartbearsoftware.com/codecollab.php http://www.atlassian.com/software/crucible/ 13 CodeCollaborator , steronnto fatsigtech- testing a of notion own their as arpressure Pair XP npriua,hv introduced have particular, in 14 rteJupiter the or , rssfo arpro- pair from arises 15 IDE 69

III — STATE OF THE ART LOC ) that is obtained by LOC If measurement is used to The most famous and probably oldest RT A This research depends on the quantification of soft- software metric is lines of code ( simply counting the number of linesartifact, of module source or code software. in Typical an derivations of lead to top efficiency, or haveand devastating effects Pfleeger, (Fenton 1997). Umarji andmetrics Shull have (2009) to argue be that usedline carefully. the Although they importance under- of boththey product warn and that process metrics, metrics maydevelopers’ careers. be perceived Also, as metrics threatstell the to might truth. fail And and finally,tives not combining is metrics problematic with because incen- developersto might manipulate attempt their numbers toet obtain al. incentives. (1994) Perry note thatdevelopment measuring is the output particularly of complicated software something because intellectual, it not is physical. Still developers are taken. Section 3.5 comes backmeasurement, to motivation issues and of rewards. performance 3.3.5.2 Software metrics Software metrics measure some attribute inprocess a artifact software or across the whole knowledgemeasuring base. internal quality of softwarebased from point a of product- view (Kitchenham and Pfleeger, 1996). Lines of code matter, e.g. space and timeand complexity the of an goodness algorithm, of itsitself approximations. may be The difficult attribute tois measure, e.g. subjective; understandability the measuringand device be must consistently be handledinfluences correctly. accurate must Environmental be controlled to come to exact results to combine both values intoIn one the comprehensive case score. of qualityjectivity, with expressing its it many as aspects a and singleproblematic. its number sub- Component is values particularly may beently weighted and differ- lead to thehave to limitation be that treated obtained with numbers care.not Yet required exact and measurements rough are approximates suffice. Caution when measuring determine performance of teams or parts thereof, it can They are an objective and context-independent way of (Fenton and Pfleeger, 1997). willing to be measured if appropriate precautions are ware process artifact quality and quantity, and it needs TATE OF THE III — S is the process by which numbers or sym- A typical attribute would be the size of a software expressed as such a value.ming language For is example, an the attribute program- ing of source it code as but express- aare number suitable is for calculating senseless; withdefinition also them. of not a Sometimes scale all the may numbers several be simpler difficult attributes because down it to projects a single number but so that they can being expressed as a a measurement parametric scale valuecommonly (e.g. us- accepted bytes). clear definition Yet formeasuring there the or is process what no of it consistson of. a While precisely exact defined measures measurementcalculations scale and are scientific needed use, for some attributes cannot be them according to clearlyPfleeger, 1997). defined rules (Fenton and process artifact, or thedevelopment. duration Measurement tries of to a quantify attributes testing phase in not always acknowledged (Fenton and Pfleeger, 1997). Definition 35 – Measurement bols are assigned to attributesfacts of or software process process activities arti- in such a way as to describe Measurement plays an important role inquality engineering management. and It is useduation to and assess track the current the sit- progresscontinuous measuring, of to development evaluate through effectiveness ofods, meth- and more. Software measurementsential has to become good es- software development, although this is static testing has hightherefore more rates important of as a false wayfollow-up of positives, activities prioritizing (e.g. and manual Boogerd is and Moonen, 2006). 3.3.5.1 Engineering, measuring and metrics for this research because itabout allows to the draw internal conclusions qualitystatic of testing source basically consists code. ofstatic software Automatic analysis. metrics and Metrics quantify some attribute of code, passed. Static analysis finds probablefor errors error by patterns looking in source code. However, automatic 3.3.5 Automatic static testing problems without executing it. The concept is interesting 70 weighting them against each other may be a subjective which may point to errors if threshold values are sur- Automatic static testing evaluates a software to detect Measurement

III — STATE OF THE ART Cuttenme fipeetddansi functions diagnostic implemented of number the “Count ihu nu aaadwtotatal xctn pro- a executing actually without and data input without aubefrqaiyassmns rade l (2000) al. et Briand assessments. quality for valuable ae(96,btJye l 20)soe ti obetter no is it showed (2009) al. than et Jay but (1976), Cabe is code source of metrics quality quality code Source 1997). (Rosenberg, metrics such for covariate as metrics, size rather are than metrics such instead predicting But bug metrics. several with correlations significant has it counted. not if or instance counted for are definitions, comments these in ( variations code further of are lines thousand or lines, ( empty Code of Lines Source are n no o-ofratcd noarvso repository revision a into code check- non-conformant disallow of to in used ing occasionally is example, analysis For static checking. style onerous Checkstyle from like for humans or spare coverage 2009), Visser, test and estimate (Alves code can source Großmann, analysis 2006; Static Moonen, and 2009). (Pl Boogerd analysis 2008; risk al., support et and code, source evalua- of gener- expert-centered tion to guide used reports, be preliminary can ate analysis This 2008; al., 2005). et Copeland, (Ayewah quite inconsistencies logical is or run-time tools violations, errors analysis security like static topics of including large, spectrum The gram. code source in problems potential finds analysis Static toolkits analysis Static 3.3.5.3 diagnostic of specifications.” number in the required to functions it compare and specified as to recommends 9126-3 ISO analyzability, predict exam- to be- For ple, relationship indicators. the implemented and to specified refer tween usually metrics quality. these in-use and But external internal, for metrics software exist. hardly do asquality directly internal quality. for sessing external toolkits of implementations i.e. and detection, Metrics fault on focus Chi- metrics is by (1994) metrics Kemerer the and A of damber 80%. implementation of an correctness with a at toolkit classes faulty metrics of best 90% the detect that find They metrics. various surveyed 16 LOC h eodt orhprsfloigIO92- list 9126-1 ISO following parts fourth to second The http://www.spinellis.gr/sw/ckjm/ LOC LOC a rpsda rdco fqaiybecause quality of predictor a as proposed was oa hr r ayohrmtista are that metrics other many are there Today . en ult erc Still metric. quality a being ylmtcComplexity Cyclomatic SLOC ckjm aosmti for metric famous A htde o count not does that ) 16 oee,most However, . LOC KLOC svaluable is .There ). yMc- by I- Testing — III-3 osch ¨ Zeee,21;Vsihh n ut,20) h im- The 2008). Gupta, and Vashishtha 2011; (Ziesemer, ad 09,adcekcdn tlslk h u coding Sun the like styles coding check and (Love- issues 2009), readability land, on concentrate to tends al., It et 2010). (Haderer tools assessment quality mature several Checkstyle code. copied detect to capabilities include detection them clone for of toolkits Several analysis languages. static programming more many are There 2008). (Ayewah al., problems et identify code. to code source byte C# Java for analyzes program similar 2008). a is Smart, 2005; two (Copeland, the between projects cross-fertilization patterns, active fault been for has looked there PMD on while focused source styles, more checking analyze was both Checkstyle Traditionally, that code. Checkstyle pendant its and was quality 3.2.2.3. code Section source in for discussed styles coding of portance ult crsteerm h rbe sta eut can results that is computes problem and The tests), therefrom. using scores unit quality smells dynamic (and code testing counts static and a Sonar page. detects shows web single automatically 3.8 a on the facts Figure project summarizes important that most dashboard project. Sonar’s of the total screenshot the in of debt overview an technical and code, source amount software duplicate the of of characteristics, several to indications regard with rules, quality coding most indicating violated figures are often there reports, project the generate In to reports. metrics and toolkits analysis static tools Meta-analysis metrics. complexity code source and JavaBean practices Enterprise best to for- conventions code naming from and and range matting rules that issues 120 with than dealing standards, more including standard conventions, the with Sun out-of-the-box comes It ignore) violations. info, for warning, (error, levels severity four using annotations. and documentation miss- Javadoc or of code, ing duplicate wrong modifiers, lines, of long order too or class, code formatting a in Identified lines many too 2008). are smells Smart, 2003; source (Edwards, of that code understandability smells and code readability of detector the a hinder is it such, As conventions. 18 17 te elkonsai nlsstoktare toolkit analysis static well-known Other hcsyelt rfie ecsoie ocekrules check to customized be profiles lets Checkstyle http://findbugs.sourceforge.net/ http://stylecop.codeplex.com/ hcsyei oli n opnn in component and toolkit a is Checkstyle oa noprtsdifferent incorporates Sonar FindBugs StyleCop PMD 71 18 17 ,

III — STATE OF THE ART RT A Software measurement was purported to be capable Automatic dynamic testing is interesting with respect Static testing and measurement can be used to quan- of leading to top efficiencyto in have devastating a effects. development The team,tions backgrounds or of and the condi- two outcomes are discussed in Section 3.5 to this research as a providercould of contribute test coverage to data, assessing which internalsearch quality. adds another This reviewing re- methodones to that the aims surveyed at cheap, implicit,terlaced distributed, reviews. and It time is in- comparablenot to keeping pair a programming: promise toaffect write the developers’ clean reputation. source code will tify quality with some limitations.assumption There that is style the violations general and code smells are in- ity can be approximated throughthe violation quality densities. model As itself is not the focus of thisa research, pragmatic approximation of internalmodel quality. could Still, the be improved bymetrics including as other well software (e.g. as is done in Sonar). versely correlated with internal quality, so internal qual- violation densities obtained through Checkstyle provide TATE OF THE III — S Figure 3.8: Sonar dashboard with a project summary ¨ osch et al. (2008) present a similar tool. It integrates Pl that help to assess internaling quality. Internal is quality mostly test- limitedat to class, testing not on integration smaller granularity)techniques and scales which to (e.g. testing necessitate with only fewson effort. is The to not rea- burden developerseffort. with It too will much additional use mostlyand automatic manual testing testing where techniques, results are cheaply available. and rework is advisable.lacking A is feature the that ability such to associate tools quality are with developers. 3.3.6 Summary provides an objective approximation ofa quality clearly based defined on model. and combines the results ofinto various a static configurable testing quality tools model.then The guides output experts of to the source tool code where manual review only be an approximation ofquality understandability characteristics. and In other fact, thenot quality perfectly model reflect might personal quality perception. But it 72 The survey of related work presented testing techniques

III — STATE OF THE ART ni ta. 06 imrane l,20;Hattori 2004; al., et Zimmermann 2006; al., et Anvik of amount the of quantification a is and ownership with about knowledge have might who identifying of ways Mcu n eblb 02 ni n upy 2007; Murphy, and Anvik 2002; Herbsleb, and (Mockus .. otiuin nweg n owner- and knowledge Contribution, 3.4.1 research. this for relevant highly is is which treasure artifacts, developers such and between One relations years. contribution last identifying the in its niche outgrown since research has trea- projects. repositories unlifted for software benefits of the their Mining to for repositories paid such is in sures attention more archives. mail and or More repositories software revision of Examples are in. repositories stored and are artifacts they process repositories software the of that analysis research the of with field deals a is repositories software of Mining Repositories Software Mining 3.4 nohrfie rbace)o h otae l fthis of All software. the of branches) or files other (meaning parts in other in developments when parallel incomplete ignoring is information the evolution On hand, repository. the other into certain introduced why was is or code it when source whom, hand, by one determine the to on impossible information, time lim- Without are approaches ited. slicing that both branches dimensions, without further Even add artifact. an of evolution the mo- or discrete time, one in at ment configuration software the at looking either ap- space Many the slice steps. plications discrete in space this help through repositories navigating Revision revision from 2003). file (Spinellis, revision) each to of directories (evolution diverse time and in files) (code and space dimensions: two in 2010). al., et artifacts certain about has developer a that knowledge and synonymous ownership is for Contribution basis the relations. is responsibility This artifacts. certain develop to started mining has the discipline need, repositories this software and of of Out expertise, knowledge. most much the share have can them, with to problems qualified best fix probably are artifacts about process commu- knowledgeable software best are who to Developers whom with. with nicate effective, know be to to communication need the developers for order in communicat- But time their ing. of 70% to 50% spend Developers orecd trdi eiinrpstr smanaged is repository revision a in stored code Source ship horizontally hc en okn at looking means which , vertically I- iigSfwr Repositories Software Mining — III-4 hc means which , ipemto st oka h rae certain a created who at look to is method simple A httx a netdit h e eiino narticle. an of revision new the into inserted was text what the show also normally can tools Such revision. what eso oto r ls n htudtsaecommit- are updates that and files, are control version ehd r rvddb eiincnrlssesfor systems control revision by provided sophisticated are More methods contributors. different by changes incorporating revisions, several where through work, evolve collaborative artifacts and far iterative is in method inaccurate too this However, artifact. process software facilities contribution tools’ Standard 3.4.2.1 arti- process software which facts. to determin- contributed for has systems who control ing revision mine to differ- ways with ent up come have researchers and Practitioners evolv- in contributors Determining 3.4.2 matter. non-trivial a in- contribution formation for repositories software mining makes e oterpstr nsotitrass st eas be to as so intervals short in repository the to under ted put objects the that they assume artifacts only the They about manage. assumptions weak make only for columns artifact: three an are of There line each 3.10. Figure in depicted running of command output The line revision. the by the contributed committed been who changed, having developer or as new considered is is line it a then If file. a of two revisions subsequent comparing from information Subversion contributor whom. its by derives and added number) was (revision it edited when artifact or process software a of line each command equivalent its showing MediaWiki of screenshot a ex- is For 3.9 artifact. Figure ample, an of revisions two between committed differences who record that wikis, by or code, source teata ieo text. of line actual the 3. and change, last the committed who developer 2. changed, last was line the when revision 1. esoigsseslk popular like systems Versioning command the provides Subversion n artifacts ing v blame svn v praise svn CVS v blame svn nasuc l is file source a on htsosfor shows that ) rSubversion or (and 73

III — STATE OF THE ART RT A does not recognize when linesearlier are restored revision from but an insteadnew. considers them all as fields in a tracker) andidentify therefore complex changes, cannot correctly cannot detect if text isthe same just file moved without around actual within modification, does not consider moving/copying ofent text artifacts, to and differ- • • • for some file with Java source code TATE OF THE svn blame III — S Figure 3.9: MediaWiki showing differences between two revisions Figure 3.10: Output of does not work with other kinds of artifacts but only is line-based, i.e. a singlere-attribution character of change the leads whole to line to a newignores developer, structural information (like abstracttrees syntax built from source code, or differently typed only considers an artifact’s immediate predecessor, with checked-in files, • • • • and useful change information thereforepre-processing involves heavy steps (Omori and Maruyama,functionality 2008). of The the Subversion methodit is limited in that precise and small as possible. Obtaining fine-grained 74

III — STATE OF THE ART hr uhseicto sntaalbe eerhlitera- research available, not is specification such where 20)rcr h rcs fcagn h orecode source the changing of process the record (2008) eyhg rcso Rbe n az,2005). Lanza, and (Robbes precision high very rcigdvlpractions developer Tracking by. come to hard or un- anonymous probably malevolent, known, are because developers focus that different assumes a however, it has, It 1997). (Gray al., contributions et their analyzing from artifacts pro- software cess of developers characterizing and termining contributor below. obtaining reported for is information methods of selection informa- A such mining tion. for methods several presents ture projects For re- 2003). is Nordberg, (e.g. who what specification for a by sponsible is there (e.g. i.e. defined policies), explicitly project defined be can Ownership research in mining Contributor 3.4.2.2 he.Cneunl,rcrigcagsa hyhappen they as changes recording hierar- Consequently, flat chies. and partners independent con- several research with multi-national sortium a late Imagine extensions project. the the using into start not might are extensions they the plug-ins; or which available, necessary for the editors install use to might they forget or to might Developers want not 2.4.2). Section tracking also use (see and extensions install dif- to be developers can convince it to organization, ficult the develop- on the Depending by team. accepted ment are and they value their because proven environments have development found software are off-the- Subversion in only like Typically systems control version editors. shelf developers’ the installed all is in software recording respective the that require operations. edit finer-grained from difficult reconstruct be to are would that these operations as complex (2007) large, refactorings often al. to et recording Dig their specialize independence. language achieve developer’s to the in happens it as Maruyama and Omori methods. seman- of a addition on e.g. them level, tracks tic and makes developer a changes reposi- the software into change-based integrated tory a uses (2007) Robbes with information will contribution it generate as to metaphor, able archeology be the never from to away has get goal to The be lost. are revisions in-between like because is repository revision a from hs prahsaelmtdi rciebcuethey because practice in limited are approaches These actions. developer record to therefore is approach One forensics Software sarltdae htdaswt de- with deals that area related a is IDE tietfistestructural the identifies It . IDE iigcontributions Mining otaearcheology software I- iigSfwr Repositories Software Mining — III-4 natxullevel textual a on wovrprom h rae ubro hne na on changes of number greater the performs “whoever eeoe eoe h we fafiea h moment the at file a of owner the becomes developer A h we 0o oefie.Zmemn ta.(2004) al. et Zimmermann files. more or 50 owned who informa- case, best the In created. were revisions whom 21)rcntuteouinb rcigtemvn and moving the tracking by evolution reconstruct (2011) created that developers the to files “attributed (2006a) aeit con l eeoesta aeee modified ever have that equally developers (2010) all al. account et into Ostrand take 2010). al., et (Hattori file” as and changes of frequency the from later and owner”, performs he changes small has of who number developer greater “the the as performed (2009) file Lanza a to and contribution Hattori define credits notices. copyright logs, and is not repository files relation or revision The developer. exists through a either determined and file that source a relation between (1999) “hacked” Holt a and define Bowman provide information. also can contribution code source and repositories revision the in information change informationrecording contribution Provisioned solution. viable is only archeology the through infor- repositories recovering revision that from so mation possible, practically be not may hnigo orecd niisuighs indexes. hash using entities code al. source et of changing Hata entities. code were of code that source the changes to the applied from developers transactions considered only reconstruct but file” a in lines most is Schr contributor artifact. last the main or of the contributor created Only as were considered contributor. that a file by a edited in lines of percentage con- and the (Rahman G by lines 2010). owned its Devanbu, of is most fix contributed bug who tributor a example, process For software a the artifact. of revisions comparing different by of content obtained text be can changes information text tailed from contribution Mining what to similar is tion by and why when, obtained. to be restricted can is lines information Usually, removed and only added Sometimes of is crude. numbers problem often obtain. The is to information change regarded. easy that well is are it operations that Delete is repositories revision shelf learn to tracker issue relationships. an contribution of data structured (2006) al. the et analyze a Anvik as information. tag contribution “@author” of the source proposes (2011) Grossbart file. a h datg fifrainpoiindb off-theby provisioned information of advantage The c 1 + ˆ rae l 20)dfiei stethe the as it define (2005) al. et ırba hne nrlto oteprevious the to relation in changes v blame svn provides. IDE mainstream , nta of Instead oede- More tre al. et oter ¨ c nit. on 75

III — STATE OF THE ART Revision 3 Hzut driiz Afwouz Huv dis2 dis2 Change sets combine several Revision 2 Hzut driiz wouz Af Huv Contribution determination must Revision repositories often have problems RT A Revision 1 Hzut driiz Afwouz Huv dis2 Figure 3.11: Illustration of a revision undoing changes for reconstructing fine-grained changes thatbetween happened two evolutions snapshots provided bysion the control revi- system. Similarity can be defined in many modifications to software process artifacts into one set. local manifestations in the artifact.the If revision not repository, supported again by change informationbe must preprocessed to findon the change change set information setsretrieval vector and first. model, using methods for the distinguishing Based moves from information true additions and deletions2007). exist (Canfora However, knowing et that al., something was moved determine contribution. Undoing changes regard the case thatartifact was some deleted, part and is ofFigure brought a back 3.11 at software illustrates a how process later thethird time. best predecessor revision of is the notchanges of its a direct previous revision predecessor. withproblem a when new looking Undoing revision at is change a sets only. 3.4.3.2 Evolutionary similarity of artifacts Merging the developer of the new revision. Origins of contents, problematic as large commits areet often al., merges (Hindle 2008). Even withoutartifact merges, results simply in renaming all an codeoper being who attributed renamed to the the artifact. devel- Copying and moving This ensures the coherence of a logical change with its The similarity of software process artifacts is important ways, and depends on the type of artifact. This research without recognizing what and where is not enough to with merges: They attribute merge results completely to which may be in somewritten a other long software time process ago, artifact are neglected. This is especially TATE OF THE CVS III — S Simple renaming or mov- Subversion annotates each line automatically reformats code, Subversion IDE does not record name changes of software pro- CVS Data from some revision control systems like of source code with itsit respective is author and there. since But when if,ferent for indentation example, characters two (space developers vs. useand tab dif- their characters), is misled. The same holds true for MediaWiki. ing is normally handled byto moving another article. the whole So strictly content not speaking, even the change artifact its does name.handled But to obtain all correct these information. cases must be White space changes does provide the original nameartifact of was an renamed artifact with when Subversion’s the renamebut facility; only then. It isto inevitable get to correct deal results. with name changes cess artifacts at all, which leads to misinterpretations lack change set informationfacts of that changed software at process theinformation arti- same on time. where an By artifact notprevious came providing names from, it i.e. had, some what preprocessing revision steps. tools complicate Further problems ariseon depending the revision repository and its correct use. Subversion ing of a softwareis process detecting artifact changes is in easy anchanging to artifact. is detect, not But as (Zou renaming andcient plus Godfrey, 2005). to look It is at not the suffi- evolution of a single artifact.developments in Like- branches. Several limitations and problems existapproaches. with the These common are summarized beforebuilding concepts a for better contribution mining are presented. Keeping track of artifacts is required. Thistions view between considers software the process inheritance artifactsbequeath and rela- text how to they each other. 3.4.3.1 Limitations of common approaches 3.4.3 Concepts of contribution mining bution mining tools, first a new view on source evolution 76 To overcome the limitations of currently available contri- (Thomson and Holcombe, 2008). In MediaWiki, renam- wise the picture is incomplete when neglecting parallel

III — STATE OF THE ART An (R done modifications are corruptions The 2001). (Navarro, re(S)s htte dtdsacscnb used. be can distances edit tree that so (AST) Tree tig hl loigfrsome for allowing while strings ones. ancestor other the two is of artifact one where predecessor; common denotes similarity similarity ary Instead, here. similarity needed and content not (Baeza-Yates of is kind words this key But of 1999). Ribeiro-Neto, use artifact the process to software according each to vectors dimensional the is similarity termining characters. special by marked lines structure are only there Their that each. is string often text they single that a in of “unstructured” consist are artifacts are the Also, artifacts out. other left artifacts, text-based with deals only ytxi end o,o hntesna sviolated. is syntax no the which when for or texts too, with defined, work is not syntax suitable does It less seman- changes). is realize tic (to it text work, plain change this developers For because and semantics. text to plain closer of similarity evolutionary from away ther ( Language Markup Extensible be data like to deep is string-encapsulated integrity merging syntactic when if maintained preferable also al., is et It (Neamtiu seldom 2005). exploit- change by names function functions that ing of or evolution 2006), al., structural et tracking (Sager functionality similar with code Syntax Abstract an into transformed be syntax. can language’s strings programming Such implied the structure that through a those them have on edit e.g. code, an strings, source not some valid But represent are they into strings. speaking, of transformed distance Strictly be can one. tree another one how describes that distances edit Tree transform that other. operations the edit into of string cost one the on based is It distance Edit – 36 Definition responsibilities. their give hence and developers, is by corruption exact the what determine to and curred, na n ogof 2012). Borghoff, and onnau h datg fate dtdsac sta ti far- is it that is distance edit tree a of advantage The source finding through reuse enables editing Tree of goal The de- to retrieval information from approach typical A ¨ dtdistance edit ..smlrt htrslsfo aiga having from results that similarity i.e. , prxmt tigmatching string approximate samti o h iiaiyo strings. of similarity the for metric a is A reei distance edit tree etrmodel vector corruption XML I- iigSfwr Repositories Software Mining — III-4 tasgshigh- assigns It . documents ) sametric a is st match to is ohv oc- have to evolution- aepoesatfc a vle ie esr fhow of measure a gives evolved has artifact process ware hl ie nta fsnl hrces ..tealphabet the i.e. characters, single of instead lines whole Uknn 93 loihsfrapoiaigei dis- edit approximating for algorithms 1983) (Ukkonen, otaepoesatfcs ayrvso eoioysys- repository revision Many artifacts. process software strings lar of complexity time developer. a by made been have changes many soft- a which through revisions various the Applying to metric deletions. this or oper- editing to white-space weights like other of give ations to sequence adjusted minimal be the and output changes, to can modified algorithm be Their easily problem. distance the solve to rithm 1966). (Levenshtein, artifact revision process new software a a creating of when copying) or like lines manipulations deleting editing character a mass of (without number strokes minimum key theoretical distance the The to another. equal into is string one transform re- to of substitutions quired number or as deletions computed insertions, is character It single text. plain on distance edit distances edit String iia icso orecd ntecd ae h idea The base. code the in code source of pieces similar analysis, code source In detection Clone 3.4.3.3 2007). al., et information (Canfora with techniques cheaper retrieval text; identified of be can blocks Instead code moved boundaries. moved artifact detect between not to moreover and sufficient not is allow characters), this that two swapping (1975) (i.e. algo- Wagner operations are transposition and there Lowrance Although like rithms artifact. process within parts software text a moved detect to allow would characters 1975). Wagner, and for (Lowrance dramatically results increase correct to proven permitted, is are complexity transpositions time character also if But tances. O to Extensive led line. also the research within changes smaller lines highlight modified to Medi- to recursively well. algorithm as the lines applies aWiki on operates The script size. edit to actual computing strings their of of fraction length a logical only much the it reducing making lines, while entire bigger of consisting as treated is with dealing by issue the solved Subversion like tems 19 ( rbe s oee,ta h loih a run- a has algorithm the that however, is, problem A algo- flexible a present (1974) Fischer and Wagner nldn h rnpsto prto o hisof chains for operation transposition the Including nd http://www.gnu.org/software/diffutils/ ) (where A and d stedsac ewe h w strings) two the between distance the is B hscnb rbe ihlengthy with problem a be can This . O O ( ( | n A The 7 3 | × | ln detection clone ) BrYse ta. 04 and 2004) al., et (Bar-Yossef eesti distance Levenshtein B | = ) O ( n diff 2 ) en ofind to means o w simi- two for tool 19 san is for 77

III — STATE OF THE ART Clone detection is a research Contribution information is a kind models that identify change types and AST is data about data. It adds further information RT A Instead, fine-grained metadata like character-by- While metadata is structured in a machine readable character or line-by-lineneeded. contribution information It is information can that then is overlying be textual considered data. as It an is a additional kind simplest case, it only means accessingitory. the revision From repos- a performancerepository perspective, is accessing an the expensiveIf operation further computations in are its necessary, thenmore own it costly. right. only Contribution information gets therefore needsbe to precomputed and cached to make it easily accessible. Data about data of metadata: Definition 37 – Metadata to individual instances of application data. any media. Metadata providesis a used context to for facilitate datainclude its descriptive and understanding. information about Metadata context,and may condition, quality or characteristics ofrecorded the with data, fine and or can coarse be the granularity. granularity But of metadata a at wholeseveral text authors is contributing too to coarse the if same there text. are changes; as well on alevel crude (Chang file and level as Mockus, on 2008). thealso Developed finer include methods code find similar code (Fluri et al., 2007). Choice of techniques field of its ownand right Jarzabek, (for 2007). several Sophisticated examples detectionincrease methods efficiency see and Basit even addpurposes fuzzy of matching. this research, For somegood simple and techniques efficient are enough.recall, While precision fuzziness is increases probable tocrucial decline. for Yet precision this is research becausea erroneously piece identifying of text asas copied a is copy worse at than all. not identifying it 3.4.3.4 Maintaining contribution information Contribution information is not trivial to obtain. In the way, the data it is about may be of an arbitrary kind in Metadata TATE OF THE III — S To detect the Clone detection is ba- Advanced methods for identifying copied source code Clone detection is used for assessing internal quality, lying structure, and adaptation ofstructure. the algorithm Cordy to et that al.copied (2004) source code find using origins an of islands-and-water metaphor, possibly teresting islands (actual source code) is ignored. rely on fuzzy algorithms to abstract from superficial tural information (e.g. function names)From and the fingerprints. fingerprints they inferinformation merging for and source splitting codeDiehl entities. (2006) use Weißgerber fingerprints and totransactions classify into revision control refactorings and changes.gerprints have proven While their fin- value, they require an under- an edit distance algorithm, thismoved is between not two possible artifacts, when or it copied. is Examples of clone detection methods origins of moved code, Zou andlyze Godfrey characteristic (2005) features first of ana- source code to obtain struc- this research as a waymoved of within tracking a pieces software of process artifact, textdifferent or that ones. between are two For example, if afrom developer a cuts file a and method pastestwo it contributors: into a one different who one, wroteone then it who it in moved has the it. first While place,code the and within detection an of artifact moved could source be solved inefficiently with gorithm is supposed to findare all not clones. whole Moreover, artifacts clones butcreasing only the parts number thereof, of further candidates in- (Kontogiannis, 1997; and for detecting refactoring needs. It is relevant to ing discussion is whether textualof similarity the or system semantics are thejor most important problem criterion. is One that ma- resistant the against detection some modifications algorithm of shouldOther the be source problems code. are that the algorithm does not know the programming language’s abstraction functionality. Overview of clone detection sically an information retrievalcould problem. be Clone seen pairs asother. documents But that there are are fine relevant distinctions. to An issue each of ongo- is that there is onebeen original copied piece to of other source places. codeinterested Software that in quality has finding control such is duplicationssirable as in they source are code. unde- They should be avoided by using 78 where uninteresting water (white space) surrounding in- what source code iswould a have clone to candidate. be treated Every like a artifact candidate because the al- Walenstein and Lakhotia, 2003).

III — STATE OF THE ART Markup (e.g. tfo h broader the from it markup metadata Fine-grained like character every of author this: layer original second the a store then B, could written and was A world!” authors “Hello by text collaboratively the respective if the example, of For authors text. the contains that metadata of fsfwr rcs riateouin(e eto 3.4.2). Section (see evolution artifact process software of through them reconstruct to CollabReview has Therefore, necessary. evolu- is such processes survive tionary to metadata (see Enabling too 5.3). repository, Section revision are the they fact, to In unknown the unknown. normally of are evolution alterations exact stored The the text. of they Moreover, snapshots data. provide text only the containing layer store first only the They MediaWiki. or the Subversion store like that artifacts; for artifacts true process holds software same of The repositories markup. such of managing port editors most that is reason merg- The like copying. changes or and to moving due first ing, lost between get easily associations can the layer second But in layer. characters respective first their the to sec- stick first the should the in layer information in ond Overlay text evolves: the it with while sync layer in layer dif- second a the in keep stored unlike be (i.e. should place markup ferent of layer artifact, second process software the a editing are who developers markup maintaining of problem The code. source the as way is same Markup the deleted. in is edited text the its Analogously when too. disappears so, text markup If is markup it. its with moves duplicated markup gets the around, moves text the characters. of granularity the Markup – 38 Definition akpsik otetx twsatce o When to. attached was it text the to sticks Markup AAAAABBBBBBB world! Hello IDE sue o n-rie eaaat distinguish to metadata fine-grained for used is sfiegandmtdt bu eta aaat data textual about metadata fine-grained is )frsfwr rcs riat ontsup- not do artifacts process software for s) metadata XML ntefloig h term the following, the In term. .Tepolmi hnto then is problem The ). tr :Mtdt layer Metadata 2: Store layer Data 1: Store archeological I- iigSfwr Repositories Software Mining — III-4 ontdistract not To analysis sn rcs it,dcmn iiaiy classifications, similarity, document hints, repositories process different using in objects docu- learning interconnect and (2007a,b) mentation al. et Prause here: sented idea the artifact’s (cf. quality provider’s of the should in developer interest artifact’s an have dependent the provider’s the So by influenced quality. is artifact quality dependent The the inheritance). of (e.g. created modified, is has dependency developer a the that provides artifact artifact the linked to a something When artifacts. dif- of across types span ferent easily relations touched Such directly developer. were the that by ones the than artifacts affectmore often in- contributions densely knowledge, is encoded software terlinked As common. that in something artifacts have process are software there between above, relations discussed other as ancestry shared Besides halo artifact process Software 3.4.4 e fiscnet eae oti eerhaebriefly are research this Only to here. related presented area. concepts its whole of the few to a introduction good a Ribeiro-Neto and contains Baeza-Yates by Ribeiro- book The and (Baeza-Yates 1999). Neto, user the to accessible make rep- to them in information organizing engages and that storing, science resenting, a is retrieval Information retrieval Information 3.4.5 creating instances. and object exceptions; of of throwing and method catching references; calls; field types; imple- return and extends relations; types; ments inner and methods, declarations fields, like of source constructs of language context using the entities determine code (2008) “Pro- al. like et changed...” Alwis hints also de give functions these to changed able who grammers be developer to capture wanting (2004) behavior, al. a et in package Zimmermann the directory following pattern. same related the probably are system) (e.g. file space Artifacts same artifacts. the process sharing relation software hidden two a the at hint between can im- packages in same Similarity the porting code. source Java in packages imported is Schr which link detected. underlying otherwise same some not the be at may happen there changes then if time, dependency: times a change at Also hint names. can concept key for searches or oeeape fsc ao rudatfcsaepre- are artifacts around halos such of examples Some TQM eig 94.Tepoiesmv nothe into move providers The 1994). Deming, halo . tre l 20b analyze (2006b) al. et oter ¨ 79

III — STATE OF THE ART ) tn fp tp + + tp fp using only bi- )( = fn }| fp ). The formula for }| + × tp fn )( − tn represents a striking feature tn + × fn word tp Pearson correlation )( is a data structure for fast searching correctly retrieved fp retrieved documents |{ Matthews correlation |{ + Expressing retrieval quality as precision = tp ( is the percentage of correctly retrieved docu- p RT A = inverted index M precision The correlation obtained from a contingency table is For inserting a document into the search index, it Precision measures if retrieved results are not clut- The inverted index therefore allows to quickly locate r 3.4.5.2 Inverted index all occurrences of a wordsearch in just all looks indexed up documents. the A a respective list word. of occurrences for and recall derived from thefore contingency correlation. table It is contrasts there- trueagainst positives false and positives negatives and negatives (Powers, 2007). a special form of the nary values (the computing the correlation coefficient is (Powers, 2007): of occurrences of words inand documents Ribeiro-Neto, (cf. 1999). Baeza-Yates It is andetection, important where tool each for clone in a software process artifact. needs to be preprocessed.the The words preprocessing from the extracts document,that and maps then words creates to an a index all list. occurrences The of list the contains words. references to tered with irrelevant documents.correctly retrieved It documents (tp) isdocuments, out the which of portion is all the of retrieved summents of (tp) correctly plus retrieved docu- incorrectly retrieved documents (fp). Definition 40 – Precision ments out of all retrieved documents. Correlation and recall is limited inber that of it correctly does rejected not documents, consider the the true num- negatives. The Precision A more expressive value than combinations of precision TATE OF THE n fn tp+fn fp+tn Totals tp + III — S . tp correct rejection = }| }| tn fn fn+tn precision Response - and for false negative. Recall measures if the retrieval recall Both concepts are perhaps best tp fp miss tp+fp n=number of samples correctly retrieved relevant documents Response + |{ |{ tp=true positive, fp=false positive, = tn=true negative, fn=false negative, is the percentage of correctly retrieved documents Totals recall Target - Target + To assess the quality of a retrieval method, the differ- Table 3.2: Contingency table for target and response 80 Definition 39 – Recall from all relevant documents. has not left outof relevant correctly documents. retrieved documents (tp) Itdocuments, out is of which all is the relevant sum portion ofincorrectly correctly not retrieved retrieved (tp) documents plus (fn). for true negative, or ent numbers for true andare false counted in positives the and contingency negatives table.cells is The sum the of number all of four samples used in the analysis. Recall and Precision Contingency table user wants to find a certainis document the or answer not. that “Response” is given to the user, again either yes actually retrieved for him (responsetrue +), positive the (tp). response is Depending a oncell the contents domain, can the have other table names like explained on the basis of a2007). contingency table Table (cf. 3.2 Powers, showslabeled how depending the on axes the of domain.considered the The as “target” table what can are be the “correctitem would response” be, for either a yes given information (target retrieval, +) the or target no is (target usually -). whether In the 3.4.5.1 Precision and recall In order to assess the quality of information retrieval ser- two concepts here are vices, measurements are necessary. The most important Recall (response +) or no (response -). For instance, if the user wants to find a certain document (target +) and it is also

III — STATE OF THE ART aepoesatfcsi eiinrpstre oextract to repositories revision in artifacts process ware h rtapoc a iiain tmigfo devel- from stemming limitations has approach first The osntpoiemtdt nterqie granularity. required the the only in So metadata provide approach not second the does and innovation, to resistance opers’ operations. edit applied reconstructing or revision from repositories, metadata reading operations, editing of tracking the including information, contribution obtaining be- relation artifacts. as and developers responsibility tween is information development. such software in One helpful be can that information soft- analyzing with deals repositories software Mining Summary 3.4.6 otaedevelopment. to software essential col- are motivation, i.e. and engineering, organization software laboration, of aspects Human coordination. and communication continuous pos- through only sible is pro- engineering software software the in Teamwork of activities cesses. all across in tasks cooperate many to the have developers goals, To project achieve work. collaborative highly is development Software Organization Collaboration, 3.5 is it because expressive. to quality more better measuring often for is correlations it on However, rely recall. and algorithms precision of are quality the determining for measurements contribution information. influence further Rela- can artifacts text. markup between artifact’s tions a process as software maintained the be over overlay can it extracted, is information contribution an Once basically problem. is retrieval which information detection, is clone of reconstruction discipline to ar- the Essential between modified. around get move and they tifacts while tracking strings like of manipulations pieces mass are problem But major operations. a edit reconstructing An is artifacts. concept process important software of similarity textual the orwn rmifrainrtivl w important two retrieval, information from Borrowing as defined is artifacts of similarity evolutionary The for approaches different ready has art the of state The n Motivation and archeological prahrmisviable. remains approach I- olbrto,Ognzto n Motivation and Organization Collaboration, — III-5 okrdnaty h ucs fsfwr development software of success The redundantly. work (Jim h pca aueo otaedvlpeti hti al- it that is development software of nature special The eymc eed nhwgo oriainwrs So works. coordination good how on depends much very ... oriain omncto,documenta- communication, Coordination, 3.5.2.1 goal. shared a reach efficiently work more their to coordinate and to order in more through or run two individuals that processes the describes Collaboration Collaboration 3.5.2 essential. are organization larger the in role perform their to individuals motivating and reasonable structures, into social them collaboration organizing allowing individuals, for between techniques and atten- Tools much require tion. therefore aspects engineering Human software 1995). Streeter, of and (Kraut and fail trouble to deep prone in are communication coor- through sufficient dination without projects (Mockus for communicating 2002), Herbsleb, time and their of most spend Develop- ers individuals. many between collaboration involving process close social highly a is engineering Software Introduction 3.5.1 esadns(rne ta. 99.Bta project a as But 1999). al., et (Grinter misun- to derstandings lead communication informal while in informally, breakdowns communicating time spend their of Developers 15% over coordination: to vital is developers coordination complicate distance perceived in and, general, cultural environmental, geographical, 1995). Streeter, also and But (Kraut increase complexity project if and instance size for when difficult, suffers more development gets software coordination that surprise no is it doing without seamlessly together parts its done fit it get to to the and coordinated the be towards must accomplishing Work collectively organization tasks. necessary at an and of goals common parts individuals of different efforts the aligning and of process the is communication tion through Coordination documentation. high and put communication necessities on requirements coordination resulting knowl- The integrating and edge. creating of consists entirely most h esni htifra omncto between communication informal that is reason The nze l,20;Hrse n rne,1999b). Grinter, and Herbsleb 2009; al., et enez ´ inadCSCW and tion Coordina- 81

III — STATE OF THE ART Six The role of doc- keep work going Different Times ing andbackground support Filtering: ofcoordination meetings tools and tion: contact Ongoing Coordina- without face-to-face Administrative, Fil- support for meet- RT A

(see Section 3.2.3.2) that aim for perfection. This

Same Time information technology enhancing activities during inmeeting person ings: ings between distant participants Cross-Distance Meet- Face-to-face meetings:

ifrn Places Different aePlace Same to act”, and “exerts some force or effect” (Farlex, 2012). related works and its rolesoftware in the projects communication can within be foundForward in (2002). the Among master the thesismakes most of is the important importance points of it cation, documentation that for documentation communi- standards like IEEE Std. 829 can only define the contentsbut of not software that documentation, goals havedocumentation to should be aim understood for first, theenough” and quality to that level ensure of that “good projecttain budget lower suffices quality to but main- up-to-date documentationparts for of most the software. Itenough” is quality interesting to is note in that opposition “good to standards like difference only underscores the different goalsmentation of as docu- a means ofproduct communication, quality. and industrial 3.5.2.2 Reputation systems agent Definition 41 – Agent Figure 3.12: Matrixhansen, of 1988) CSCW tools (Bullen and Jo- Documentation as communication umentation is as a mediumhas of communication. been This discussed role inpoint Section of 3.2.2.2 quality. An from excellent theumentation, treatment stand- its of current software situation, doc- different perspectives, Sigma The Free Dictionary (Farlex, 2012) defines the term (1998), IEEE Std. 1074 (2006), ISO 12207 or ISO 6592 An agent is “a person or thing that acts or has the power to- TATE OF THE different . But the same time CSCW III — S and CASE and same place overlay that has been coined CSCW with his same time support for computer-supported collaborative Web 2.0 ) and space ( ) for its use of computer tools to facil- tools are important assets in enabling is on collaboration support while the CSCW CSCW is a multi-disciplinary science that intercon- Software development can be seen as a spe- CSCW ) are used to classify tools into a matrix (see Fig- different times A particular facilitator of distributed collaboration is CSCW Nonetheless there are attempts to bring The only way to reduce coordination problems is to 82 the Internet, and its tivity happens.” (DiNucci, 1999).tions It to enables collaborate organiza- locally andallows over weak a organizations distance, to and form even withoutmeeting people or ever without legal foundations,source e.g. software projects. some open and places ure 3.12). coordination through communication andtion documenta- in software development. Language (UML) figures. nects sociology, psychology and computertries science. to understand It social interactionlearning like to work, support play it or withmensions computer time tools. (with The the two two di- discrete values software engineering implementation. The traditional engineering influences in software development also let muchhappen communication through documentation inlike standardized forms source code, requirements templates like Volere is still done in isolationsiderable by overlap developers. between There isfocus a con- of latter one focuses on supporting the developer. gether with software development. For example, Cook CSCW cialized field of broader Computer-supportedtive Coopera- Work ( itate efficient distributed collaboration. Aits difference is innate affinity to computersupport tools. specifically for Yet software collaboration development iscommon. not Most very development work like implementation inconvenience, and frustration (Grinter et al., 1999). reduce communication problems, and fortool this adequate support is needed,taking particularly place if over distances development (Carmel is and Agarwal, 2001). grows, the number of linesthem of communication communication complexity and increase with exponentially aggravates the cost of communication with difficulty, (2007) demands for (Robertson and Robertson, 1999), or Unified Modeling (Curtis et al., 1988; PMBOK, 2008). Perceived distance “a transport mechanism, the ether through which interac-

III — STATE OF THE ART A presented is systems reputation of analysis and survey A agents help and chaos, into structure bring They way: Reputation iulsoefrec hn raetta sa expression an is that agent or thing each for score vidual itoay 1994). (Oxford Dictionary, qualities or abilities something’s or someone’s Reputation – 42 Definition and (Farmer 2010). systems Glass, reputation their by evaluated and continuously agents are other and actions agent its every of time, effects same the the of At absence the information. agents in better lets anything about Reputation judgments sound efficiently. make life two-fold through a get in it, to of aware not are we if even life, day ery ilcsoesta hycnhv ofiec nreally in confidence have can they poten- that to customers shows tial reputation good Internet, seller’s the a instance, of created. for e-Commerce it anonymous things highly the the from or In agent, certain a from expect qualities. its of subset certain a of indi- an compute can system and reputation rules the internal algorithms, using By through feedback. agents and or observation things about information collecting of system Reputation – 43 Definition term the otherwise, noted system Unless reputation (2007). al. et Jøsang in systems. reputation are eBay mechanism or feedback 2005), transaction (Hirsch, h-index the (Brin 1998), PageRank Page, The and 2010). Glass, other and from (Farmer feedback agents collecting by or observation direct capture They reputation. of kind fined work, this In systems 2010). tation Glass, reputation and how (Farmer determine formed that is practices of set of or consists model it a community, a a need Given not component. does technical system ele- reputation interface a user Generally, or ments. technologies, rules, practices, of 20 euainsystem reputation euainsse hw oohraet htto what agents other to shows system reputation A term The ev- everybody’s influence heavily systems Reputation http://www.ebay.com/ swa sgnrlysi rblee about believed or said generally is what is euainsystem reputation CSCW r pcfial en santceryde- clearly not a as meant specifically are means: sasfwr ytmta scapable is that system software a is ol httyt atr nagent’s an capture to try that tools euainstatements reputation a ecieawd array wide a describe can I- olbrto,Ognzto n Motivation and Organization Collaboration, — III-5 through repu- 20 ’s ileooi gnsivs naheigago repu- good a achieving in invest agents economic will Karma aubet them. to valuable n ls,2010). Glass, called in- and also create is behavior to user used for is centives that This reputation of well-behaving. sub-type to the special agents up open educate thereby to systems opportunity Reputation 2007). an al., to et leads for which cost it, equivalent an establishing entail reputation the good that of concludes benefits theory Economic 1982). (Kreps Wilson, gains if and long-term losses secures short-term reputation endure acquired even his will agent An tation. is reputation good A art. of the lot of a state had having the means on site; h-index influence web high its a on scientist a visitors company, to more a means To PageRank 2003). high Schiff, a and (Kennes money their shop spend his to in likely competitors, more over means advantage are reputation an customers has because good he A that seller the for. to pay they what getting o kl,btbcueo i odesfrwrigon working for fondness his problems. difficult of his because of but because skill, not review; low his additional that an mean get still should can code however, reputation, bad bug-prone A and difficult tasks. select to preference reputation his Gamma’s includes 2011). implementa- Devanbu, an and finishing (Rahman only tion more than are bugs that cause tasks to design likely ambitious plausible on is taken It has he skill. that poor is him code attesting his necessarily in counts not bug from (hypothetical) computed A reputation Eclipse. low into bugs who developers most is the the of introduced Gamma one Erich being for code. renowned devel- better famously skilled write more always that will or skill, opers to the equal that is mean score necessarily karma not does quality artifact cess their in put may attackers reputa- that attacks. efforts good the of higher gain the possible reputation tion, the good higher obtain The to fraud. try by that agents target by the attacks become may of measure itself and system define the to reputation, how good of difficulties mere the sides behavior. user for incentives create to Karma – 44 Definition h ihrtebnfi fgo euain h more the reputation, good of benefit the higher The eainhpbtenkrasoeadsfwr pro- software and score karma between relationship A Be- too. problems, their have systems reputation But srptto bu sr.I ssree norder in surveyed is It users. about reputation is equilibrium karma (Farmer (Jøsang 83

III — STATE OF THE ART , zone: chaos far from Disordered anarchy and cowboy coding old-fashioned political decisions agreement

Self-organizing zone: paradoxical is a structured set of activities that

irregular regularityallow novelty and unpredictable and creativity predictability, dynamics of complexity RT A Standard management zone: regularity, stability, predictability

close to judgment decisions

ls to close a from far certainty A typical example of the problem is However, this area is process-hostile by definition. But processes make an organization inertial and in- business process as a reference to theshooting mythical from American west. the hip It means with no process. The cowboy’s produce a specific service orinstructions product. necessary Besides for the creating actual thecan result, a include process quality assurance activities,and e.g. how state a that result isset to of be rules documented. forof collaboration A production that process reliable, and makes is creates a theinvolved certainty outcome persons. among the It reducesfor the acting individuals’ selfish freedom — whichis according a to Deming big (1994) problem inenables effective the and Western efficient world collaboration. — Nordberg and thereby models require strong coding styles. flexible. Therefore agileorganizing development teams. relies However, self-organizing ontions organiza- self- have problems to establishachieving processes, quality. i.e. Quality rules cannot for berules achieved through in such environments.common sense, People and have they must toabout be use able the their to perceived communicate quality oftamed artifacts. by motivating Selfishness people is to showbehavior. cooperative (social) Figure 3.13:organizing Stacey and chaotic diagram zones (Stacey, of 2012) conventional, self- (2003), for example, mentions that weaker ownership A TATE OF THE and an III — S principal of self-organization organizational culture Stacey diagram is emergent and adapting to new chal- is a social system that has relatively . The principal offers compensation to an agent in organization ¨ ohlich, 2002). The more the two parties’ goals differ, the bigger the 84 lenges; it is nearoccurs this (Powell, edge 2006). of This chaos issearch, especially where where important innovation there in re- is poorchallenges agreement and on proposed ever-changing solutions areFigure highly 3.13 uncertain. shows a in the middle between order and chaos. 3.5.3.1 Self-organization Humans are complex systems: onecomplex, mind but alone put is together highly ina groups, greater, many even beings more form complex organization.cope Creativity with to problems, however, occurs mostof at disintegration; the in the border zoneand between stable the operation, reign of unorderly instability (Stacey, 1996). processes. Implicit and explicit rules,bols norms characterize and the sym- so-called long-enduring and definedIt objectives has that a it formal pursues. structuregroups that consisting have of roles, individuals and and it has behavioral rules and also only have fixed-termas contracts) the can agents be thatinterested regarded pursue behavior their is favored own by goals.of weak the internal temporary Their structure consortium self- organization thatthe represents principal. 3.5.3 Organization information systems that hamper deception (Eisenhardt, problem. If the relationship isless only likely temporary, to goals be are aligned.partners In and a their consortium, employees for (who instance, themselves possibly agent exchange for his efforts. Aferent conflict expectations stems and from reasons the that dif- have principal for and entering agent into the contract.agents will To a pursue certain their degree, owntheir goals principal. and thereby Two important deceive propositionsception state can that be de- reduced by aligning of goals, and through 3.5.2.3 Principal-agent problem Originating from information economics,ory agency deals the- with the contract between a Self-organization (Fr An 1989).

III — STATE OF THE ART hr aymr atr ieifrens,probability, informedness, like factors more many where theory Game Zmad n erg 03.T elwt hsfact, this with deal To 2003). Gerrig, and (Zimbardo 2008). Eisenhauer, and (Prause h bec flw n hrfs(..poess leads processes) (i.e. sheriffs and laws of absence The n lrimhv ensont eprl inheritable partly be to shown been have aggression altruism example, and For modern shows. as psychology brains in their research into hard-wired genetically be to behavior cooperative as motivates act that to measure argued counter a is ownership per- and However, behavior. responsibility sustainable sonal but all and selfish to ownership. code weak with environments in quality nal are disdained reviews, and rules coding like documentation, programming comments, cooperative of team-play aspects pushes Social artistry aside. Ingenious hacking. on is focus nawnwngm rhrsi,dpnso h ue of rules the on depends it, harms or game win-win a in self-interest. pure their of out community the (1776) serve Smith the profit. this own calls their some by for forced but do are entity they not planning because do or they benevolence But for it the fellow-citizens. create baker by of the division desired and the goods butcher is the example brewer, One the group. labor: the therefore for and win efficiency a overall to optimized to lead can called be would game game a Such loss. one’s other decisions. actual influence utility and reality, of simplification human great describing a is at it successful although is decisions theory (Brockhaus, The strategy suitable 2005). a optimize pursuing to by tries payout player) his (the game a of participant called Every situations de- conflict economic, and in concurrency behavior cision, has human describe It to used processes. been decision deterministic and strategic design Mechanism 3.5.3.2 system the letting while game, itself. the organize can change they to But try apart. falling also from system processes, the through prevent control and more exert to man- zone standard agement the toward closer move can organizations 21 hte niiulsl-neetsre h community the serves self-interest individual Whether an- is gain one’s that games of necessity a not is It seem traits character Many are. they as are Humans e also see nta h lyr’srvn o esnlgain personal for striving players’ the Instead . 21 http://c2.com/cgi/wiki?CowboyCoding ssc,cwo oigi hett inter- to threat a is coding cowboy such, As . niil hand invisible samteaia hoyfrmodeling for theory mathematical a is hc ae individuals makes which , I- olbrto,Ognzto n Motivation and Organization Collaboration, — III-5 zero-sum games . hnteeaetopaeseggdi neconomic an in engaged players two are there When eoe omiti h orecd nago hp so shape good a in code source the maintain to veloper o e. h rbe swe h oe ftegame the of model the when is problem The get.” you vroeo hi osbti o ekesysls oto so selfish recklessly not is but toes their on everyone termed Mix- been tition whole. has a competition as and market cooperation the ing in survive better to compete of kind or outcome 2005). desired (Brockhaus, a game achieve can to system regulated market a be to admission how with concerned actions players’ side-effects. the hidden or cause times, affected, other is But resource secondary resource. com- a obvious simply an players for Sometimes directly life. pete aspects real all in understand game and a analyze of to difficult community. often the is serve It self-interest mechanics make game to The designed are community. project the serve to de- the obligates code source for ownership the project, that states 14GG) obligates ship Federal (Art. the for Germany Law of Basic Republic the example, For game. the nasottr oslatt mlmn oepr fa of part some implement to bring consultant might short-term company a a in exam- problems, For schedule to 2010). due Melvin, and transaction ple: (Boyes the place in involved first not the was in that bore party typically third is a It not by is transaction. that the players in two included between explicitly transaction a of cost the called is actions system. the entire on the and of players, performance other on negative, side-effects have measured may not actions players’ Then all is incomplete. is measure you what likely, More measure you get. what you “Perhaps what is it: But put benefit. (2007) add not Johnson do as that activities on down using performance cutting improved by by to game leads a rewards, and as measuring work Modeling their towards goals. actions of individual strategies their align they game, Externality 3.5.3.3 mechanism of design. concern the the benefits is that organization way superordinate a in competition them- Designing to selves. left when organization superordinate the components destroy selfish that warns (1994) Deming However, 1996). Brandenburger, and (Nalebuff destructive become neapei i raiainta esisdivisions its lets that organization big a is example An h oto eeto ieefcso lyr’market players’ of side-effects of benefit or cost The tlast ueirpromnebcuei keeps it because performance superior to leads It . “ietmvrflctt) nasoftware a In verpflichtet”). (“Eigentum rnato spillover transaction ehns eintheory design Mechanism or externality owner- coope- tis It . 85 is

III — STATE OF THE ART , re- denotes a describes the as direction, in- is the power put Direction Intensity Persistence motivation that are intended to make developers try out RT A 22 http://visualstudiogallery.msdn.microsoft. Gamification can be used in environments where it Gamification models user behavior in a serious con- 22 com/bc7a433b-b594-48d4-bba2-a2f24774d02f trieved 2012-09-20 Management deals with the organization of workting and people get- to get italso done. has How an this influence happens, on of theagement quality course, deals of with work the results. application Man- ofand economic provides theories, the tools to make them work as desired. 3.5.4.1 Motivation Nerdinger (2001) describes tensity and persistence of behavior. decision for a specific behavior.given It alternate explains choices, why choose persons, the one they choose, and into the respective decision, i.e.is how for strong one the of decision the alternatives. ing the goals previously set.stay Why on is course one even person when able facinganother serious to one problems, is while easily distracted by the smallest issues? is difficult to motivate people to show desired behavior into domains like self-organizing work contexts,a where high freedom of individualsexample, is consortium-based, to research be or maintained. agile For development software organizations. But itdividual can work also situations. fit Figure 3.14 intobadges shows inlike collectible “Regional Manager (have moregions than in 10 a re- single class),processor Close To directives), The Stubby Metal (use (generate 5 method pre- stubs a file)” certain development features. text and associates with it thebadges possibility or to collect other points, rewards inof order desirable to behavior increase shown the by amount users.ered as It an can in be vogue consid- game sub-field and in mechanism the design transition theories. zones of 3.5.4 Management and motivation why they reject the other ones. will to overcome problems that are in the way of reach- when no other means are available. It would therefore fit 10 times) or Interrupting Cow (have 10 breakpoints in TATE OF THE III — S Pigovian taxing , which encourages activities that , i.e. calculated into the cost of transac- has emerged as a means to motivate users internalized The alternative to strict regulation is One way to prevent externality is regulation, i.e. to 86 Definition 45 – Gamification Gamification refers to the useacteristic of for design elements games char- in non-gameet contexts. al., (Deterding 2011) of some information technology platformdesirable to behave ways. in Thetention idea in has numerous gained domains like considerablehealth, productivity, finance, at- education, sustainability, news or entertainment media (Deterding et al., 2011). and optimization. Thenegative inverse Pigovian of tax Pigovian taxhave a is positive the externality. It canfavorable activities increase that the are amount carried of out (Collard, 2009). 3.5.3.4 Gamification efficient resolutions than strict regulation does. that is levied on anbe externality. The externality is saidtions. to It allows steady regulation of the amount of un- have to do the samesource amount code of they documentation for develop.one the developer is It more apt maycleaning to be, up writing technical documentation however, debt. and that So theyent can distribution negotiate of a documentation differ- workload.to According the theorem of Coasequantifiable (1960), for making inclusion an in externality transactions, leads to more an externality. However, this canoverall result performance. in Regulation suboptimal preventsfrom two together parties negotiating an efficientthey solution, in find which a way tocost-effective way. deal For with instance, the strict coding externalityreviews can rules in and prevent a externality. more Both developers would cal debt arises. This is an externality of the transactions. technical debt is left to the original development team. rushes on to his next task, creates negative externality. set up and enforce rules that forbid actions that cause software. In the transaction betweensultant, company money and is con- exchanged for addedthe functionality source in code. If qualityconsultant is gets not his part job of done the quick game; but if dirty, the then techni- Gamification wanted activity, and leaves room for individual decisions A developer who does not document his source code but When the consultant leaves, the efforts for cleaning up

III — STATE OF THE ART Mtvto nopse l rcse eae odelib- to related processes all encompasses “Motivation digaltl eadt h euto oeatosmay actions some of result the to reward little a Adding example, For values. various of combination weighted where 20)so htdsrdbhvosices ihgrow- with increase behaviors desired that show (2009) rto nicnie n xetnisfrtepurpose the for expectancies and incentives on eration Motivation – 46 Definition ec a aeimdaeo nietvle n based and value, indirect or immediate have motivation can lence extrensic and Intrinsic linear. not is relationship the However, rewards. Vojnovicing and DiPalantino example, For shown often. is behavior more desired that probability the increase individual. different a for valence different may much actions a of have result identical the seemingly on A depending individual. value subjective different money a like have sev- component may and each recognition, where praise, feelings, money, other of eral sum a be can it subjectively- a be can Valence value. monetary surable, valence punishment). (e.g. negative or prob- with event indirect an ability causes one’s turn from in result that behavior, directly will event certain a probability in- expectancies, of products values: and of strumentalities sum the as action tain ( Instrumentality model VIE 1987) Gollwitzer, and (Heckhausen implied action.” the of and courses goals alternative between choosing of hsrsac nepesthe interprets research This valence P expectancy ( instrumentality valence osnthv ob n ige betvl mea- objectively single, one be to have not does h au a epstv eg ead or reward) a (e.g. positive be can value The . ro 16)i i Valence-Expectancy- his in (1964) Vroom × VIE ecie htoetik ihwhat with thinks one what describes instrumentality iue31:Bde omk eeoestyotfntosi VisualStudio in functions out try developers make to Badges 3.14: Figure Motivation hoyse oiainfracer- a for motivation sees theory ) hc a oesbetv value subjective some has which , I- olbrto,Ognzto n Motivation and Organization Collaboration, — III-5 VIE = oe nawythat way a in model × expectancy ieie va- Likewise, ) hnfial xrni ead r ae wy Support away. taken are rewards extrinsic finally when Bokas 2005). (Brockhaus, crowd can motivation extrinsic that postulated (1918) nti,casclpyhlg iciiae w id of kinds two motivation: discriminates psychology classical this, on fet fetgoppromneadmtvto (Wegge,2001): motivation and performance group affect effects situations economic in change its for reasons and havior closely to is related and It groups organizations. of collaborative behavior in the individuals well as perception, and behav- human ior and conditions re- working researches between It lations work. human shaping and nosticating, ogy effects Social out. crowded necessarily not meta-study is a motivation in intrinsic found that (1981) Boal However, others. and str left the is for motivation no that so motivation, intrinsic out Woodworth motivation. extrinsic and intrinsic of ences • hndaigwt ruso niiul,sxsocial six individuals, of groups with dealing When influ- mutual about hypotheses different are There • • madJhnesn(08;Fe n ee (2000) Jegen and Frey (2008); Johannesson and om ¨ el ihdsrbn,aayig xliig prog- explaining, analyzing, describing, with deals ecpin Ra n ei 2000). Deci, and (Ryan unfavorable perceptions or favorable causes which sequence, motivation extrinsic and satisfaction, inherent its for just motivation intrinsic spoal o osiu process. conscious a it not effects, probably negative is other to as- opposed performance As individual sessment. absence the in mance The oiaincrowding motivation oilloafing social uiesPsychology Business nutiladognztoa psychol- organizational and Industrial fetrdcsidvda perfor- individual reduces effect hthssm eaal con- separable some has that htdie on nactivity an doing drives that hoywsfudb Mell- by found was theory htsuishmnbe- human studies that 87

III — STATE OF THE ART ap- take ), and tangible as- is an environ- social relation- confidence curiosity and learning social relationships confidence and is clearly associated with ). can be granted as a reward to employees ), Self-fulfillment Workers pride , though meant as a social interaction, can ), RT A if they are accepted and likedin, by are the team important they in work it,a or team if create the a social harmonious structuresteam atmosphere, of copes and if well the with difficultiesships ( if they experience appraisal forgoal reaching a as certain an individual orconstitutes as an a affirmation group. of an Appreciation preciation achieved task ( if they perceive a secure workingtinuity situation, in feel con- their tasks and sensein certain their predictability colleagues and managers ( if their demand for winning,bols, achieving or status advancing sym- job positionsaspect gets is satisfied. mostly This accompanied bytary receiving mone- or non-cash/time-based benefits ( sets and status 6. 7. 8. 5. networking platforms shows that enhancementcilitation and through fa- information technology are possible. profit from information technology based intelligence. Incentives and rewarding schemes aretoday’s omnipresent world in to promote alliors. different Their kinds impacts on of motivation behav- andhave been work performance studied in managementPMBOK and economics. (2008) The lists recognitiondesirable of behavior as and a rewardsment. key for Humphrey (1997) process stresses in itsing key team high role manage- performance. in achiev- However, Demingthat (1994) badly argues chosen reward programsheartened can and leave asking people themselves dis- whyperformed they supposedly worse than others. Rewardsall cannot motivators: address are strong intrinsic motivators but aredress impossible with to rewards. ad- Similarly, mental parameter set by theto organization that influence. is difficult the perceived quality of themerely software awareness being can developed; be enoughBut to it activate this avoids individual motive. contentionthe that is inclusion essential of for non-performingpower team to members. shape The in the form of privileges.place While outside the reach of machines, the advent of social 3.5.4.3 Examples of rewards and incentives Appreciation TATE OF THE workers’ power to effects im- III — S ), effect increases performance ), social compensation effect reduces performance and oc- and effect reduces performance and is a re- curiosity and learning ¨ ohler ), ), K free rider sucker social facilitation self-fulfillment to deeply look intocontext things ( and to understand the if they can takeachieve pride from in their any work kind suchcepts of as and results products in they or their con- pride company as a whole ( through the prospect of being ablesions to and influence to deci- design processes,power but it over can customers also and mean shape colleagues ( if they can continuouslyand increase broaden their their expertise experience. Such people want solely through the satisfaction ofabsorbed being by completely the tasks theyon are ( currently working The prove performance but are notsearch. relevant to this re- The action to free riding whenbalance others the refuse free to riders’ counter- deficit. The based on perceived rivalry. The curs when persons consciously decidebecause to they work think less that otherstheir can effort. counterbalance • • Heneman and Hippel (1995) note that effective team- • • 3. 4. 1. 2. 88 Lehment (2007) (cited after (Prauseple et become al., motivated 2010b)) peo- Individual persons and entiretypes teams of motivation. display different Asources theoretical basis for of software motivation developmentpsychology. can According to be Krumbach-Mollenhauer taken and from is rewarded, then there arefree places for riders social to loafers and hide.then individual selfishness If will only harmneeded the individuals are team. are according What mental rewarded, is modelsfor and both measurements group and individual assessments. 3.5.4.2 Leading with psychology based and individual merit pay only. If only team play work can be developed through a combination of team-

III — STATE OF THE ART agbeast n status and assets Tangible ekaad Pui,1995). (Poulin, award” Week adn ytm htlt niiul rgop compete groups or individuals lets that systems warding freapea lasu-odt akn al) ronly or table), ranking up-to-date always an example (for ( address can projects) source open some in (found s osmbe( Consumable vs. ( Achievement vs. in- or orthogonal completely not are variable, vidually ... iesosi h netvsspace incentives the in Dimensions 3.5.4.4 the index through expressed reputation high are with well-performers rewarded and community, respective the by Web Wide World the are In privileges way. if offending spirit an team in the out lived for danger a structures, bears organizational and traditional with conflict into comes meritocracy a However, environments. enterprise the as shape to The incentives. based group and 2010b). al., et (Prause rewards by dressed vr o n hn(..i eti eido ieis time of period certain a if (e.g. then and now every continuously either away given ( are Continuous rewards vs. dimension, Intermittent guaran- the a without on it Depending winning tee. of chance it the or guaranteed increases either just is reward the then ( well differ- Luck performing The vs. again. Guaranteed lost in be ences cannot that and ones durable time against over destroyed endure or up used are that Durable group. whole the of that progress system absolute achievement evaluates an against other each against Competitive names. without if or or aliases published, under are act participants people of names real if mines summarized. identified shortly the are following, the dimensions it In this show-off. than to more is something but instance, away, is for take can cup, one prize that A something other. each of dependent indi- mostly several although with dimensions, space These incentives an dimensions. derive (2010b) Prause al. schemes, et rewarding various of discussion a From the of “Thief AT&T’s like ones are reward funny example a An of publications. and donations systems with reputation combined and more scoring some ranking, present like examples (2010b) al. et Prause respectively. WWW h nnmu s esnl( Personal vs. Anonymous The individual of examples compare (2007) al. et Ahlgren Hrc,20)or 2005) (Hirsch, tn society stone rsinicwrd ult oto sperformed is control quality world, scientific or ) omlztoso eiortcmdl such models meritocratic of Formalizations . DvC CvA Mrl 99 rv esblt for feasibility prove 1999) (Merel, otat osmberewards consumable contrasts ) PageRank iciiae opttv re- competitive discriminates ) r h yia oie ad- motives typical the are I- olbrto,Ognzto n Motivation and Organization Collaboration, — III-5 GvL AvP meritocracy Bi n ae 1998), Page, and (Brin r hti n is one if that are ) ieso deter- dimension ) principle power IvC h- ) os r uihd hwofv.Tk-wy( Take-away vs. Show-off punished. are worse adi advrulyo physically. or virtually paid is ward hyaernwe o netgtn nopsil rela- possible into investigating for renowned are They The s lsd( Closed vs. agtdrwr eti olms erahdbefore reached be must goal certain a reward targeted a Open- ( vs. Targeted ended some- others). is to it it (without presenting home if necessarily take or to owner, value material/personal its of of show-off thing to achievement is or reward status the of a purpose main the if scribes perform that those if or rewarded, are better ones perform other that the than those either ( Reward if vs. specifies Punishment won dimension The be first revealed. must is op- it it as where before be, one will discoverable reward a to their posed what hand before in Discov- know vs. Proclaimed ( Regarding erable group. mem- the within of confidential bers kept are or bosses externals) (including other everybody and to published either Public are that together. reward the receive also individuals to other group with together a act in or fighters lone as act ( Group vs. Individual over). hnigwre rdciiysml ygvn them giving by simply productivity worker changing the responsible. particular, held be In could incentives economic disci- or pline of in- exercise of reduction, fatigue discharge workers, like subordinate factors other works, that Later found relations. however, human in changes unmeasured to 1978). (Franke Kaul, the associates and their through and only experimenters of them presence changed rather manipulate but consciously conditions not The did experiments satisfaction. two and later performance in first changes to record The conditions production levels. manipulated experiments illumination four in decreases in- or of irregardless creases worker increased that satisfaction job was and finding output surprising The pro- rates. worker duction and levels illumination between decade. tionships one about for extending and 1924, Chicago in in plant starting manufacturing electric an in conducted effect Hawthorne 3.5.4.5 ( Real addressed vs. is Tangibility Virtual achieved. by is grow more will the reward grow open-ended and an or given, is reward the h hne npouto ae eefis attributed first were rates production in changes The aton studies Hawthorne TvO PvD lsie ead codn owehrfor whether to according rewards classifies ) PvC ,wt rcamdaad h participants the awards proclaimed with ), ocrstedseiaino results of dissemination the concerns ) aton effect Hawthorne VvR eeasre fexperiments of series a were ,wihsy hte h re- the whether says which ), IvG espriiat either participants lets ) sascae with associated is SvT PvR de- ) 89 )

III — STATE OF THE ART RT A Similarly, Jahn et al. (2010) provide users with feed- The tools and techniques presented below have facets back on their energy performance.be Feedback enough alone to can lead to apresenting change to in developers behavior. how Consequently, they arenal affecting quality the of inter- source code, could have a similar effect. electronic payment systems with pervasiveto computing alter consumer behavior incentives every-day in situations. form In- of micro-payments arelected given from to users or to col- promoteable environmentally or sustain- commercially wanted behavior. For example, air-conditioned alternatives are available, then additional micro-payments are demanded. The pricements, of however, these is pay- low in comparisonactual to service. the In price their of evaluation the the they effectiveness compare of some their of fourreasoning incentive is models. similar to Their this research,main but (ubiquitous in computing a instead different of do- distributed soft- ronmental sustainability instead of quality improvement). negligible cost can change human behavior. of software tools. Theirthe rise need was for mostly discipline in promotedof the by Web anonymous 2.0 users. with its millions Opportunitiessource and grew open in contribution the communitiesthe facilitated open Internet. by In parallel, theuser possibilities behavior for through measuring hardware, technologyrithms and are algo- ever-increasing. Also motivationfor and harnessing patterns motivation-addressing tools havemuch made progress. Today reputation systems notopen only source target and Web 2.0 domainsconquest but of have closed begun corporate their habitats. similar to the work presentedthem here. combines However, them none into of from one quality assessment, encompassing measurement, concept andity responsibil- identification, over reputation systems,schemes to in rewarding order to changenal motivation quality. in favor of inter- 3.6.1 In ubiquitous computing They show that a small nudge in form of a reward with when occupying air-conditioned space while also non ware development) and with different objectives (envi- Yamabe et al. (2009) develop the idea of integrating TATE OF THE tools III — S CSCW There are several theories regarding motivation, which So, collaboration also builds on organizations. The Collaboration becomes possible through coordination 90 of scientific studies. 3.6 Comparable Works further add complexity. Motivation is aof major management, which concern can addressmotivation. different sources of A typicalthe method choice is of also which toFinally, just use spans observing rewards, work a practices high canimproved already dimensional performance, lead which to space. is a threat to the validity It also allows to reduceand negative effects to of motivate externalities, developers with gamification. is an indication ofbalance between the intrinsic complexity and extrinsic ofthe motivation the and social effects topic. of groups on The an individual’s motivation do not depend on the perfection of internal quality but most adaptive and agile formsorganizing, of organizations where are self- a largeassumed degree by the of individual. responsibility is To align such individuals the willingness of individuals tofluenced cooperate, by which reputation is in- as adiscipline means each of other. groups The of principal-agent agentssituation problem to in is which a the individuals’ goals are not aligned portant finding was that coordination and collaboration lated work with regards todevelopment. the human factors in software between individuals. Technically, coordination requires communication that is supported through and documentation. But collaboration also depends on man developers. Consequently, collaboration, organization and motivation are important for successful projects. attention, and changing something inments their work (Franke environ- and Kaul, 1978).threat The to effect the is validity a of potential ments. productivity studies and experi- 3.5.5 Summary The invention and explication of the concept of technical This section presented theoretical foundations and re- with the organization, mechanism design can be used. would benefit most from a moderate increment. with the goals of the organization it belongs to. An im- A central component in software development are the hu- “reputation systems”, has given rise to this emerging kind

III — STATE OF THE ART hc siae h rswrhns ftx asgsin passages text of trustworthiness the estimates which TopCoder presented is what to similarities some has work Their expected the as them by defined is Trustworthiness oepwri ae narptto system. reputation a members on based such is grants power that more mechanism other The of posts the members. reviewing when more influence earn and members rights reviews, and the posts to with much community contributing By other community. by the reviewed of are members that messages post community the Slashdot the of nisms peers. earn their and from badges reputation unlock publishing can developers by internet, or the projects on source open inspired.” to be contributing and build, share, talented connect, most “tech’s to for makers network social a is Coderwall collaboration online In 3.6.2 rmigcmeiin oslepolm pnoe by sponsored problems solve to competitions gramming development software In 3.6.3 model. software reputation at simpler aim a not has does and scope, development, articles, the the in of restricted analysis more light-weight is uses only it but here wiki. department’s re- the a to of contribute members to motivate institute to search system reputation a and community. the the by inof kinds that contributions desired find increased to They leads rewards sites. creasing web community on behavior disclosed. not are scores authors’ the reputa- as classical system a word’s tion not is certain tool author’s A their However, its on reputation. based then scores. is estimation karma words, trustworthiness long-lived high of earn author the authors a being for By unchanged time. remain longer will assumption statements The correct that article. evolving is an in words of lifetime trustworthiness. their on an based of colors words different highlights in that article Their add-on web-based Wikipedia. a is encyclopedia tool online the of articles 25 24 23 os 20)adHile l 20)uesca rewards social use (2007) al. et Hoisl and (2007) Hoisl developer simulate (2009) Vojnovic and DiPalantino the present (2008) al. et Adler mecha- the researched (2005) Johnston and Lampe http://www.topcoder.com/ http://slashdot.org/ http://coderwall.com/ 25 sa nentpafr htognzspro- organizes that platform Internet an is 24 ita omnt.Mmesof Members community. virtual WikiTrust I- oprbeWorks Comparable — III-6 system, 23 By ilsetettlaon fmnybdfracag,and change, a for bid money of amount total the see will ihspotn h olbrto fidvdasi wikis in individuals of collaboration the supporting with empha- less puts and end, motivation the on more is work hs ok hwta ml ead a led influ- already can rewards small that show works These eiw oass h ult fsuc oecetdby created code source of quality the manual assess with to analysis reviews static combined grading. has automatic (2005) for Smith it on relies (2003) Sim- Edwards assignments. ilarly, student in quality” objective, code automated, of quick, assessment a “provide to particular) in is what and community first. the implemented by wanted the most between changes equilibrium an process towards This software the implemented. drives be money will enough change when the So offered, offer. is the accept to choose Developers can software. free a a in of bug a implementation of the fixing or for feature money of anonymously amounts can software offer a of users where marketplace software. developed the to rights license TopCoder the whereas retains earn recognition, can and competition money a like of prizes winner The devel- created. of are ratings opers overall including Statistics parties. third ieHile l 20)b eadn otiuin oa to contributions platform. rewarding exchange by knowledge (2007) al. et Hoisl like deal that works certainly are research particu- this Of for direction. interest desired lar the into behavior user ence behavior. user “optimize” to there used do- are many and into way mains, their found have systems Reputation work comparable of Summary 3.6.4 artifacts. developers their between and relationship responsibility the on their sis of focus The developers. the awarded to of emailed by overview is The often points commit. more a repository for points revision awarding the to commit code to source developers for student (2012b), motivate Schneider they and example, Singer In program- practices. good ming adopt developers make to methods tion area. educational the outside mecha- environments rewarding for consider nisms or code, source than process artifacts software of collaborative kinds with other deal development, not software do They clear. are for contributions designed individuals’ are the tools where Their environments, grades. educational for basis a gen- as are erated Reports assignments. programming in students oead(09 s ttcaayi adCheckstyle, (and analysis static use (2009) Loveland development software a propose (2009) al. et Bacon igradShedr(02)wn ouegamifica- use to want (2012a) Schneider and Singer 91

III — STATE OF THE ART RT A Software engineering is an attempt to make software Mining software repositories deals with another per- As software is a highly collaborative process with Finally, related concepts and approaches from other domains and in the domain of softwareble development through recent advances innologies. various supporting tech- development more rigorous through formalizations.those But formalization, especially those fromtrial other indus- production environments, tend to be toosoftware. heavy- Instead, it may benologies and possible understanding to offered combine by tech- testing, the mining advances in software repositories andinto management re-designing game mechanics.tool It support allows for to a create new techniqueopers that by grabs their human motivation devel- toof increase the software. internal quality Thisenough” may internal be quality. sufficient to lead to “good ment can also serve toing quantify has quality. Moreover, some test- limited capabilitiesquality to of measure software. the internal spective on the software underto development. obtain It information allows about whatedgeable developers about are what knowl- parts oftion the of software, and responsibility. give Clone a detection no- retrieval and information are techniques thatinformation assist from in repositories. excavating such humans in the center, alsonization aspects and of motivation collaboration, are orga- relevant. Asquality far is as concerned, internal it isfection important to as note mandated that by notstandards software per- is quality the management goal, but just “good enough” quality. various ways, including manual vs.vs. automatic, and dynamic static methods. Software testing and measure- were discussed. These approaches are only made possi- weight for the fluent and knowledge-intensive product . TATE OF THE CMMI III — S Testing is part of software quality management and, The goal of software engineering is to obtain higher None of these works, however, defines responsibility Reputation systems have also been successfully ap- 92 in particular, quality control. Software can be tested in ment standards are still confrontedof with not the being accusation wholly compatible with agile development. their advantages for documentation (whichin has agile its development, place too) are accepted. quality software. Software quality managementically specif- deals with this goaladhering by to modeling managements quality, standards. and by Butdustrial the standards typical like in- ISO 9000applicable or to Six software development. Sigma Consequently, are other less standards were created, first and foremost the attention away from the craftingsoftware and development arts towards the perspectives virtues of of engineering. mentation of decisions and achievements,a and chosen following development methodology. thiness) of texts andtexts. passages for the readers of these 3.7 Summary Software engineering has emerged as a term to draw at the granularity level often strings texts. in collaboratively Only writ- the WikiTrustscores system from computes text karma on theinstead individual of contributor publishing level. such But scoreshavior, in they order use to it change to be- highlight the quality (here trustwor- plied to the software development.seems Particularly relevant to be thealso works use of gamification Singer to and adjust colleagues, developer behavior. who They are still considered as too heavy-weight; although These are the use of appropriate tools, the careful docu- While better suited to software development, manage-

III — STATE OF THE ART S hssStatement Thesis ilsatt rtc hmwt esnbeefr.Aco- A effort. reasonable with them protect to start will project agile an in example, For self-organization. with victim fall will code) source like artifacts process ware hpe 4 Chapter riefrefo usdr lk aaeeto quality or management (like outsiders from force ercive they artifacts, process software from value have developers immediate When property. common unvalued or needed. is enforcement strict their than and means rules other bureaucratic through way property A common else. protect somewhere to from com- come the must protecting property mon forces and quality Consequently, of standards. appreciation premium to tech- inferior and are processes nology people, consortium where distributed, projects research in or severe more is Additionally, problem practical. the not be may planning extensive incompatible naturally are they But properties. common team. whole fu- the the among into distributed far and too ture are rewards the costs while because immediate quality are internal into effort few Developers too commons. invest process to software leads of It tragedy exchange. the information on The riding individuals. free of is goals reason selfish and carelessness the to (soft- properties common projects’ without the that protection, however, proper is, problem The chaos. organized h ouini ogv es fvlet h under- the to value of sense a give to is solution The protect processes heavy and rules strict Traditionally, ELF we ihareetadcrany n li un- plain and certainty, and be- agreement compromise high a tween and projects, agile in ganization - ORGANIZATION .EwrsDeming Edwards W. — ecnntafr h etutv feto competition. of effect destructive the afford not can We ytmms emngd twl o aaeisl.Lf othem- to Left itself. manage not will It managed. be must system A oprto ewe opnnstwr h i fteorganization. the of is aim secret the The toward components system. between the cooperation destroy thus competitive, and selfish, centres, profit become independent components world, Western the in selves stepeoiatfr forof form predominant the is ilcr o h omnproperty. common they the more for the care and will reputation, their for do developers will atgosabdrptto,rsetvl s h more the is, — respectively reputation, bad a vantageous nhwmc eetago euain—o o disad- how or — reputation good a benefit Depending much how property. on common project’s some- the good treats how explicit one make can Reputation play. into documentation. of importance Young rec- long-term to the ineffective. experience ognize too the lack and furthermore future might the rewardsdevelopers as into enough far be too not still are may it and and structures plans; organizational business is to approach then due this and impractical But weeks company. often few different a a in to done on job moves the get to who wants consultant, a just different than a quality have internal will towards forever, attitude with wrote work he will code he source that the knows who developer them- artifacts a process selves; software the in interest tainable then necessary. is longer abidance no their and rules monitoring managers) hl otaeproduct. software whole nbte nenldcmnainadqaiyo the of quality and documentation internal better resulting in themselves, artifacts process care software more take for to developers motivate can reward- system, according ing an and that scores, tools reputation computer compute through gamification that claim I hsi hr aicto ae nrptto comes reputation on based gamification where is This sus- and long-term a developers give to is strategy One 93

IV — THESIS STATEMENT Responsibility and quality data can be combined to At the same time, this kind of quality assurance does TATEMENT fosters the sustainability of common property. not restrict the necessary freedomlines. through strict Instead, guide- developers are engaged in a market game ciency between the polescommon of property pro-social and short-term caring effort forto constraints due the their other pressingtechnical tasks. debt when they Developers deem can it borrow necessary. change of opinions about softwareity process is artifact established. qual- The channelthe is communication geared between not humans, only but toward machine also and between man. Tools likestantly software metrics monitor that artifacts con- send informationchannel as through well the as the developers’their peers impressions who just when capture working withbuilds an up artifact. a database This withscape. a detailed Developers receive artifact the quality feedback land- dition that for is learning, a and precon- feelis that watching someone their or actions. something However, theyto are comply. not They forced can stilland chose work to their ignore way. the feedback earn developers a karma score.is Typically, someone mainly who associatedcess with artifacts high will have quality aone higher software who karma pro- is score mainly than associated some- Karma with scores low additionally quality motivate artifacts. well-mannered behavior through suitable rewarding mechanisms. This where they optimize their behavior for maximum efﬁ- S for HESIS IV — T responsibility about software process ar- can be given for good quality. This communication rewards Third, compensates for the personal costs implied by in- marginal costs of poor quality that the whole team Second, tifact quality is facilitated in formfeedback of non-obligating by peers and quality assessment tools. point, creates a notion of peer pressure as nobody First, it automaticallyquality assigns in collaboratively created work. This pre- vesting in improving quality, and thereby reduces vents diffusion of responsibility. This enables learning and, together with the ﬁrst wants to disgrace himself. would have to bear. A new communication channel dedicated to the ex- To obtain responsibility information, computer tools • • The proposed tool support has three effects: • 94 responsibility for their quality. Consequently,repositories revision of artifacts can beinformation. mined for It responsibility is important tofocuses note more that on the the tool support relevancequantity for of quality, contributions. than on the observe how developers interact with themon project’s properties com- like sourcerepository, code or in general a documentation revisionacting in control with a software wiki. process artifacts Inter- means assuming

IV — THESIS STATEMENT T h olbeiwpooyeipeet upr o the for support implements prototype CollabReview The framework conceptual the is concept CollabReview The einPoesadClaRve Concept CollabReview and Process Design 5 Chapter prototype situations, these term In the concept prototype. the theoretical of the deployment For through tests necessary. experiment not an is example, prototype and distinction concept a work, between this throughout cases software some In a in system. concept the implements which prototype while framework, software. in CollabReview concept the by described framework conceptual prototype CollabReview – 48 Definition the thesis. this of question research the tackles that concept CollabReview – 47 Definition re- the the The is of requirements. result through frame characterized the as sets goals search and model, conceptual final So in implemented later is concept CollabReview The olbeiwprototype CollabReview HIS al h trtv einpoesta mosdthe embossed that process design de- iterative It the chapter. tails previous the in presented question olbeiwconcept CollabReview hpe ecie h oceino h research the of concretion the describes chapter CollabReview epithet. olbeiwconcept CollabReview olbeiwprototype CollabReview o Cargill Tom — h rt9%o h oeacut o h rt9%o h development the of 90% first the for accounts code the of 90% first The ie h eann 0 ftecd consfrteohr9%of 90% other the for accounts time. code development the the of 10% remaining The time. sue ihu the without used is . ecie h theoretical the describes . concept en the means or rte riat seScin5.3). Section (see artifacts written inpoes h rmr olo euigsoftware reducing of goal primary The process. design iterative an following developed was CollabReview Process Research The 5.1 Sec- in 5.4. discussed tion are adaptations possible situations. and project to software Schemes respective have the schemes to adapted rewarding be But schemes. rewarding collaboratively of authorship the tracks that an algorithm uses reconstruction The repository. the revision project’s of snapshots reconstruct from to processes Chap- requires development model in software the presented of was Construction that art 3. the ter of state It the parameters. on mathematical builds the provides it reputa- as computing scores for tion foundation The necessary modeled. a is is project model software a in artifacts process too. presented, is requirements of milestones sources several as by designated as This process matured. maturation concept they the Instead, while research. iteratively developed the of clearly beginning were the requirements from the defined of the all and not concept But CollabReview are prototype. the that requirements both key of of list demanded de- the iterative into introducing delves After it sign, 5.1. Section in scribed h rcs o olbeiwwsdvlpdi de- is developed was CollabReview how process The h optdrptto crsaefial e into fed finally are scores reputation computed The software of evolution the how presents 5.2 Section 95

V—DESIGN PROCESSAND COLLABREVIEW CONCEPT ONCEPT C EVIEW ware quality” R In a research project, requirements are often less The research followed an iterative development Requirements keep the development process focused OLLAB reduce the costs resulting from internalin quality software. problems Internal quality problems result in source process. They allow to measuretimately the validate progress the and final to project ul- original outcomes against goals. the rigidly formulated than, for instance,software in development a project. commercial Thisof necessary freedom degree allows research projectsand exploit to new better insights adapt that arise to requirements from keep research. the focus Still, oncumstances goals and so events that do project not cir- leadastray. the research project 5.1.2 List of requirements Several conceptual and technical requirementsmanded from are CollabReview. de- The highest-level andimportant most ones are listed herefirst in appeared the or order were in distilled whichthe during they research. the Table iterations 5.1 of summarizes the requirements. 5.1.2.1 Requirement-1: “Improve internal soft- Summary: The goal of the CollabReview concept is to process running through several iterations (see5.1.3). Section Each iteration involved some kindexperimental of setting prototype to or better understandproblem the researched and itsanalysis environment. of the experimental results Afteranalysis was conducted. generated each This lessons trial,understanding learned of an the to problem strengthen andof to the guide a development viable solution.of a In some lessonthe cases, learned research when the was success,be value considered the transformed into lesson significant arequirements would for high-level were additionally requirement. stored in These a requirements database on desired outcomes. Without requirements, project concentrating on partial aspects.requirements enable As assessing opposed the todiate success and of this, final interme- results obtained during the development (see Section 5.1.2). work is prone to losing track due to obstacles and over- C ROCESSAND P ESIGN V—D is a software development methodology The advantage of iterative design is that an unpre- An iteration starts out with a prototype that is then 96 Especially if humans are involved,out even ideas well-thought- may turn outchanges to in be the inadequate, project making direction drastic problems necessary. will Fundamental therefore beleading detected paths cheaply, can and be discarded mis- withoutof the having excessive cost been down the wrong road too far (Mantei and functionality and quality of theincreased design and are improved. successively The wholecontinuous design research process process is that a informsof the the evolution project outcome. dictable reality is regarded from early on in the project. users. However, the exactdepends kind on of the fidelity appropriate of testing anything the from prototype, paper which drafts may to be riences functional from artifacts. the Expe- tests aresign then of analyzed the to prototype. refine Each the newbuilds de- iteration on of experiences development from previous iterations, so that user testing and other evaluation methods. tested under close to reality conditions, e.g. with real on and in different stagesthe of costs development. It and so risks reduces ofcan user result rejection. in amendments A to testand the with Teorey, 1988; requirements users Nielsen, (Mantei 1993): Definition 49 – Iterative design ing in and improving onprototype. the CollabReview Iterative concept design and involves humans from early quirements, and how outcomes andrepeatedly requirements were refined through feedback andSection lessons 5.1.2 learned. enumerates the final list of requirements. to the iterations detailedprocess in is Section summarized in 5.1.3. Section 5.1.4. The thesis 5.1.1 The iterative process of this research development costs resulting from internalures quality drove the fail- research. Yet thements concrete evolved through set various of iterations. require- introduces Section 5.1.1 the overall design process, how it led to re- Iterative design Teorey, 1988). The research was organized as an iterative process result- They are the result of the design process and are linked with steady design refinements being made based on

V—DESIGN PROCESSAND COLLABREVIEW CONCEPT eoe esnlt Scin5.1.3.1) (Section personality veloper 5.1.3.1) (Section personality veloper fitra alrswt otaetosi oeie not sometimes is tools cost software the with reducing failures Since internal maintain. of to costly is that code airatog hyaentfre ocml,ie they choice. i.e. comply, a to have forced still not are they although havior direction. right the in developers give nudge not should a to concept motivation CollabReview a The given be it. should do but so al- do away be to flexibility still lowed should take developers would Instead, it developers. but from code, in source checking bad inhibit could of script check-in regulation. a strict instance, by For proto- compliance and enforce concept not the should that type means, That the software. address qual- of internal to ity the is for research more care the to of motivation individuals’ goal The Summary: individuals’ to “Appeal Requirement-2: 5.1.2.2 development the by team. created documentation of quality quan- developers. and by quality created increased documentation an of to tity lead should use is CollabReview its for that of requirement quality prime internal The the software. improve to CollabReview therefore the is of concept purpose inter- avoid The to problems. as quality so nal quality internal improving fo- on by cusing earlier begin instead should mitigation effective, rgn eatiuinfo sblt rbest de- to problems usability from Re-attribution Origin: be- desired the of more show Developers Criterion: de- to problems usability from Re-attribution Origin: and quantity the in increase an is There Criterion: motivation” 9 8 7 6 5 4 3 2 1 # aiiyo computations of Validity motivation individuals’ to Appeal einrqieet fCollabReview of requirements Design aiyunderstandable Easily users for overhead Few dependence without contributors Involve attacks against resistance and Robustness team the to hazard No applicability Flexible quality software internal Improve al .:Smayo requirements of Summary 5.1: Table - h eerhProcess Research The — V-1 aepoet ayi ple ehdlg,distribution methodology, applied in vary projects ware h ocp hudb eil nuht cetdifferent accept to enough flexible be should concept The fdvlpet emszs oeonrhpmodels, ownership code sizes, team development, of Soft- particular. in contexts social and development environments, software different flexi- various be to should applicable prototype bly and concept The Summary: applicability” “Flexible Requirement-3: 5.1.2.3 t fatfcsadteassmn ehd h idof kind the method, assessment the and artifacts of quality of assessment the responsibility, of determination methods. both human of assessments subjective hybrid and and valuable feedback, as well tool-based as automatic assessment objective and cheap should CollabReview incorporate best. suited are quality assessing standards. processes. different partners’ to the impossible of be because would consortium, establish all a In for standards project. a quality of fixed duration short the for only done when especially difficult, is from organizations processes different of emerges Harmonization that data projects. to the these up handle from of to projects needs in and tested size; be medium of to sizes is all prototype to the While well projects, scale code. theoretically source should of concept lines thousand the ten few code a a of and base revisions, few thousand a few to a up developers, with dozen projects medium-sized and small- from artifacts, systems. rewarding process and software of types respon- quality, of and definitions sibility different support prototype to and need concept therefore be CollabReview to The need prototypes error-free. where science, rocket dif- modifiable, from stay fers to need prototypes for where Research, example, quality. external necessary or hierarchy, rtro:Tepooyei eil ihrgr othe to regard with flexible is prototype The Criterion: of ways different environments, the on Depending data handling of capable be should CollabReview 97

V—DESIGN PROCESSAND COLLABREVIEW CONCEPT ONCEPT C EVIEW tions” without dependence” R Origin: Deeper analysis of the ecosystem (Section Criterion: Responsibility and quality information, and Origin: Exploration of rewarding schemes (Section Criterion: The prototype does not require broad user OLLAB Summary: CollabReview should causehead only in a low work over- effort for contributors. While contribu- 5.1.3.5) 5.1.2.7 Requirement-7: “Validity ofSummary: computa- karma The scores CollabReview from responsibility prototype andtion. computes quality informa- Karmashould scores be and valid the tocorrectly enable underlying acceptance guide information by users users, toshould and the be to desired a metric behaviors. thatresponsibility suits There for the collaboratively purpose written of text. identifying be It valid has (scores to derived through metricsreputation correlate in with reality) and consistent (computations in- metric should be implemented and used in the CollabRe- the resulting karma scores are valid. 5.1.3.6) 5.1.2.8 Requirement-8: “Few overhead for users” 5.1.2.6 Requirement-6: “InvolveSummary: contributors The CollabReviewshould concept not and be dependent prototype on broad userthe support start. right For from example, somereject developers might CollabReview initially out ofenvironments laziness. without a In distinct some hierarchy, itpossible project may to be force im- developers toEven actively use if a forcing new developers tool. werehearted possible, cooperation then could not whole- be2.4.2). expected (cf. Section Instead, CollabReviewmechanism needs to a get started bootstrapping evencontributions. without initial Additionally, active especially user users whotribute less con- to quality, will want to avoid CollabReview. project. support from the start, and individualshun contributors cannot its effect. volving responsibility show consistent results). Such view prototype. Therefore it should be inevitable for all contributors of a C ROCESSAND P ESIGN V—D tance against attacks” Criterion: The prototype and its reputation system Origin: Research into responsibility and robust au- Criterion: There are no or only weak manifestations Origin: Showing feasibility of the prototype (Section Origin: The initial concept and ﬁrst reviews (Section 98 should be robust and resistant against attacks. thorship (Section 5.1.3.4) 5.1.2.5 Requirement-5: “Robustness and resis- Summary: The CollabReview prototype shouldcompute robustly its reputation scores andness have against a attacks certain from robust- malevolent usersutors. and While contrib- it is not feasible to makedefense a mechanisms. prototype in- 5.1.3.3) be demotivating because it is perceived asor being unfair. arbitrary of hazards to the team. that damage team workgoals. and It thereby could endanger create project ratlessly races pursue in the which objectives developers promoted reck- by CollabReview against each other, or the CollabReview prototype might Summary: The concept anda prototype hazard should to not the pose developmentstrong team. social CollabReview component has addressing a factorscial such loaﬁng, as free so- riding, orsome reputation, form and of may competition, include whichsportive. must It be can a potentially friendly have and negative social effects 5.1.2.4 Requirement-4: “No hazard to the team” artifacts and their revision repository, sizedistribution of of development, the and project, the rewardingnism. mecha- 5.1.3.2) vulnerable to attacks, it will need to have some basic without considering other important objectives inwork, their harm the team spirit by inciting team members

V—DESIGN PROCESSAND COLLABREVIEW CONCEPT hl okn ihsuc oeadohrdocumenta- other and code source with working While rtn la orecd.I re ooti necessary obtain to order In code. source clean writing hsscindtistemlsoe htteresearch the that milestones the details section This erc rsai analysis. static or from metrics information review take should exam- CollabReview For ple, automatically. generated is review that incorporating information of capable CollabReview be the also But should it. prototype use for not effort will additional developers much time, too the consumes If prototype CollabReview system. the the using them into for it easy enter be to to has it developers, from in information time more invest should or they submitting but with reviews, time much reading spend not example, should For users inter- prototype. to CollabReview related the directly with effort action added few there only quality, be internal should into effort more invest should tors r . ipastemlsoe ncrnlgclorder chronological in milestones the Fig- displays iterations. several 5.1 its ure through went it while reached iterations research the of Details 5.1.3 artifacts. of quality the improve on improve can they how it. and qual- documentation and their quantity of the ity about feedback and receive stored Develop- should be improvement. ers can for opinion spots This identify later bad. to or utilized good minds, is the it in if it about e.g. opinion an form developers tion, quality. their improve to feedback them provide helps should that it contributors secondly, to what But understand to on. users going for is easy learn- it makes flat that a curve have ing should prototype the means that Firstly, should prototype The Summary: understandable” “Easily Requirement-9: 5.1.2.9 hl hmt mrv hi scores. their improve to them help 2. and users, its for understandable easily be 1. rgn glLbfil xeiet(eto 5.1.3.8) (Section experiment field AgileLab Origin: to how and prototype the understand Users Criterion: 5.1.3.7) (Section experiment Moknowpedia users. Origin: to caused is overhead direct low Only Criterion: - h eerhProcess Research The — V-1 :“mrv nenlsfwr quality”). software internal “Improve 1: ihlwcranyadareet(e eto 3.5.3.1). Section (see agreement and certainty low with prototypical the build to therefore was step first A a ubrdi h rp)bifl eotteresearch the report briefly graph) the in numbered (as ( h nta oreo eurmnswr esnlob- personal were requirements of source initial The rjcslk oa o hyfloe hi work their followed they How Bora. like software projects different on working developers of servations to problems usability from Re-attribution 5.1.3.1 to milestones. the corresponds of subsection one Each progress. and history ( concepts prototypes with and confronted where users where experiment ( feasibility technical show problem the solving help to concepts and ( research domain sic its understanding and better problem the reached: what of was indicate directions milestone four of The kind development (1986). software Boehm of model by vaguely spiral is the It by outwards. spiraling inspired line curved a on drawn r owiedcmnain(eurmn-:“pelto motivation”). “Appeal individuals’ (Requirement-2: develop- documentation write of motivation to ers the address that therefore should is concept requirement the A cause. was root documentation as write identified to motivation of lack a Here, factor the Instead, (Requirement- needed is quality, documentation internal better the and improve more To not 2005). con- is (Raskin, which or information there write generate cannot to they costly but sume less Tools documentation were documentation. make There as can serve could sufficient. that not texts few was too way made this be in could available that information of amount the positive, (Prause, browser developer’s easily 2006). be the would through that document accessible hypertext a com- into be bined would information this information all Using techniques, 5.2). retrieval Figure (see sources through-out projects scattered the files text and emails media Javadoc, several like from documentation written accessible of Dendro˘c of lack and 2.3.6). for Section need (see information desperate implementation-related a revealed practices ocp research concept rcse aelmtdsiaiiyi environments in suitability limited have Processes of reception the While system. eeoe personality developer experiment Dendro˘c ,a mlmnaino rttp to prototype or implementation an ), human ol olc isadpieces and bits collect would Dendro˘c a ob drse directly. addressed be to has .Tefloigsubsections following The ). ehia prototype technical rbe analysis problem mn t sr was users its among ,o an or ), ,ba- ), 99

V—DESIGN PROCESSAND COLLABREVIEW CONCEPT 2 Concept research 5.1.3.9 TornBond ONCEPT Technical prototype C ). However, feedback would allow to EVIEW R 5.1.3.6 Rewarding 5.1.3.4 Authorship OLLAB opinion about it. In thisthem situation, the a opportunity software tool to gives writeimpressions, down and their attach feelings them and source to code. the Such tool respective would file then of look up the develop- kind of documentation and hasa therefore broadcast been defined medium, as i.e. itchannel is without a a one-way backwards communication way (see Definition 11 – improve texts to make themaudience, understandable allows to the a authors wider todocumentation, learn receive how praise to for write good better documentation, and generally lets complaints besible. heard by Adding those a way respon- torived communicate as back the was second requirement hence (Requirement-9: de- understandable”). “Easily 5.1.3.2 The initial concept and first reviews From the initial set of requirements, the basic concept on three different files ofing source source code. code, While the developers develop- automatically form an was drafted. Figure 5.4 depicts three developers working C “Documentation” 5.1.3.10 ReputationForge 5.1.3.3 Feasibility

jksd slkd fasld klsadf lskdjlsa dfksjd fölaks dﬂkas dlkf asldkfj aslkdjf alskdjskdjdkjdijd fdk d s lskdfj sd fsd kasldjdiend sd fnmsen defn efe ejnsjen fsa esejk fseklskjen dkjfs uejsd nms ejsdn

5.1.3.2 Concept

uejsd nms ejsdn ejsdn nms uejsd

esejk fseklskjen dkjfs dkjfs fseklskjen esejk

defn efe ejnsjen fsa fsa ejnsjen efe defn

kasldjdiend sd fnmsen fnmsen sd kasldjdiend

s lskdfj sd fsd fsd sd lskdfj s

alskdjskdjdkjdijd fdk d d fdk alskdjskdjdkjdijd

aslkdjf aslkdjf

dﬂkas dlkf asldkfj asldkfj dlkf dﬂkas

lskdjlsa dfksjd fölaks fölaks dfksjd lskdjlsa jksd slkd fasld klsadf klsadf fasld slkd jksd jksd slkd fasld klsadf lskdjlsa dfksjd fölaks dlkf asldkfj dflkas aslkdjf alskdjskdjdkjdijd fdk d s lskdfj sd fsd sd fnmsen kasldjdiend defn efe ejnsjen fsa esejk fseklskjen dkjfs uejsd nms ejsdn Dokumentation Auf einen besonderen Punkt darf ich Sie aber jetzt schon hinweisen: Punkt Angesichts Auf einen besonderen Erfolge des FIT in letzter Zeit lädt die IL im einiger besonders erfreulicher Anschluss an die Institutsversammlung herzlich zu einem kleinen Imbiss im Schloss ein. skins\scales zu entpacken. in den Ordner Meine erste Idee war den Inhalt des ZIPs Dazu must Du scales natürlich erst anlegen. Scales liegt dann auf der gleichen Danach nur das skin in Forrest im Forrest-Programmzweig. Ebene wie Pelt und danneintragen, es sollte gehen. install-skin noch mal ausführen properties scales und co die Verzeichnisse - kopiere nicht gibt es folgende Alternative: Wenn 5.1.3.7 Moknowpedia ROCESSAND jksd slkd fasld klsadf lskdjlsa dfksjd fölaks dlkf asldkfj dflkas aslkdjf alskdjskdjdkjdijd fdk d s lskdfj sd fsd sd fnmsen kasldjdiend defn efe ejnsjen fsa esejk fseklskjen dkjfs uejsd nms ejsdn P 5.1.3.1 Re-attribution 3 Hypertext Dokumentation jksd slkd fasld klsadf lskdjlsa dfksjd fölaks dlkf asldkfj dflkas aslkdjf alskdjskdjdkjdijd fdk d s lskdfj sd fsd sd fnmsen kasldjdiend defn efe ejnsjen fsa esejk fseklskjen dkjfs uejsd nms ejsdn 1 jksd slkd fasld klsadf lskdjlsa dfksjd fölaks dlkf asldkfj dflkas aslkdjf alskdjskdjdkjdijd fdk d s lskdfj sd fsd sd fnmsen kasldjdiend defn efe ejnsjen fsa esejk fseklskjen dkjfs uejsd nms ejsdn 5.1.3.5 Ecosystem ESIGN Lucene V—D 5.1.3.11 MicroEditCastigator system for constructing inter- Detect URLs Retriever Figure 5.1: Iteration milestones in the spiral of conducted research Class <-> Javadoc in-reply-to 5.1.3.12 ScrumLab 5.1.3.8 AgileLab Experiment Enhancer s, they would complain about barely ... preformatted text ... preformatted Dendro˘c Problem analysis

jksd slkd fasld klsadf lskdjlsa dfksjd fölaks dflkas dlkf asldkfj aslkdjf alskdjskdjdkjdijd fdk d s lskdfj sd fsd jksd slkd fasld klsadf lskdjlsa dfksjd fölaks dlkf asldkfj dflkas aslkdjf alskdjskdjdkjdijd fdk d s lskdfj sd fsd sd fnmsen kasldjdiend defn efe ejnsjen fsa esejk fseklskjen dkjfs uejsd nms ejsdn kasldjdiend sd fnmsen defn efe ejnsjen fsa esejk fseklskjen dkjfs uejsd nms ejsdn IDE jksd slkd fasld klsadf lskdjlsa dfksjd fölaks dlkf asldkfj dflkas aslkdjf alskdjskdjdkjdijd fdk d s lskdfj sd fsd sd fnmsen kasldjdiend defn efe ejnsjen fsa esejk fseklskjen dkjfs uejsd nms ejsdn

jksd slkd fasld klsadf lskdjlsa dfksjd fölaks dﬂkas dlkf asldkfj aslkdjf alskdjskdjdkjdijd fdk d s lskdfj sd fsd kasldjdiend sd fnmsen defn efe ejnsjen fsa esejk fseklskjen dkjfs uejsd nms ejsdn (...) CVS Archiv E-Mail Javadoc Analyzer 1. @ @ Another observation was that developers were miss- (sda) @ @ 1. 2. 3. a xy @ ing a communication feature. Whenfront they of were their sitting in understandable source code, cursing itsuation author. is This also sit- expressed by Figure 5.3. Source code is a Figure 5.2: The connected documentation from scattered pieces of text (Prause et al., 2007a) 100

V—DESIGN PROCESSAND COLLABREVIEW CONCEPT ihsuc oe n nu nyfwadtoa efforts additional few only incur and code, source with 2008). Eisenhauer, and (Prause code better write quality, for competition beneficial a trigger should which Wde,2012) (Widder, iw hnti al spbihd l eeoescan developers all published, is table this When view. esadbe) nodrt euetene o review- for need the reduce to order In understandable”). “Easily (Requirement-9: impressions reporting for familiarizing are developers while inherently happen requirements: expert further its following and the concept to the led of reviews drafting But detail 7.5. more in Section discussed in are results The problems weaknesses. potential and some the revealed Yet also investigations. review promising concept further worthwhile fundamentally is a that being idea as con- valued The was weaknesses. and cept strengths identify to The was goal feedback. qualitative gather to backgrounds ferent to learn to developers help feedback the through and base, code project’s the to well how contributed who see overall statistical, a in code source held their are for developers responsible Hence, reputation. quality according their developers to ordered com- which and table code, ranking source a pute of files for responsibilities ers’ it communicate to trying and strong code, a source having about for opinion notorious are Developers 5.3: Figure nevrnet ihwa oeonrhp reviews ownership, code weak with environments In dif- with experts to presented was concept first This - h eerhProcess Research The — V-1 a ul.Tepiayproewst ettetechni- the test to was purpose primary The built. was ol hrb e xase,o elc te project other neglect or exhausted, get thereby would review second the of Results experts. to available were h gr spoie sarfrnefrteoriginal the for reference a as provided is figure The a esblt fterqie ucinlt.Tebasic The functionality. required the of feasibility cal prototype CollabReview first the iteration, next the In prototype the of feasibility Showing 5.1.3.3 team”). the to hazard team “No endangering by (Requirement-4: project spirit the harm implemented not the does that concept to is danger requirement a Another be team. might the that team tool a a within competition Furthermore, triggers applicability”). “Flexible (Requirement- 3: implementation the in should exchangeable metrics be responsibility and “Valid- computations”), (Requirement-7: of defined ity be responsi- to suitable has A metric bility context. project the is on defined dependent is responsibility actual collaborative how a and in environment, trivial not is De- responsibility users”). termining for overhead “Few (Requirement-8: on metrics based reviews automatic support Col- objectivity, should and labReview independence some reach and ers, ac gis attacks”). resis- system against and tance reputation “Robustness the (Requirement-5: attack them as through to robust be try to might need that developers would out metrics pointed also responsibility Reviewers the team”). “No the (Requirement-4: to occur hazard not should races Rat duties. They would ranking. developers quality the race, in rat ratings a top in chase recklessly occur: might race rat CollabReview final the of description prototype. a to for Refer 6 experts. the Chapter to presented was that architecture 2008). Apelt, and (Prause mea- code which source arbiters the of from aspects and sure reviewers, of developers role from the come in Reviews ana- is prototype. which the system, by control lyzed revision check-in a developers into the project: code a source of of parts context different the in the software between relations basic and 7.5. Section in found be can round specification architecture an computational the and of screenshots examples process, time, round This assessment experts. another by for as version serve fidelity to also higher was a prototype the the Additionally, of created. purpose was secondary functions of set with reduced implementation a an and designed, was architecture ae ntefis rttp,rveesntdta a that noted reviewers prototype, first the on Based architecture original prototype’s the shows 5.5 Figure 101

V—DESIGN PROCESSAND COLLABREVIEW CONCEPT

. every used

quality ONCEPT

nobody’s

… is

because

beneficial sometimes

a about

property are

EVIEW

inhibited,

R triggers

models knowledge

This

up .

everybody’s

effectively

is code build

OLLAB ownership

. his C

because

for coding

code style

Quality developers

Reputation quality, weak

coding

cowboy

code,

responsible code but

… and intuitively form an opinion ... opinion form an intuitively … and for Personal quality scores are derived… are scores quality Personal common

… of each piece of code they work with. work they of code piece each … of

foreign

a small,

… and used for a quality reputation ranking. reputation quality a for used … and

development, ROCESSAND

with

… and computes average document quality scores. quality document average computes … and statistically are P

trains problem

in the version history yields personal responsibility scores. responsibility personal yields history version in the a held

A novel tool records and collects the developers’ tacit reviews… tacit developers’ the collects and records tool novel A and

software Looking up the individual developer’s influence for each document document each for developer’sindividual influence up the Looking efforts is working

. Figure 5.4: Poster of the original concept can ESIGN when

modern

Additional developer competition In This property But, V—D 102

V—DESIGN PROCESSAND COLLABREVIEW CONCEPT h de rcagdtesuc oe ticue clone included It code. source the changed or added who Rqieet5 Rbsns n eitneaantat- against resistance and “Robustness (Requirement-5: h etieainivsiae notre cssesof ecosystems target into investigated iteration next The def- the elaborate to was iteration next the of goal The olbeiwpooye(rueadAet 2008) Apelt, first and the (Prause in prototype communication CollabReview user and Internal 5.5: Figure olbeiw n into and CollabReview, ecosystem the of analysis Deeper 5.1.3.5 data. of sets was larger It with 5.3. work Section to in tested algo- described the is algorithm of The robustness rithm. improve to detection copy those and i.e. authorship, of means de- by to responsibility made was fine choice alternatives The several responsibility. of defining data. details for of the sets into larger goes with 5.2.3 prototype Section the try to and tacks”), information responsibility algorithm corresponding robust obtaining a for develop to responsibility, of inition au- robust and responsibility into Research 5.1.3.4 applicability”). “Flexible (Requirement-3: lines code of of ten-thousands several and developers, revisions of typical of dozens thousands of few data a the with handle projects medium-sized to able be next The should space. prototype database of several gigabytes take required and could days) project it existing made CollabReview, an by (importing processed slow be to has of that amount data The problems. larger scalability from revealed corpora projects artifact process However, software continued. with on tests be based should development architecture that initial supported the It concept. the h rtpooyepoe h ehia iblt of viability technical the proved prototype first The thorship EU rjcsseicly nem- An specifically. projects - h eerhProcess Research The — V-1 iepoessrvasptnilfripoeet(see improvement for potential reveals processes wide a rae,adasalsuywt oeta sr inves- users potential with study small a and created, was Rqieet3 Feil plcblt”.Wr during Work applicability”). “Flexible (Requirement-3: Rqieet8 Fwoeha o users”.) for overhead “Few (Requirement-8: h tde losoe htrwrsohrta a than other rewards that showed also studies The hsieaincnetae nbign CollabReview bringing on concentrated iteration This .)atse oeta oapyn h ocp ocollab- to concept the applying to potential attested 7.5) euainsoe oiflec eeoe behavior. developer influence to computed scores harness to how reputation ways on focused environment iteration this target the in situation social the hierarchical on depending and needed are list ranking simple schemes rewarding of Exploration 5.1.3.6 dependence”). without the contributors “Involve from the right (Requirement-6: for support start inevitable broad be require not to open and needs with developers It cope to rejection. has hidden situation and tool this A improve to 2.3.4). Section is (see that study case experiences Karakos with the congruence from in is This 2.3.5). Section project- of implementation the that showed study pirical edfrmkn uhcnrbtoshst eminimal be to over- has contributions the such that making and for contribution head costly but valuable reviewing (see a that context is showed work also experiment real The a 7.6). in Section code. trialed source successfully as well was as It articles wiki support was to prototype modified CollabReview the Subsequently, ity”). applicabil- “Flexible kind (Requirement-3: one documentation than of more support therefore CollabReview should The prototype wikis. like articles written oratively Section Ex- (see workshop scientific articles. a wiki from feedback e.g. pert investigated, than was documentation code ap- other time, source to same concept the the At of users. plicability real with contact first into experiment Moknowpedia 5.1.3.7 computations”). as “Validity of identified (Requirement-7: was fairness of scores factor karma important of computation Validity the 5.4). of Section (see rewards effective into tigated (called schema rewards classification for a 3.5.4.2), Section (see were analyzed leadership psychological of themes Motivational netvsspace incentives e eto 3.5.4.4) Section see , 103

V—DESIGN PROCESSAND COLLABREVIEW CONCEPT . The prototype was mostly 6 ONCEPT C EVIEW R A pseudonym for the software project described in Section 7.1.6.5 6 Therefore, Agaraj (2012) developed a social game to OLLAB tions. The finalscribed experiment in was Section 7.11. a success and is de- that was also used in the evaluation. one was the wiki ofsuccessful, Vyrygh especially in theSection difficult 7.9. EU context; see 5.1.3.11 MicroEditCastigator field experiment editing behavior in Moknowpedia,readability which of harmed the the change logs.havior argued Users that showing acting this differently be- wouldcosts cause for higher them, or necessityoptimal a candidate change of to habits. investigate why Itfailed some was while an prototypes others succeeded:test if This a prototype minor would although change users in do not behavior agree couldsirable. that be Although their the behavior achieved prototype is was notit hated de- by succeeded most at users, suppressingbehavior or (see reducing Section the 7.10). undesired 5.1.3.12 ScrumLab field experiment code domain. Two software teams would be rewarded to reduce their debt.goal for each That individual. The seemedintended experimentation to to was generate not be quantitative experimental ahigh data desirable with validity but toprototype. More bring details forward and some thein results Section can design 7.8. be of found the 5.1.3.10 ReputationForge prototype mechanisms that shouldprojects be without applicable much peer-pressure, in hierarchicalor power distributed availability of financial compensations.and Instead, intrinsic fun motivation were the way to go. promote earning of higherForge karma was scores. trialed in Reputation- tworonment wiki was environments; the one original envi- dia environment where of ReputationForge Moknowpe- showed slight improvements over the original Moknowpedia prototype, and the other The MicroEditCastigator prototype penalized an article The last iteration’s prototype moved back to the source The next prototype emphasized immaterial rewarding with better grades for closely following coding conven- C , 5 . An 4 ROCESSAND P development prac- ESIGN XP V—D A pseudonym for a student software project used in the evaluation. The name is a pseudonym for the software project used in the 4 5 A suitable environment was found in TornBond While the experiment failed to achieve a significant tracking technical debt. Owners ofbe technical presented debt would with theowned. They amount could of decide for technical themselves if debt they they wanted For more details refer to Section 7.1.6.6. evaluation, see Section 7.1.6.3 for more details. tored an open source software.technology The of software is several aprojects, base of and that some of organization’stheir the own current developers research would on also this base software. Karma scores were feedback took too longconsequence and of was these too lessons, infrequent.intended the to As next experiment a prototype with the was design of the CollabRe- 5.1.3.9 TornBond design experimentation scores, that they did not seethem how and the measures what influenced the practical implications were, and that change in behavior, it provided valuableabout lessons social learned effects infor detailed agile information teams. on theally, usability trial See and results. Section understanding Addition- 7.7 issues were revealed ity as the percentage ofdocumenting source source code code entities comments for were which employed available. static It analysis to derive quality scores. Money agile development team following tices (including pair programming) wouldoverviews receive of daily their personal scores. Dueions to about different opin- what constitutesinvestigator good and quality the organizers, between the the definition of quality quality model needs to be exchangeable (Requirement-3: 5.1.3.8 AgileLab field experiment trial in the domain of source code in the AgileLab view prototype to improve onvironment was various needed factors. where continuous An development en- The main lessons learned from the previous experiment The first field trial with wikis was followed by a field (Requirement-9: “Easily understandable”). where a small development team developed and refac- was ongoing, and with easy access to the developers. were that developers had problems understanding karma was used as a reward in this trial. was a matter of discussion. The iteration showed that the 104 “Flexible applicability”). The prototype defined qual-

V—DESIGN PROCESSAND COLLABREVIEW CONCEPT . h opttoa Model Computational The 5.2 in given are implementation 6. its Chapter and Details chapter. prototype this the of rest on the in explicated resulting is The concept form. final the its evolve into and prototype CollabReview concept the develop to iterations several development different environments. to team adaptable be the CollabReview should harming the prototype that not requirements like technical and work spirit, team of so- key aspects encompass cial They in defined. implementation were prototypical CollabReview its and regard- concept requirements the of ing number a research, the During process research the of Summary 5.1.4 nlzn eeoe eairare behavior developer analyzing 5.2.1. Section in is CollabReview presented in The computations documentation. underlies better that and model more write them make document to developers particular, well In how behavior. at looks desired CollabReview the of less broad show less behavior that or a subjects desirable desirable than on of scores lot resources higher a have its should show and that Subjects society the scope. what for to responsible do held they are commu- subjects a observed in nity; well-behaving people promote how to analyze order to in behave is purpose quality Its the contributions. computes on of depending that developers for system scores reputation karma a is CollabReview • • • for answer to has CollabReview questions central The with process design iterative an followed research The a seScin523,and 5.2.3), Section (see way o stecniino riat fetdb the by 5.2.4)? affected Section artifacts (see subject of individual condition the is how individual a the in either has them, much affected contributor how and artifacts what process 5.2.2), Section software (see affected artifacts the are important how – – – oilyindirect socially or indirect, technically direct, - h opttoa Model Computational The — V-2 hc trstesfwr rcs artifacts. process software the stores which software. the developing be will or were with h software The The U Team – 50 Definition the in developers of only team. consists model the Initially, artifacts process software Modeling 5.2.1 Section in examples by 5.2.6. in illustrated described and is 5.2.5, scores Section reputation to lead quality ternal raedt fteatfc)wihi o ealdhere. detailed not is which artifact) the of date create m information authorship fine-grained repository; revision newer a author by replaced or or deleted revision), (e.g. repository the in date the tory; identifier artifact’s an model, this In time. in change artifact a signify to needed time until team artifact software empty the of agents the by artifacts process software repository Revision – 51 Definition a nadto oteta,teei h eiinrepository revision the is there team, the to addition In . e hr etedvlpr ftesfwr team software the of developers the be there Let in- and responsibility how of computation exact The o ipiiy h index the simplicity, For : o ahcaatr n xeddifrain(like information extended and character; each for                eiinrepository revision s en h cultx fteatfc sasrn;the string; a as artifact the of text actual the being c x c m o i s a h rae h riatb omtigi othe to it committing by artifact the created who sa6tpeof 6-tuple a is eevdfretne information extended for reserved artifact the (committed) created who information authorship fine-grained akdas marked revision) and branch (name, identifier string content o ∅ team a hnteatfc a akda obsolete as marked was artifact the when ε nl)i h riati tl cie the active; still is artifact the if (null) . i sotie rmtervso reposi- revision the from obtained as U obsolete stesto l gnsta are, that agents all of set the is stesto l atadcurrent and past all of set the is a ∈ t slf u in out left is A t o eee ae or date, delete) (or hthv encreated been have that A nesi is it unless t lsthe plus , ∅ u 105 ∈

V—DESIGN PROCESSAND COLLABREVIEW CONCEPT , ) ) C (see bcd ( zabc is not ( 1 with 0 00 , t a ) a is straight 6 7 holds that 1 A (bcdef) ∀ A ε ) 0 a is minimized. abc +1 ∈ ( t ) , a 2 a A 00 , a 7 5 0 a (bcde) B ∈ a ) = 0 0 therefore becomes a ∆( a a ∆( ) ( 1 ≤ to the root p (bcd) 4 4 ) C : n software archeology 0 0 zabc a ( +1 . The dashed line is a possi- t ) a, a 3 2 1 A 3 B ∈ bcde ( 00 ONCEPT a , the true parents have to be guessed ) = ∆( C is closer. 0 3 ) (abc) 1 A 2 , ... (zabc) , a , and ) 0 ) +2 abc a t ( ( is its optimal parent, i.e. the one that is most EVIEW p 2 3 . The case that two potential parents have the ,A 1 R A ) A would have to be checked. bcdef ∆( +1 ( t ∈ t 5 , A ) ) bcde 0 ε ,A ( Figure 5.7: Parent relations forming a straight tree However, the true parent of an artifact The path from an artifact a ∈ t 0 OLLAB ( bcd dead end. The dottedof lines are alternatives for parents exact same distance isbecause resolved it non-deterministically rarely occurs in reality. that the edit distance to theIn parent the best case, forp each artifact similar to it for all necessarily the one with theIf smallest the similarity agents’ distance. actual actionsbut are not only available anymore theA series of revisionin repository practice. This snapshots was named Section 3.4.2). Additionally, itcostly may practically to be determine too thea optimal parent as all artifacts end artifacts that introduced changesdone. that were Consider later the un- insertion of the following artifacts ble parent relation that isis rejected inserted, because when Figure 5.6: Software process artifactilarity evolution distances with noted sim- at thenodes edges, (Prause and and Apelt, authors 2008) in the (of strings) in this order in Figure 5.7: within the set of revisions. It does not include dead- Revision: ( C . n = so → . has and t t with with } ...c +1 0 } A 2 0 t a s A 0 . As a a c , modi- a ) a with its A ∈ 1 0 A c t its parent, a ∪ { \{ as the root. ) A t a, a 0 ROCESSAND = ε a A P ∈ +1 ( . a from s t ∆( p n = a a A = ≤ ∈ +1 a t j 0 ESIGN as the tree’s root, any a A ≤ ε . a i with } characters 0 0 ≤ a a n V—D 1 . The artifacts therefore form ) 0 ∪ { a t ( takes an artifact replaces artifact creates a copy of artifact with p A . u j u } u 0 c = = 1 a a − its parent, so that is attached to a parent artifact +1 j t ) 0 ∪ { 0 which is marked obsolete, so that by adding to or removing substrings from is any consecutive sub-sequence of a A . While the agents are developing ) a 0 ...c ( } a a A ε p is a sequence of \ +1 a i ∈ t { c = i A a , they repeat one or more of the following actions c commits it into the revisiona repository as delete: agent copy modify: agent fies its contents and commitsrevision (i.e. repository saves) as it into the so that copy: agent ( = +1 string = Starting with empty artifact By construction, each artifact The initial repository contains only the empty artifact • • • After some time into the project, the agents start de- substring 0 t 0 measure of similarity, a string3.4.3.2) edit measures distance the (see minimum Section numberoperations and (e.g. size inserting of and edit deletingconvert of one characters) string that into another one. new artifact a parent artifact a tree structure withSee the Figure empty 5.6. artifact It showsartifact. the The tree agents of who the created evolutionare of the noted an different in revisions the nodes.the The edit numbers at distance the between edges two specify artifacts A until the project is finished: Definition 53 – Develop Developing is the process of modifying, copyinging or the delet- contents or propertiesfacts of software process arti- the artifacts’ content, changing theirthem identifiers, as marking obsolete, or changing the extended information. repository. Definition 52 – String s A veloping and add software process artifact to the revision A A 106

V—DESIGN PROCESSAND COLLABREVIEW CONCEPT naet( agent An order in quantified be to has that artifacts process ware weighting exchangeable using artifacts to weights ih emr motn hnasalrone. smaller a than important artifact more process be software might bigger “El A famous week’s group’s matches. work last Classico” the with from page results soccer a table instance, more by for is attention information than, more valuable important much contains attracts and that readers wiki its a in Likewise, page software. the a be from can removed that or one others replaced obscure several easily an by than invoked important more is central be that might a code of source maintainability in the class example, For than important more others. are artifacts process software Some pro- software of weight Importance 5.2.2 epniiiyi h tt fbigrsosbe(Farlex, responsible being an 2012). of on state effect the is an Responsibility have ( to artifact project obligation software the artifact has or had has Responsibility – 54 Definition responsibility as: Let defined for. be responsible for are care they agents artifacts make the and scores, reputation compute to soft- and agents between relationship a is Responsibility for contributors of Responsibility 5.2.3 (cf. 3.4). repositories Section software from mined information possible using are measures weight elaborate more However, functions • • o hsrao,ClaRve sin different assigns CollabReview reason, this For • egtt riat htaera oeotn(cf. often more read are that artifacts to weight and artifacts, larger to weight weight ecea 2010). Dencheva, more assigns which measure weight view-count higher a a assigns which measure weight size a same the assigns which measure weight unit a esartifacts cess otaepoesartifacts process software w u scalled is ) ( w a ) oalartifacts, all to ∈ [0 .. 1] responsible h rttp a available has prototype The . a hni ihralready either it when ). - h opttoa Model Computational The — V-2 hl wesi ed orsosblt,responsibility responsibility, to leads ownership While ihtesmo l gns responsibilities agents’ all of sum the with hc sondb ooe—eg eas tdoes it because e.g. — no-one by owned is which ehiusfrotiigatosi nomto were information authorship obtaining for Techniques is ownership from arises that responsibility Therefore, owner- to related closely is responsibility of concept The o eti artifact. certain a for in5232,o rmsca eain ntedevelopment the in relations social from or 5.2.3.2), tion software ( the artifacts in process links from an result rather can responsibility is direct but termed effect is this an then had obligation, yet not has sponsibility called is This effect. direct manifested a is artifacts, responsibility process his software modifies agent an If sponsible. agent respon- individual the case, an this of In sibility thereof. parts or artifacts process osntncsaiycueonrhp Responsibility ownership. cause necessarily not does responsibility indirect Technical 5.2.3.2 3.4.2. Section in discussed responsibility. a and legitimate ownership is defining artifacts its of process way pragmatic software becomes creates for artifact Authorship who owner. process developer the software consequence, a (Hoppe, In it with be labor” 2004). can one’s — “mixing code by source appropriated to-be-written like exist, yet not termed them. for software responsibility the obtains in and knowledge artifacts, his process of agent traces The direct 2.4.4.1). leaves Section (cf. knowledge through ship responsibility Direct 5.2.3.1 CollabReview. responsi- in the measure of bility choice the explain subsections following ( team hr r ifrn asfra gn obcm re- become to agent an for ways different are There software for responsibility share can agents Several hlspe onLceblee htsomething that believed Locke John Philosopher social epniiiy(eto ...) fteaetsre- agent’s the If 5.2.3.1). (Section responsibility direct nietrsosblt,Scin5233.The 5.2.3.3). Section responsibility, indirect responsibility. technical u X r ∈ ( U ,a u, r ( ) ,a u, ∈ nietrsosblt,Sec- responsibility, indirect indirect u [0 1 = ) o artifact for , 1] epniiiy In- responsibility. a is 107

V—DESIGN PROCESSAND COLLABREVIEW CONCEPT , 3 . and . 25 . ) = 0 ) = 1 ) = 0 B, a ( r A, a ( D, a r . However, works ( r 05 , i.e. . . ) = ) = 0 ) = 0 C, a ( r D, a ırba et al. (2005) or Hattori and D, a ˆ fair shares ( ( r r ) = ONCEPT of the responsibility, while B, C and C ) = 5 . B, a , and ( r C, a 15 . ( r ) = 0 EVIEW ) = tion R ) = A, a ) = 0 A, a ( ( r r Also all authors could receive equal shares Responsibility values determined by one responsibil- C, a B, a OLLAB ( ( bases is the developer specialization index. ity measure can also be distorted to obtain better results. lowing situation: The lines ofby a different source developers. file A were wrote written C 50 wrote lines, 15 B and wrote Dhad 30, wrote 5 ones.D It would would receive be their fairr if A by other authors like G Lanza (2009) assign full ownershipto (i.e. responsibility) the single majorr contributor, i.e. Or shares could be inverted basedminor on contributors the might reasoning be that moretifact’s influential quality. to These the are ar- designdepending decisions on to the be context. made 5.2.3.5 Excursus: Computed developer specializa- quality assignee for certainDevelopers software could process get artifacts. randomly assigned asfor co-officers artifacts. Inmuch pair more programming, influence than the just navigatorcould an has be observing reflected. role. In This aopers large could group be of assigned developers, to devel- employees logical of sub-groups, one e.g. partner the ineach a developer consortium has project, a where certainfacts responsibility created for by the the arti- other developers in his sub-group. 5.2.3.4 Mixed responsibility and fairness does not necessarily rely onbility measure. a single Direct responsibility kind can oftechnical be responsi- or mixed social with indirect responsibility; differentof kinds direct responsibility measures can be used; and varied measures. To understand what is meant by this, consider the fol- The computing of responsibility scores in CollabReview A figure that is later used to characterize different code weights can be applied when combining responsibility C . 7 responsibil- ROCESSAND P ESIGN V—D technical indirect JavaDoc tag which explicitly tag because they do not want to @author @author cf. Conway’s Law, “organizations [...] are constrained to produce 7 However, it can be worthwhile to explicitly override Social indirect responsibility often is difficult to an- Using suitable algorithms, a computer system is capa- Indirectly responsible developers will fix problems or augment direct responsibility insame order peer-pressure effects to as utilize with the sponsibility. technical indirect For re- instance, a singlepointed person quality could manager be for ap- a whole project, or become designs which are copiesorganizations” of (Conway, the 1968) communication structures of these also ban the create the impression of exclusive2011). ownership (Grossbart, Apart from similarresponsibility few is examples, difficult this to detect. kind of alyze but can have strong influences.posts of Sometimes, indirect sign- responsibility can be found in the soft- knows of an assigns responsibility. However, this information can be 5.2.3.3 Social indirect responsibility ond kind of indirect responsibility.the Social team happenings leave in traces in software process artifacts concerned with their reputation score. ble of detecting indirect responsibilities inSection close 3.4.4 artifacts. discussed mining contextscess of artifacts. software This pro- can be employed to reveal closeness. ity. themselves or try to influenceones the because directly the responsible indirectfects responsibility their negatively own af- reputation score.can Indirect therefore responsibility create an additional peer-pressure in cases condition because a bad conditionown could artifacts. irradiate on For his example, badcan application adversely interfaces affect the developer’s sourcetested code, but poorly required sub-routines canown lead code, to and failures modules in dependenciesmodules on pose undocumented a danger forof future responsibility adaptability. is This called kind can also stem from linkstifacts. between software If process a ar- developer’s software process artifacts are increased interest in that these other artifacts are in good The social context of the software project poses the sec- wrong and not reflect reality. Some software projects ware process artifacts. For instance, the Java language where directly responsible developers are not so much 108 “close” to another’s ones, the developer has probably an

V—DESIGN PROCESSAND COLLABREVIEW CONCEPT eeipeetdadtse.Tega fteinvesti- the of goal The tested. and implemented were h eeoe pcaiainidx( index specialization developer The otaepoesatfcso h u fdevelopers’ of sum the ratios contribution of squared artifacts process software developers. single on concentrated is closer responsibility The repository. revision a artifacts in process dis- software are non-obsolete all contributions over developer tributed well how of measure specialization Developer – 55 Definition loih sdsrbdi eali eto 5.3. Section in The detail files. in between described copied is or algorithm moved are iden- that to strings algorithm tify detection authorship clone The a includes responsibility. algorithm in shares are fair artifact line. assigned process a software of a editor to last contributors the Different to responsibility assigns that sure investigation. the this presents of 7.3 results Section results. measure sufficient responsibility produces suitable that a identify to was gation measures responsibility direct several research, this For CollabReview in responsibility of Choice 5.2.3.6 then artifact, each for responsible 50% each are one for then responsible artifact, solely is developer each If artifacts: (0 . olbeiwipeet ietrsosblt mea- responsibility direct a implements CollabReview two and developers two be there let example, For The 5 2 +0 i . ds 5 2 )+(0 sdfie steaeaeoe l non-obsolete all over average the as defined is 2 i . 5 ds i 2 ds +0 = = . 5 2 (1 ) a X 2 ∈ 0 = )+(1 A 2 u . X 2 5 ∈ ) . | U A 1 = | r ( ,a u, fbt developers both If . i ds ) i 2 ds - h opttoa Model Computational The — V-2 → ) ∈ 1 h more the , [0 , 1] i ds sa is = line A rp ymtial rud0 ota saneutral a is 0 that so 0, around symmetrically wraps A Fre n ls,2010). Glass, and (Farmer iecaatr’idxs,ie ti either is it i.e. indexes), characters’ line (with characters line new any string content fact’s Line – 56 Definition igenme?Aseigti usini o h goal the not a is in question thing this complex Answering a such number? express single to possible Section it (see Is opposing even 3.2.1). are which of some tributes, (e.g. domain respective wiki). the versus of code source needs the should to range adapted The be recommendation. a How- only ratings. is this quality ever, bad represent ratings ones negative quality and good indicate numbers Positive rating. artifacts an to Once scores message. text a agent and rating, score the integer an representing contains review Each artifact. process comment text a with artifact process software a of ratings Review – 57 Definition deter- democratically quality. that mine votes cast are form that the reviews in of come CollabReview in assessments Quality artifacts process software of Quality 5.2.4 review • • • ult ossso ubro ifrn ult at- quality different of number a of consists Quality software single a to attached be can reviews Several • • fe e iecaatraded ttenx one. next the at ends and character line new a after substring true a or character, line new last the after starts suffix a character, line new first the at prefix a character, string whole the h mt string empty the u l oso oClaRve,i a sinquality assign can it CollabReview, to on logs sannetnil usrn eghi narti- an in length substring non-extensible a is sa betta lmstgte n o more) (or one together clumps that object an is l l = = s s p s a = : s = l s = l c c ( = l 1 ,a u, = i c = k ...c 1 +1 ε s c s = ) 2 i if , ...c 0 1 ...c i ∈ = 1 s with i , , n s n [ c − 2 i with otisn e line new no contains htde o contain not does that j .,i ..., , 10 +1 0 n < i < , ...c +10] 0 k i < i en h new the being j +1 k h range The htstarts that htends that n < that 109

V—DESIGN PROCESSAND COLLABREVIEW CONCEPT ONCEPT C here. Arbiters either review artifacts im- EVIEW R arbiter It could be possible that human reviewers might feel Still, it is appealing to support human assessments A toolkit that provides automatically generated qual- OLLAB might refrain from writing a bad review in fear of bad thus immediately provide an objectivequent basis for human subse- review; orinformation an into external CollabReview tool (e.g. can through feed web the ser- automatic static or dynamic testinglong (see as Section the 3.3), results as canon be converted the into quality artifact scores level.for Java code CollabReview to uses createhow automatic Checkstyle reviews many depending code on smells andthe source style code violations (see it Sectionbehavior finds 3.2.2.3). can in But be this easily default environments extended as and long adapted as an tois appropriate other available. software toolkit For example, arbitersreviews could for also submit highly bad complexsome classes source as code metric. determined Test by generated coverage through reports an that continuous are integrationbe build evaluated could to provide quality assessments for classes ineffective compared to codebiter’s reviews arbiters. should not outweigh Hence humanCollabReview, the reviews. the ar- In reviews of humanstreated and equally. arbiters are 5.2.4.3 Anonymity and rejection of reviews 5.2.4.2 Manual and automatic reviews their thoughts so this informationto does foster not communication get inside lost, and theand team. fast With interfaces simple in place,developers the can overhead be burdened kept on minimal. to measure automatically, automated testingbly adds inaccurate (possi- but) objective datalabReview (see therefore Section also 3.3). allows to Col- incorporateautomatic data testing. from Such data caninteraction help between to humans get and the CollabReview complex started by feeding the initial data6: into “Involve the contributors system without (Requirement- dependence”). ity assessments effectively becomes an agent,called which is mediately after they enter CollabReview’s database and vices) when it has it ready. Arbiters can use data from The idea of letting developers write reviews is to capture Although software quality would benefit, a reviewer with a low or high test coverage. with automated testing. While quality is indeed difficult C string of a re- ) timeliness ROCESSAND u, a 0 ( P , t . Because later ) ) ) 0 , a , a ε ε a, a a a ESIGN ∆( is the edit distance be- ∆( ∆( 0 a − 1 V—D . The timeliness per agent since agents can always ) 0 and its actually reviewed ancestor when the artifact changes. 0 a 0 a, a for artifact votes ) = max 0 → u ∆( ) u, a ( u, a t at review time, negative timeliness is excluded: ( t a 3 1] CollabReview computes timeliness by means of An agent can revise its assessment of an software in relation to its creation cost , tween that artifact a revisions can have more changes thanartifact was the size of the gives older reviews lesser weight inup-to-date comparison reviews. to The more review is[0 said to lose edit distances old review may still beto appropriate some to degree, the so new itOn artifact should the not other be hand, discarded itanymore outright. is as probably not well. fullyreview to Asking would the the put point additional reviewer burdenguaranteeing to on that update the him review his while would not before, updated. CollabReview There- uses an aging of reviews, which Software process artifactschange. are subject More to precisely,artifact, when continuous the an modified agent artifactartifact modifies is that an considered supersedes as themarked “obsolete”. a old A new review, one. however, remains attached to the The now-obsolete old old one artifact. is On the one hand, the modify their review to keepcalculates it an up-to-date. artifact’s CollabReview quality rating asall quality the ratings average over for it. 5.2.4.1 Timeliness — Aging of reviews first and foremost maintainabilityity). (including readabil- process artifact at any time.have However, more an than agent one cannot review perno artifact need at for a more time. There is also that CollabReview doessuring not quality. rely on Making exactly sensecovered mea- of here. mere Concrete numbers ratings isby are not multiple heavily factors influenced like qualityInstead, goals CollabReview or captures team impressions experience. of theceived quality per- of software process artifacts, and this is of CollabReview. It does notin want its to full represent entirety quality butmight to give perish a in voice ignorance to otherwise. somethingand For which defined achieving quality hard goals, classical formal technical review by agent views might be more suitable (see Section 3.3). Note 110

V—DESIGN PROCESSAND COLLABREVIEW CONCEPT hl gnsaeecuae oicueatx comment text a include to encouraged are agents While ihhrrjce eiw h a dtadre-issue and edit may She review: rejected her with submit to necessary se per not is this ratings, their with iwr hc hudsfc nacoeaieenviron- cooperative a in suffice should which viewer, cr n omn,adlt vrbd iwthem. view everybody lets and quality comment, a a of and consisting score reviews all records labReview submit to anonymously. feature review its a through assimi- problem the this makes CollabReview for allowance i.e. behavior. conformity, and opinions to of subject lation front are in group publicly a made judgments of that Insko note And (1985) does. al. closeness et physical as well as report defects, to willingness the among decreases familiarity participants that inspection report Sea- (1997) 1989). Basili (Elster, and colleague man developer a individual with the trouble to having costly than less qual- seem source overall might in ity loss A himself. for consequences eas eiwr a eansce,wihpeet per- prevents which spirit secret, team remain can to reviewers menace because a re- not just situation. therefore the of is escalating CollabReview instead thereby first, and review artifact will the the jecting artifact in the problem for the responsible fix are who Those ment. a 1997). to (Humphrey, escalated decision be a make should to issues instance possible, higher not dissent. is their settle this to If person in disagreements dis- their to cuss encouraged are reviewer and anyways. developer disclosed Instead, are identities CollabReview the in This modeled because her. not anymore with is contact interaction in the get of to part others for possible is now review,it has non-anonymous her reviewer time- through the identity its As her until revealed out-of-date. or gone author has its liness by revised the either until counted is score review rating non- its review has review An the unrejectable unrejectable. submit review the can makes she then which anonymously, or course, of again, then, rejected can be it although anonymously, it visible. remains quality still its have but counted, not of rating does submitter review rejected the A by review. viewed the the be to then reason can rejection which a the review, add rejected may having rejecter as The noted review. publicly then any- is important by who rejected is one, be This may review anonymous rating. an the because justify and and explain artifact, process they software the to for helpful responsible however, those are, comments Such score. rating the h nnmt etr rtcsteiett ftere- the of identity the protects feature anonymity The deal to options two has now review the of author The Col- anonymously. submitted are reviews default, By - h opttoa Model Computational The — V-2 N aadt h team”). the to hazard “No ihterjce eiwfunction review rejected the with rating quality artifact’s process software A h eeoeswiesuc oea en fachieving of means as code source write developers The ... optn otaepoesartifact’s process software a Computing 5.2.4.4 hmevs o,b eetn h eiw(Requirement-4: review defend the to rejecting way by a too, have themselves, review, artifact the unjustified for an responsible of ones reviews case the reject In may unfair. authors deem they why that reason the ratings. is bad this with developer But specifi- that then by could artifacts He target responsible. cally arti- is an who of infer log and revision fact, the scrutinize reviewer instance, malicious for A could, CollabReview. directly noth- do has to This ing concealed. necessarily not not is are responsibility artifact obvious, process software respon- a ones for the sible While conflicts: personal out carrying reviews. the with neutral more be reviewers team”) the lets to and hazard “No (Requirement-4: grudges sonal ol ..otiigago nenlquality. internal good secondary a the obtaining support i.e. to goal, wants CollabReview it of Instead, intention this. the support not to is It goals. project set documentation. its to value does it contribute but necessarily software the not from expected is the that implement behavior can however, scores. code, the source into written go should Badly texts written the of quality quantity both and direct So a documentation. is internal article to internal any contribution of almost wiki, amount a and In the quality documentation. towards the work improving to suited of best goal what is artifact, score process karma software of of kind kind the on depends It arti- this for fact: ratings review non-rejected all from puted .. utn ttgte okrascores karma to together it Putting 5.2.5 tl,ClaRve antb iue savhcefor vehicle a as misused be cannot CollabReview Still, ] ( r = ) ult rating quality q ( a = ) 1 0 u otherwise of review if X ∈ U u X ∈ s U ( ,a u, t ( ,a u, ) t a ( ,a u, ) by ] ( ,a u, u ) ] ( srejected is ,a u, ) q ) ( a ) scom- is 111

V—DESIGN PROCESSAND COLLABREVIEW CONCEPT of an ) that is that the u ) ( q q k for specific : k ) ) u, a ) a ( u ( a r ( ) ) ( b w q score ) u, a u, a and responsibilities ( ( r r ) u, a ) ) ( a r a a ( ) ( ( q quality a w w ( w A A ∈ ∈ X X A a a , CollabReview depends on arti- ∈ X a A , quality for contributing to all the software ) ONCEPT ) = ∈ a u C u ( ( a ) = w r u k ( q k EVIEW R is defined as the average of the twice-weighted . The quality karma scores are closed-range u ) sum up all the partial responsibilities For example, there can be a bonus for writing reviews. The points awarded by a bonus mechanism should To compute the overall u, a r OLLAB ( Consider the following scenario from Dencheva et al. be computed from the aboveple, parameters. karma In scores the are used exam- to improve participation in a 5.2.5.3 Bonus points contribution quantity and quality, the scoresther can modified be to fur- honor othermeans desirable that behaviors. developers That get bonusdesirable points efforts in addition to their base karma score. Reviews are no core contributionprove the but quality they of help software totherefore process an im- artifacts. important They contribution are toconcept. the CollabReview be few in comparisonthe to bonus the to core the karma scoresbe scores. can a valuable get contribution. Still everything Bonus startedthe points and can bootstrap interaction between agentsdence”). and CollabReview 5.2.6 An example of karma scores k agent has made to allkarma scores. artifacts. Within These the are formula,facts open-ended software are process weighted arti- by their importance awarded to agent process artifacts cle importance r scores. With this data, theagent reputation score artifact quality scores: (2011) as a simplified example of how karma scores can (Requirement-6: “Involve contributors without depen- While the core of the karma score models consists of C ROCESSAND karma scores P score (Farmer and ESIGN Quantity V—D karma score allows the users to robust karma karma score stays within a certain at their heart. q k open-ended quality closed-range and quality Combining an open-ended and a closed-range score A Therefore, different ways of combining attributes into r input depending on the documentationels context, used all here mod- have two basick karma scores for quantity linear (e.g. logarithmic)remains function. theoretically infinite. But Another the optionuser’s is score rank to still with take regard a to thisitself, score and instead adding of it thescore. score as a modifier to the closed-range 5.2.5.2 The fundamental metrics quantity and better than them with less effort. usually results in an open-endedpossible score. to reduce However, the it effect is reducing of its the open-ended weight score in by comparisonscore, to or the by closed-range scaling the open-ended score with a non- such score. A new userand can is directly not gain disadvantaged the in topBut comparison score in to other the users. casepossibly of fleeces average new quality, users thisenter the kind because game of even late, score they ifas can they users immediately who score first have as done high well over a long time, or score also the problem: ithow is few frustrating points for they new have usershave in to gathered comparison a see to huge users amount who of points over years. the average quality of artifacts contributed by a user is 5.2.5.1 Open-ended and closed-range scores range. An collect ever more points.example, There the is number no of upper artifacts limit.is contributed an by For open-ended a karma user score.ended The karma advantage scores of is open- that theybecause are a a user perpetual will motivator beforever. able to It continue honors to collect long-term points contribution. But this is karma models to support various documentation contexts Basically, karma scores can be open-ended or closed- one karma score are needed.score Such is a also combined called Glass, karma 2010). CollabReview can be used with different value range no matter what the user does. For instance, (Requirement-3: “Flexible applicability”). While the karma score models can incorporate different 112

V—DESIGN PROCESSAND COLLABREVIEW CONCEPT hsscinitoue h opttoa oe behind model computational the introduced section This iw) n ai ftecnrbtrscnrbto oall to contribution contributor’s the of ratio and views), valuable. iaad e eiwo go”t h ril,giving article, the to “good” review. of old review his new for a points adds 0.7 Gina of article. bonus the a of retains revision only older Rico different, quite a to loses applies review Rico’s points). (three addition, Gina In and proportionally points) split (seven now Sonny are the between points of ten third article’s The one text. Gina about Later, replacing article, article. Sonny’s average-quality edits his for points receives Sonny ten while For point, “average”. one receives of Rico rating review, his a with article the reviews Rico considered is wiki the scores to contribution quality any negative as no used example, are the in exchange. that knowledge increase Note, to order in wiki corporate olbeiw tdsrbdhwteeouino soft- of evolution the how described It CollabReview. Summary 5.2.7 article. the to contributions contributors’ page size, (e.g. article the review of democratic weight above importance by process, determined quality as criteria: article three the on of con- depends a article for an awarded to are tribution that The points articles. reputation wiki of to amount contributing by is points reputation revise time. can any but at article, review per his review have active cannot one reviewer A than his more point. refresh full to the has regain reviewer to The later review to article. due the ages to review How- changes the point. as one decreases user worth the this ever, earns review an fresh of A quality the article. determine democratically assessments to quality a used The of are comment. form review in a article and to an rating is of one assessment first quality The a points: provide reputation collect to users for again. point full Rico the When gets he points. review, 0.7 his with refreshes Rico plus and points review, five her with for Gina one points, ten This with article. Sonny leaves the are of ten) contributors of among (instead distributed now points “good”. fifteen almost quality, is its article of the Because of rating There- average review. the old fore, Rico’s than Gina’s more reviews, weighs from review voted determined new the is When article the itself. review of the quality for point extra an her h eod—admr motn a oearn to way — important more and — second The ways two are there that illustrates example above The wiki. documentation the in article new a creates Sonny timeliness - uhrhpAlgorithm Authorship — V-3 eas it because aepoesatfcsi oee,hwtekydimen- key the how modeled, is artifacts process ware hc en ata uhrhpo olbrtvl writ- collaboratively of authorship partial means which h rnlrt fteifrainta sobtained is that information the of granularity The ia o h orc ucinn fClaRve.The CollabReview. of functioning correct the for vital . uhrhpAlgorithm Authorship 5.3 evolution. artifact process software snapshots of from quality sufficient of au- information reconstructs thorship that algorithm an introduces section next and complex both therefore core and the computations, at karma is of It authorship. determines through particular responsibility that Of direct algorithm evolution screws”. the is “adjusting importance of host a CollabReview 2010). provides are Glass, they and that (Farmer context to/in project applied and domain the to adapted to uses CollabReview behavior. that developer scores influence karma of the All to considered. leads are this reviewers reviews of of anonymity aging the the and artifacts, this of In importance reviews. the automated model, and manual can from artifacts obtained of be quality the in- how and and direct responsibility, from direct computed be can responsibility sion rohrlnsta cu rqety n o-rva lines non-trivial and frequently) occur that lines lines empty other (like or moved not when them do with which authorship lines take trivial between distinction algorithm a the makes not Therefore, into is line artifact. the it process put software means has the That who developer same code. originally the source necessarily who of developer line the the is created author of line This per author text. one raw exactly is algorithm the through algorithm the of Characteristics 5.3.1 evolution the of history. of snapshots context from artifacts the evolving in other evolution algo- artifact The reconstructs texts. rithm written collaboratively fine-grained for (i.e. metadata) markup on maintaining based for algorithm (2009) an Prause presents section Sec- This (see trivial 3.4.2). not tion is quality satisfying in authorship information with metadata obtaining However, texts. ten responsibility, direct on based Responsibility primarily articles. is wiki information or code source like process artifacts software textual for and information responsibility quality from scores karma computes CollabReview euainadkrassesne ob carefully be to need systems karma and Reputation 113

V—DESIGN PROCESSAND COLLABREVIEW CONCEPT text 0 . a ) a ( p into the set a = 0 a ) and its markup. The a parent as its root, and the edges are either inherited from ε ) a a ( p = that is most similar to the text of to minimize the number and cost 0 0 a a a ONCEPT are treated equally, no matter if they C A means that the artifact gets its text from tool is retained only to identify a software ∈ A a EVIEW rithm SCM R into the new artifact. 0 For areas inserted by theand where edit they script, were determine copied from, if from and there. copy markup Get artifact’s plain text from therepository. project’s revision Find the artifact the new artifact of edit operations (see Section 3.4.3.2).becomes This the artifact new artifact’s Determine an edit script thata transforms the parent Apply the edit script to the parent artifact’s edit script leaves gaps in theoperations markup were where applied. insert (effectively converting it to The basic idea of the algorithm is that each artifact The structure with parent relationships allows soft- 3. 2. 4. 5. 1. OLLAB are new to the artifact and the whole evolution history. has one parent from whichtext it with inherits associated the markup. majority Only ofcations some its come minor either modifi- from otherexisting artifacts artifacts. or Inserting are a new new toof artifact all all artifacts the originating repository, and thenmarkup gets assigned its to authorship it beforeis it made gets up stored. of This seven process steps: 5.3.2 High-level description of the algo- come from a different branch, are marked as deleted, or from the process artifact in the repositorypurpose. space, but So only the for set this ofchanges all is software one process huge tree artifacts with and representing parent relationships between theartifacts. individual Also timefrom stamps the and revision repository revision are information in used to the insert correct artifacts order, butunderlying are model is otherwise explained discarded. in Section The 5.2). from their ancestors. Textualartifact differences and between its the parent other artifacts (i.e. they were copied into the artifact), or All artifacts were renamed. Name, branch and version information ware process artifacts to inherit most of their markup C svn ROCESSAND P , so that it can A ∈ 0 (described in Section ESIGN a ) 0 a for ( p a V—D = a ). Building this history includes two to this one (described in Section 5.3.4). 00 a “Markup” produce the same results. In the lower half, how- System.out.println("Hallo Welt!"); Figure 5.8 shows source code with authorship markup. Both search steps bestow a certain tampering resis- So as to obtain the authorship information, the algo- to English by Red,artifact and to later change it reinserted back from toSubversion German an notes by older Pink. Pink This as is author,Green. why while While it Subversion is can output originally too, author it information, is less accurate,text and or can by more restoring easily text be from misled deleted by revisions. blame comment was copied into theartifact artifact by developer from Green. a This different Subversion: is it not notes recognized the by developerment who into copied the the com- artifact asand developer not of the the text originalline (Green); developer (Blue). Likewise, the ground patterns and fonttributed colors. to the Several source authors file. con- and The went file through had multiple its iterations. nameof changed, In the the upper image, half the algorithm presented hereever, there and are some notable differences: The Javadoc mark an artifact as deletedown and name. later Second, restore it it is undertext also his not from possible somewhere to else simply andname. copy claim The clone it detection under wouldis his recognize taken own that from the somewhere text else. tance to the algorithm (Requirement-5:resistance “Robustness and against attacks”).malevolent First, developer to it conceal is or difficult forgea authorship whole for for artifact. a As theprocess first artifacts search that looks ever at existed all until software now, he cannot 5.3.3). It reconstructs the editingtwo operations artifacts. between In the the seconddetection search, on it text performs that a has clone If been it inserted finds into that the the artifact. it text transfers is the copied original from markup aother of different artifacts copied artifact, code from the tory of the evolution ofenables software maintaining process of authorship artifacts markup that tion (see 38 Defini- – search steps: In the firstfor search, the the most algorithm similar searches artifact be used as its the parent the authorship of the developer who last edited them. rithm constructs an all-encompassing unique version his- The different authors are signified by different back- (originally added by Green) was changed from German which do retain their authorship. Trivial lines assume 114

V—DESIGN PROCESSAND COLLABREVIEW CONCEPT atyls oprsn hna xasiesearch. exhaustive an signifi- than with comparisons result less approximate cantly an discusses find 5.3.3 that Section heuristics Therefore, compared. be parents to potential need its the of between all distances and compar- the artifact new of result, optimal amount an sheer for the isons: to due problem pensive of 3.4.3.2). means Section by (see defined distances is edit Similarity string it. to that similar one most is is parent optimal artifact’s An parent. artifact’s parent artifact artifact’s the the mines Find 2: Step artifacts all of set the artifact gets next only the It of stored. text are artifacts which process in software repository all revision the of implementation the text plain artifact’s Get 1: Step little a in them detail. describe more to steps seven algorithm’s the 6. Adtenwatfc otesto l artifacts. all of set the to artifact new the Add 7. idn natfc’ aeti opttoal ex- computationally a is parent artifact’s an Finding of all through go briefly subsections following The iue58 akpa bandb h loih lf)cmae to compared (left) algorithm the by obtained as Markup 5.8: Figure sinatosi otedvlprmkn the making then developer areas, the an changes. for to found authorship is assign origin suitable no If A A . 3 a a hti ob netdit the into inserted be to is that 0 = p ( a ) hsse eed on depends step This hc ilb the be will which hsse deter- step This a - uhrhpAlgorithm Authorship — V-3 0 ∈ A would hr e hrceswr netd hyaedatwith dealt are They inserted. were characters new where tp3 eemn dtscript edit Determine 3: Step ntenx steps. next the gaps in markup to lead operations edit During insertion move. they 4, when Step them are with to moves attached or is deleted, it characters when disappears Markup is content. artifact text new its the with of consistent markup again the way, artifact’s This parent parallel. the in of copy a modifies also text artifact the new into text the its of in transforms effectively found This parent its 2. to Step applied are operations edit structed script edit Apply 4: Step the reducing of significantly allowed, magnitudes are text of of deletions lines and full insertions only Therefore, 1974). cher, n string of two complexity computational between a script has edit the said finds the that be Yet algorithm can operations. editing It developer’s a operations. reconstruct edit to least of (or an- list shortest into the such string expensive) denotes usually one script transforms The applied — other. when order — correct that the deletions in and insertions of list a en h egh ftetosrns(anradFis- and (Wagner strings two the of lengths the being eadn h dtoeain eeeadinsert: and delete operations edit the Regarding m v blame svn and a oeipraty h dtscript edit the importantly, More . n . et h eeoe’ recon- developer’s the Next, rgt Pas,2009) (Prause, (right) nei citprovides script edit An m × n with a 0 markup m 115 and

V—DESIGN PROCESSAND COLLABREVIEW CONCEPT nearest neighbor with an exponen- ıve approach com- ¨ tools. diff ONCEPT C curse of dimensionality EVIEW decessor search in a high dimensional Euclidian space using R While in the original form of the edit distance both As opposed to this, the similarity of the surface rep- The Levenshtein distance is a metric based on the OLLAB based on trees perform badly in high dimensional spaces. tial growth in the number of dimensions, has not been code and make aalgorithm good can separator work for with sub-strings.up line strings The granularity into by lines breaking first.complexity and That makes reduces computational itlarge a numbers sufficiently of fast distances. metric for operations (insert and delete) haveable the costs same can cost, be vari- set,inserting too. a The line algorithm to sets the thedeleting line’s cost length for has in the characters, while costcosts of better reflect half the the effort thatto line’s a length. put developer into would have These theedit text scripts when that modifying the it algorithmthe manually. ones generates known The are from similar common to 5.3.3.2 Nearest neighbor heuristics to speed up pre- Finding the most similaran software artifact process is artifact equivalent to for solvingproblem the string editing as distance metric.fact In is represented this as space, a anpares point. arti- every The point na (i.e. everyall newly other inserted points, artifact) resulting to in a high run time complexity. is the vector model. Ittexts compares (or term strings) frequencies through in vector two tance multiplication. aims This at dis- semantic orand deep Ribeiro-Neto, similarity 1999). (Baeza-Yates resentation — i.e. the stringtent itself, — not is its more semantic interesting con- and here. move Developers lines write, and copy characters around; not necessarily algorithm uses plain string editedit distances distances that and would not rely tree Section on 3.4.3.2). syntactic structure (see number of edit operations (insertions,stitutions deletions of and single sub- characters) thatinto transform the one other. string However, computing ittion is 3.4.3.2). costly (see Therefore, Sec- tance the computation algorithm by simplifies not dis- larity, editing but sub-string on granularity. character Some granu- the characters newline like character appear very regularly in source The so-called whole semantic text blocks. For this same reason, the Weber et al. (1998) already showed that optimizations C Fi- ROCESSAND P The second pos- . If yes, then also . The more similar A ESIGN ∈ If text is inserted by an 00 cloned a ∀ ) V—D 00 that is most similar to the new a, a A ∆( ∈ 0 ≤ a ) 0 a, a facts artifact’s parent ∆( Searching for the most similarnecessitates software that process a artifact distance metricis between defined. two The artifacts search tasktance. is In then information to retrieval minimize a this typical dis- distance model artifact both artifacts are, the lesssubsequent is processing the steps. amount of A work betterbetter for match results, the here too, means because ita reduces mistake the in chance to the make later step of finding the origins of text. 5.3.3.1 A distance metric for software process arti- algorithm in Section 5.3.2.find The the artifact aim of this step is to nally, the new artifact is addedinternal to revision CollabReview’s own repository. Somestructures of that the are search neededto data during be Steps updated when 2 ahappens and new during 5 artifact this need enters step, too. the set. This 5.3.3 Details of algorithm Step 2: Find the repository. New metadata storing theship respective author- can be generated, andand fill 5. the gaps left in Steps 4 Step 7: Enter artifact into the set of all artifacts sible origin of inserted textthe is existing that artifacts it before. was not This in means one that of theconsidered totally text new if it wasfrom not where cloned, it i.e. has been noare copied origin enriched was with found. markup These that areas author has who that inserted developer as the an artifact’s text into the revision from another place, i.e. itthe was missing markup is clonedaddressed from in there. Section The 5.3.4. details are Step 6: New markup for new code Step 5: Handle copied code edit operation, then this textassociated does with not it. yet have The markup firsttext is possible that origin the of text inserted has been copied from somewhere else. This section details Step 2 from the description of the This step tries to find out if the text was copied or moved was probably written by the developer anew. Text is 116

V—DESIGN PROCESSAND COLLABREVIEW CONCEPT nivre ne seScin3452 a evl re- heavily can 3.4.5.2) Section (see index inverted An recall the while prob- a very is one added recently most the these Among h rdofi ewe erhcs n ealo the of recall and cost search between is tradeoff The candidates of sets The ainalw he ersis hr sacac htthe that chance a is There heuristics: three allows vation ae o ers egbr.Ol hs addtsare candidates those Only neighbors. nearest candi- for propose dates The that heuristics artifacts. discuss all sections of following similarity not pairwise needed; a is even result or one many only that nearest is is the problem simplifies result neighbors that the aspect that important do guarantee An to to correct. need or no search is exhaustive An there an So required. not suffices. is result neighbor approximate com- nearest be correct to A candidates of pared. applies number it the reduce Instead, to heuristics cost. computational it search to cannot due algorithm linearly An dimensions. of number 2008). Indyk, na little the from over suffer gain solutions performance All years. recent in overcome uetesto ers egbrcniae hl still while candidates neighbor nearest of set the duce heuristic index Inverted 5.3.3.4 neighbor. nearest a in candidates all testing set for cost computational the and ( prefix directory their in names only those with differ especially that names, similar with files ( candidate able ( name same the has artifact an of ancestor obser- This in MediaWiki). name in file name (e.g. article name or similar Subversion, a by identified often repository are revision a in artifacts process software Related heuristic names Similar 5.3.3.3 tested. R ( tigeiigi pc ihahg n dynamic and high a with space a is editing String S c ( NGL C x ) C P c slna ntenme feeet nteset the in elements of number the in linear is S RDCSSR ( NGL C S P NGL RDCSSR R S NGL ( P x ) RDCSSR ) ≤ is P ⊆ C R RDCSSR x C ) ( rpsdb h ersisare heuristics the by proposed S S M M ≤ ≤ N ¨ v prah(noiand (Andoni approach ıve N AME AME .Ohrcniae are candidates Other ). c c ( ( ⊆ C C ) S S C IM M ≤ S N N IM AME R AME - uhrhpAlgorithm Authorship — V-3 N ( S AME S ) S ) IM IM M N N N AME AME AME ). ). ) ie u-tig rlns epciey falsfwr pro- software all of respectively) lines, or sub-strings (i.e. fafaue ht pc hne r oeie done sometimes are changes space white feature, a of artifacts related. what closely on be hints good might provide not do Therefore, lines feature. trivial distinctive quality high one of share number that low candidates a give they count Only occurrence suitable. low are a with Actu- lines line. distinctive blank only the ally, or white-space) “ of is amount line arbitrary recurrent a highly languages, a of C-like example In typical IDE. an by generated matically trivial in results This index. of inverted set the a in artifact that of lines an needed, the is When artifact an index. inverted of the neighbor artifacts is nearest the This to appears. link it a which has in line stored Each artifacts. cess features stores index inverted The guesses. good making eto 7.1.6.2. Section distinctiveness normalized of feature/line A instead time copied. are being every lines anew Trivial re-written used. rather be probably cannot and precision, low to (e.g. artifact per } times multiple other (statistically) Some occur line. lines nearest this probable from highly guessed one candidate is neighbor there artifact line, new such a one If all has Karakos. in once of exactly artifacts occur process lines software the all of but 90,000. 30% project about example, the FreeCol For and in features), code different of of lines number of is number this the (note, FreeCol not lines different and 160,000 Karakos about has projects Karakos software two in lines index. up inverted them the looking in before them lines storing all or in space white For normal- the automatically izes code. algorithm source the reasons, of two look these tools the such standardize the by to adjusted On often anyways is code). space source white in hand, nesting other to changes due or indentation versa, vice in tab and (e.g. spaces space to white converted the character be changed longer tool no a might because feature usable good actually an that hand, like tools by automatically D .Teelnshv eybdpeitv aaiiydue capability predictive bad very a have lines These ). I hl ht pc naln dst h distinctiveness the to adds line a in space white While repeat to tends programs computer of code Source al . rsnsa vriwo h eurne of recurrences the of overview an presents 5.2 Table NDEX 8 noe orepoet o ecito fti rjc ee to refer project this of description a for project; source open An ie,ie ie htaees owieo r auto- are or write to easy are that lines i.e. lines, n ersi ok poiisfrms distinctive most for origins up looks heuristic n addtst etested. be to candidates t ( x 2 = ) − x occur | A IDE t | x sls hnthreshold than less is T < .Ta en,o h one the on means, That s. sconsidered is 0 = . 55 } trivial I wt an (with ” NVERTE T fits if 117 : 8 - .

V—DESIGN PROCESSAND COLLABREVIEW CONCEPT , a into 0 . Step 4 a ) a 1 ( 15 p 4% 5% 8% 9% 0% 5% ...... 5 4 3 = 17 54 14 leads to markup 0 a a that is most similar

. Inserted areas are 0 (abs. no.) a a (abs. no.) δ that transforms 86,621 features a

all new 0 7 1 a δ 3% 2% 5% 7% 0% 1% where sub-strings were inserted...... heuristic exploits this structure: 0 a 30 15 12 26 14 ONCEPT finds a node in three iterations using C (abs. no.) (abs. no.) for a new artifact ) have a parent. Adding artifacts one by EARCH ε S A a . Then it compares the distances between the x occurrences in all software process artifacts Number of lines that have a total of EARCH 159,580 features ∈ EVIEW S 0 REE copied code R a T REE The OLLAB and applies it. The executiongaps of in the markup of Such sub-strings may either have beenother copied from artifact, some or they are 5.3.3.5 Tree search heuristic into a single finite treethe structure. tree Each thus leaf represents (or one node) artifact. of It starts out atsearch the node root of theartifact tree. and the This search is node theIf or current its the children, distance respectively. betweenartifact is the minimal, search i.e. node smaller thanthe and the new distance the artifact between and new allsearch the search node node’s children, is then returnedsearch as continues with candidate. the childdistance Otherwise that to the the has new the artifact. minimum the Figure T 5.9 illustrates how nine comparisons in total. 5.3.4 Details of algorithm Step 5: Handle algorithm in Section 5.3.2.nearest neighbor In search the found previous the section,artifact one software a process to it, and hence isdetermines the an artifact’s edit parent script the first one one to the database implies that all artifacts are integrated This section details Step 5 from the description of the All software process artifacts in the database (except for C - N I candidates. ROCESSAND lines. Hence for rare lines. n P , which means 1 1 85 . ≤ → 0 1 z ESIGN − 1 | y 0 A ≤ | occur > x 2 = 2 V—D | − < y / for trivial lines was chosen by y A may vary for a different source | occur 2 for rare lines for frequent lines x 55 T = 1 . −∞ < y = 0 = 0 n heuristic returns a set of 1 −∞ T z for frequent lines and 0 NDEX 247,356 (or 962,972 resp.) 1 occurrence 2 occurrences 3 occurrences 4 to 9 occurrences 10 to 99 occurrences 100 to 10,000 occurrences 10,000 or more occurrences trailing white space) hasartifacts, in i.e. a different low software number process means high distinctiveness Number of occurrences x that a line (without leading & → I → y z When this heuristic makes a guess for probable can- Table 5.2: Distribution of occurrences per feature in Karakos/FreeCol software process artifacts (Prause, 2009) artifact. For this, it can useline. the A occurrence line counts of with each onlydistinctiveness. a All low artifacts occurrence found count through hasindex the a are inverted high proposed ason candidates. to pick The the heuristic nextcandidates most goes has distinctive line reached until the the desired set cardinality. of The corpus, it need not be adaptedof individually. lines The have majority only a few occurrences in the sourcelines. code. didates, it first picks the most distinctive lines from that looking at empirically determined distinctivenessIt values. corresponds to a distinctivenessthe of line has about 15 appearancesthe in optimal 100 value artifacts. for While so that In the above formula, is the number ofcardinality appearances of of software a process line artifacts. divided It by follows that the VERTED Table 5.3 lists some distinctiveness examples of different The threshold with with 118

V—DESIGN PROCESSAND COLLABREVIEW CONCEPT seScin3433.Hwvr n ytci structure syntactic any However, 3.4.3.3). Section (see aldmarkup called iue59 he trtoso resac (Prause, search tree a of iterations 2009) Three 5.9: Figure a origin. probably original more the is the origin context, certain matching considered: the be line larger can a lines the for surrounding found of origin context possible its one then than more However, is structure. there basic if this on rely detection only Clone therefore artifacts. may process in software available of necessarily texts not the is lines separable from apart situation of similar kind a special with a deals is and It retrieval code. information source origins moved the or finds copied detection of Clone found. be cannot from twofold: is gaps with Dealing yet. them i o,dtrieweetesrn a ae from. taken was string the where determine not, if and 2. new, all are contents gap if decide 1. tigi l e,i noii hr twscopied was it where origin an if new, all is string A al .:Eape o omlzddsicieesso aasuc oefo aao Pas,2009) (Prause, Karakos from code source Java of distinctivenesses normalized for Examples 5.3: Table gaps steei omru sindto assigned markup no is there as String } { e) * (IOException catch } result; return owlModel.get java.io.IOException; = import name String final org.ap (java.lang.String) return File(directory).mkdirs(); File(argv[0])); new new Limbo(".", new / - uhrhpAlgorithm Authorship — V-3 99.Frcoedtcin h loih rsne here presented algorithm the detection, clone For 1999). ihec ftesalrgp.Nwi ahgp tneeds it gap, each in Now gaps. smaller the of each with hslt h ln eeto ra h asseparately gaps the treat detection clone the lets This rtslt otaepoesatfc noislns Each lines. its into artifact process software a splits first al., et (Ducasse representation internal an into to code step bring transformation a involves always detection Clone sub-strings of origins possible Identifying 5.3.4.1 h eto rvosln ntegp oee,growing However, gap. the in line previous include or also next to the grow to tries line covering a (overlapping) of Each context the honor overlays Origin 5.3.4.2 section. next the in described as processed temporarily further them are of (called line all respective occur- to the original references overlie more line, or a one of is the rences in there occurrences If original up index. the looks inverted normalizes and it gap, line, the each in For lines lines. many or one into and artifact new artifacts. the old in in lines lines between mapping a find to dealing when effort computational the reduces this as at substrings inserts gaps. disjunct often several creating it thereby one, positions, distinct new a into artifact maintain to need not does itself. it it so 5.3.3.4), Section in- (see inverted the tics same by the maintained use is can that It dex index. inverted an in line another. and from fingerprint line line’s one the distinguishing considered help are line con- a and of formatting tents remaining and All trailing space. leading, white omitting surplus by normalized then is line h ln eeto pistetx ntemru gap markup the in text the splits detection clone The script edit When a stores and hashes algorithm the normalization, After [...] [...] 247356 x δ 20612 a occur 2144 2130 1069 0 222 143

93 a 1 rnfrsa led inserted already an transforms 0 0 0 0 0 0 0 0 0 I ...... t NVERTED 000 153 823 824 907 980 987 992 999 ( x ) covering I NDEX eoethey before ) heuris- n 119

V—DESIGN PROCESSAND COLLABREVIEW CONCEPT . The nec- a

0 a δ ONCEPT C EVIEW R A modified Levenshtein distance is employed to com- OLLAB not get into conflict with the environment’s constraints. prototype have led to the developmenting of schemes. a set of reward- character-based one to reconstruct the “how”It of normalizes changes. white space andblocks favors of long lines. consecutive 5.4 Rewarding Schemes Karma scores alone dokarma not scores change must behavior. be deployedpositive Instead, in or a negative way that exchange they valueSection create for 3.5.2.2). the A users rewarding scheme (see forms does karma this. scores It into trans- exchangesection value for summarizes users. the This general design considerations. scriptions of the respective experiments. 5.4.1 Karma scores and exchange value the primary concern of thisimportant thesis. one, They but are a a still tool tool — — an to create anthat exchange karma scores are deliveredsame to time the they users, minimize while undesired at side-effects the and do artifacts that are marked asfull deleted. history of It the is artifactif and aware an does of not artifact the lose is information temporarilyto deleted evolve. but Line-based later edit continues scriptstifact’s allow markup keeping synchronized the with ar- the artifact’s content pute distances between artifacts inrelatives order and to generate find short closest editessary scripts nearest neighbor search issearch facilitated by heuristics a which set are of comparedSection to 7.2. each other Substrings in insertedclassified by into (i) the strings edit copied from scriptcluding somewhere are obsolete else (in- artifacts), and (ii)of new new strings. strings Markup (case ii)mitter. is assigned Copied to strings the (case artifact’s i)the com- retain most the probable markup original from sourcenal of sources of the strings string. aredetection Origi- determined algorithm. with a The basic clone detection clone lyze does changes not on ana- a deep semantic level but on a surficial value for karma points. A successful scheme ensures The various field experiments with the CollabReview The actual rewarding schemes are presented in the de- The development of a suitable rewarding scheme is not while it evolves. C ”) is at- ROCESSAND P ”) from “Source exit(0); ESIGN , it is overridden by “Source V—D i++ // print greetings In a final step, when covering was resolved, markup Figure 5.10 presents a case where there are four poten- approach is different from usualat metadata the that crude is level stored ofity software process that artifact loses granular- information whenfrom sub-strings one are artifact moved to another.granularity Metadata also at leads artifact to level inaccuratedisappears information during if artifact text evolution. Instead,rithm considers the parallel algo- branches of artifacts, and “dead” be the author of the respective text. 5.3.5 Summary of authorship algorithm grained metadata with collaboratively written text. This is copied from the identifiedhas origins. been Text found in to the not gapnewly have that created an authorship origin markup. is associated Themation with authorship comes infor- from the revisioncommitted log: the The revision developer to who the repository is considered to a prevalence of 2.mapping (“ However, the2” second is in line turn overridden oflonger by the (four “Source 4” lines). because The ittributed last is to line even “Source (“ 3” althoughBut it this only line has is a not length claimed of by 1. any other overlay. the bigger one prevails.covering exists So for each in line. the end, only a single tial sources for the content ofhas a a gap. match Although in “Source the 1” 2” line because it is a longer mapping, and therefore has 2009) by a line is onlythe allowed origin if also the match. next or Whento previous all their line coverings maximum in have extent grown possible,flicts any are overlapping con- resolved. If two origin mappings overlap, then Figure 5.10: Selecting origin by size of footprint (Prause, This section presented an algorithm for associating fine- 120

V—DESIGN PROCESSAND COLLABREVIEW CONCEPT hl uihethsavr togiflec nbehav- on influence strong very a has punishment While elmnyi prcae CsyadRcado,2009). Richardson, and (Casey appreciated is money well Pas n udk 2012). Durdik, and (Prause h olwn usin a epwe utn reward- a suiting when help can questions following The eto swne n eesr,teitreto can intervention the necessary, and wanted is vention nocmn tutrsmyntb vial rntbe not or available be not hierarchical may or structures projects, enforcement source open be some not in may possible compensation monetary instance, For ment. source open in than role weaker much a among have karma may contexts, peers industrial unfold In not effect. does it desired or its aside, goals karma other pushing of interven- value scores, miscalibrated exchange A the overemphasizes weak. either too tion or strong too a be inter- some While they life. if user’s behavior, the in user intervene change somehow only will scores situation Karma the to scheme rewarding a Fitting 5.4.1.1 n ceet h iuto Pas ta. 2010b): al., et (Prause situation the to scheme ing how in and differences choice, cultural best and the interpersonal always are mone- not there be is to money have fact, In necessarily tary. not does herself. reward desires the traded individual the But be that can else Money something for for monetary. in value be exchange can the points case, karma simplest the Section In (see is 3.5.4.1). effect motivational its strong how person, choice. better rewarding the make often side-effects demotivating its ior, side-effects. unbearable create out to ruled expected be is could it scheme because rewarding a unethical. cases, be other much In pretty may punishment application the physical and of rewards, monetary of use the restricts projects. consortium in enough strong • • • • • • a for reward a of value subjective the on depends It can contracts role: a play considerations legal Also imple- to impossible be may scheme a Sometimes osi o xld eti users? certain exclude not it Does context? social the of administrative and structures organizational the into fit it Does cost? financial maintenance and to efforts regard with both cost-effective, it Is nor strong too be weak? not effect too the of strength the time? Will long a for attractive be to continue it Will user? every for attractive it Is - eadn Schemes Rewarding — V-4 safis tpt nesadteefcieeso different of effectiveness the understand to step first a As effective. be might schemes rewarding what hc esnnaoyos( non-anonymous lets which strong a have and users to acceptable both be would ( ( ( ( ( ( hypeitdta hwofrwrsaen more no are rewards show-off that predicted They hrceitc frwrsadfrnvgtn nti com- this space. in plex navigating for and rewards of characteristics The resulting dimensions. different on based of schemes categorization rewarding a introduced 3.5.4.4 Section rewards, rewards of Effectiveness 5.4.1.2 at hint to aims section next The karma the employed. where is domain the system to adapted rewarding be the can Then scheme environment. pos- social and the constraints of the best. sibilities understand work and to at look expected must and One possible, be may schemes nemtetyo otnosy(IvC). continuously or intermittently ( take-away than effective ( targets specific ( real ( system. continuous or intermittent an towards indifferent ( groups lets which preferred scheme survey rewarding the in a subjects the space, incentives the dimension: each about questions two asked It effect. characteristics what investigated It scientists). non-student computer male mostly colleagues, (all environment GvL CvA AvP VvR SvT CvA Wihatraied o hn smr effective? more is think you do alternative Which 2. more? like you do alternative Which 1. tdpnso h oilstainwa rewarding what situation social the on depends It • • h ujcscniee ceems effective most scheme a considered subjects The from dimensions different the of terms in Described work a from subjects 16 interviewed survey small A hog pcfi agt ( targets specific through ) ili eacpal oalstakeholders? all to acceptable be it Will self-effective? feel to going users Are n ulcyvsbe( visible publicly and ) a.Te a osgicn rfrnefrreal for preference significant no had They way. ) VvR napbi ( public a in ) ( proclaimed and ) ( durable ) netvsspace incentives ,polie ( proclaimed ), DvC TvO PvC ead ( rewards ) napeitbe( predictable a in ) PvD SvT ots o ual ( durable for contest ) PvD a eue o leigthe altering for used be can AvP ead,o personalized or rewards, ) PvC nsta r ihrgiven either are that ones ) ead ( rewards ) rus( groups ) TvO eut.Te were They results. ) PvR napredictable a in ) o take-away for ) PvR IvG IvG GvL compete ) through ) achieve ) way. ) DvC IvC 121 ), )

V—DESIGN PROCESSAND COLLABREVIEW CONCEPT pendant to PvR is not the direct ONCEPT C EVIEW R individual penalty OLLAB loafers. But, as a negativetension side-effect, and this also friction creates insideprizes the should group. be combined Therefore with team individual rewards, too there is an award ceremonythis in is the not closed necessary, group, then while others are aware of the prize. show-off but to take away. Prizesachievements are at given predetermined for targeted times,nature. and are of real 5.4.2.4 Individual Penalty the individual prize.threshold. A penalty If is the boundpreviously user defined to transgresses moment a in this time, certain thresholdapplied. then at the As penalty there a is is nopunishment award is ceremony, the more individual anonymous andthe internally group. closed to 5.4.2.5 Team Prize Similar to the individualto prizes, a a group team of prize individuals.is is that The given it advantage honors of teamworkthe this and prize group. distribution of Members work ofhigher in efforts the through peer-pressure group and push social sanctions, each other to In general, ranks continue toalways work possible for ever to because collect it morethe is and rank in more the points. table is But a purely virtual reward. 5.4.2.2 Shame ranks Shame ranks are very similarstead to of the presenting praise good ranks but karma,Basically, in- they the present feature bad that karma. distinguishes themranks from is praise that they builders, on punishing and the not worst praising perform- the best. A shame ranks scheme 5.4.2.3 Individual Prize to the top-performer, there maya be draw. a In lottery principle, the in prize case is of given anonymously. If vidual top-performer. While it is basically guaranteed The prize is something durable for the winner, not to The The individual prize is a reward that is given to an indi- (Heneman and Hippel, 1995). while weeding out hiding places for free riders and social was used in the TornBond trial. C Self- ), and , PvR ) rewarding ROCESSAND P ), personalized PvD CvA ESIGN V—D as their main motives (Prause Curiosity and Learning ) and proclaimed ( TvO Community and ) rewards in a competitive ( VvR ), targeted ( Praise ranks were used in most field experiments be- Section 3.5.4.2 presented an overview of typical psy- Comparing data from above sets, the difference be- AvP than the others. And theto reward everyone: of a being higher better rank is in clear the table. cause they are cheap, easy to realize,the and ranks effective. were There visible mostly onlysome inside occasions the externals team to but the on group would get insight. an outstanding first rank,punishment poor in performance the results in formHowever, of the being worse ranks the do lastence. often in Ranking have positions the a are table. lower usuallynot pres- durable: someone if else there who is overtakes a user, then thatis user guaranteed to the individual as long as he is better ing a continuous incentive through showing-off.ranks Praise are a highly competitiveonly one reward first as rank there (Farmer cana and be competition Glass, does 2010). not As degenerate long intobut personal as stays conflicts friendly, it is known to bring the best results 5.4.2.1 Praise ranks to their score. The table is constantly visible to all, caus- section disassembles these experiments toing distill patterns emerg- from them, anding categorizes the the dimensions patterns of us- thepresent incentives the space. actual It rewarding does scheme not of the experiments. et al., 2010b). 5.4.2 Reward patterns the various field experiments of CollabReview. This system. chological work motives.the Asking subjects for considered thoserealization motives, tween what subjects likeslightly and shifts what opinions they towards punishment feelreal ( effective ( validation in the evaluation chapter. The praise ranks is a leader board rewarding scheme, The rewarding schemes are detailed later as part of the (Humphrey, 1997). While top users are rewarded with ( will maintain his rank. Progress and success in the table where all users are sorted into a ranking table according A lot of different rewarding schemes were realized for 122

V—DESIGN PROCESSAND COLLABREVIEW CONCEPT world. ... Notifications 5.4.2.9 dur- most contributed who user individual ( the award to week given the is of not Hero the title week) meant (e.g. seriously interval too time predefined a per Once week the of Hero 5.4.2.7 2011). al., drive et users’ (Dencheva the achievement addresses for mechanism This (Dencheva 2009). icon al., corresponding user’set a a shows to next often certain indicated and a is name, acquired level Each has points. user of the A number if newbie. reached a is as level starting new initially level, a has user Every Levels 5.4.2.6 icvrbe n ueyvirtual. purely and discoverable, somewhat shown-off, not individuals, for posi- achievements consumable guaranteed, or are or neutral notifications positive However, often tive. be most can are feedback notifications re- While negative, in events. users to to sponse feedback private provide Notifications virtual the shown-off to limited are mostly time, are certain but community, a the at to awarded are dependent not luck, are on consumed, be cannot (Agaraj, Medals off 2012). them given show instance can first individual the the in anonymously, are medals specific While a score. reaching karma like individu- goals by targeted achieving earned for are als that awards virtual are Medals Medals 5.4.2.8 even audience. potentially wider and a unit), to mem- (or all organization to the setting) of real bers but non-virtual (in announced the is re- been having being a possibly as without unless membered lost behind; is benefits it any otherwise re-earned, leaving be to has title the CollabReview, for tested the rewards other As many to 2011). opposed al., et (Dencheva extensive contribution rewards short-term It but newcom- something. for achieve chance quickly a to is ers mechanism This time. that ing HotW scnual,ie fe eoigthe becoming after i.e. consumable, is HotW h title The . - umr fDesign of Summary — V-5 HotW HotW HotW ) , hra ubrcoe to closer number a whereas CollabReview, with experiments the In 2012). Agaraj, a xeddadrfie,wiedvlpetcycled development while refined, and extended was documen- software internal improving of goal the with ten- slight a have trials field the in implemented were ef- most be to considered they what and scheme, warding h ubr xrs o togaseiccharacteristic specific a strong how express numbers The to try who others to support and guidance provides This hsrsac olwda trtv rcs.I started It process. iterative an followed research This h ih n ftesetu.Freape h team the example, For spectrum. the of end spectrum, right the the of end to left closer the like number more A is pattern the pattern. that a in present is space. incentive the of dimensions to regard with patterns environment. their to scheme rewarding a suit space. incentives and the of extracted dimensions were the patterns within few classified a realized. schemes, were the schemes From rewarding 2010; different of Glass, series and a Farmer (e.g. own its a of building science well, a Indeed, environment its is into one. well fits best that choice, the scheme rewarding proven necessarily and not simple is a it is is money value exchange While an effective, needed. be to scores karma For schemes rewarding of Summary 5.4.3 hog ein mlmnain ra n evaluation and requirements trial of implementation, set design, The through goal. this require- of to set related initial ments an and reputation, through tation Design of Summary 5.5 for. wished very least as were regarded ranks were shame corre- while prizes Pearson effective, team are and presented Prizes values lations. The pattern. ing with matches users’ pattern the rewarding a well how show rows environments. and project constraints later other the to of due possibilities regarded wishes fully user the be the However, what not towards have. could to one liked strong users a set and initial experts, of set a determined by as characteristics “best” the towards dency that schemes rewarding the estimated, Roughly fective. re- a in have to wished they what regarding interviews personalized dimension. AvP mostly but the anonymous on bit a is pattern prize al . rsnsa vriwo hrceitc fthe of characteristics of overview an presents 5.4 Table hl aigfwsaitclsgicne h w last two the significance, statistical few having While user the from results the present columns two last The ihdfor wished r p w and 1 osdrdbest considered en hti smr like more is it that means r p b 0 reward- means 123

V—DESIGN PROCESSAND COLLABREVIEW CONCEPT - - 0.8 0.6 0.4 0.1 0.6 0.6 0.1 0.3 0.8 0.5 0.1 0.7 Best - - 0.6 0.4 0.4 0.1 0.5 0.6 0.4 0.4 1.0 0.8 0.3 0.6 Wish 1 1 0 0 0 1 1 0 0 0.4 0.3 0.7 0.22 -0.04 Notif. 1 0 0 0 0 0 1 0 0 0 0.3 0.7 0.34 0.31 Medal ONCEPT C 1 0 1 0 0 0 0 0 1 0 0 1 0.40 0.56 HotW EVIEW R 1 0 0 1 0 0 0 1 0 0.7 0.7 0.3 0.39 0.51 Level In the end, when karma scores for the developers are OLLAB detection (Section 5.3). available, they lack aCollabReview well-calibrated feeds the exchange karma value. scores intoscheme a to rewarding generate the exchange value. The exchange have to. Other possibilitiesinto are the countless. building of A successful firstbeen rewarding step made schemes but has the topicdealt would with be in too this broad research. toa be Instead, fully CollabReview sufficient uses rewarding schemeexchange as value a for tool itsdoes karma to not scores deliver research (Section an into 5.4) buildingscheme. but the After optimal all, rewarding whencan humans be are involved, surprising. results rewarding scheme It into a is project my environmentand is difficult experience complex due that to the fitting manyfactors social that a and sometimes organizational rule outmore rewards, or or less make effective. others of information retrieval, string manipulation, and clone value can be of monetary nature but does not necessarily C 0 0 1 0 1 1 0 1 0.3 0.3 0.3 0.3 0.71 0.57 Team Prize ROCESSAND 0 0 0 0 0 1 1 0 1 P 0.3 0.3 0.7 0.61 0.29 Prize ESIGN 0 1 0 0 0 0 1 0 0 1 0 1 0.10 0.06 Penalty V—D Table 5.4: Overview of characteristics of the reward patterns 1 0 0 0 1 0 0 0 1 0 0.7 0.3 -0.71 -0.07 Shame 1 0 0 0 1 0 0 0 1 0 0.7 0.7 0.10 0.04 Praise w b p p Authorship information for the responsibility and CollabReview models the software process artifact SvT TvO VvR r r GvL IvC IvG PvC PvD PvR Dim. AvP CvA DvC trivial. The optimal parentdeveloper’s needs to changes be found, have andsnapshots to of the software be process artifact reconstructed evolutionrevision from repository. the from The algorithm relies on techniques the quality of artifactscomputes robust into karma its scores for model.information developers from in the its In model the (Section 5.2). end, it timeliness computations is maintained asmetadata on fine-grained the text content of software process artifacts. evolution as a tree withbetween parent two artifacts. and child By means relationships of its model, CollabRe- the authorship for artifacts) andReviews, which timeliness are of written reviews. manually orare automatically, CollabReview’s means of getting information about phases. Most of theprototype, work which based was on developed the andthis CollabReview enhanced process during (Section 5.1). view can assess key factors like responsibility (based on Yet tracking this metadata in acceptable quality is not 124

V—DESIGN PROCESSAND COLLABREVIEW CONCEPT C h olbeiwPrototype CollabReview The ae ..isacietr n ein eto .. gives 6.1.1 Section design. and architecture its i.e. ware, formed is system this of part technical The work. hsscindsrbsteinrwrig ftesoft- the of workings inner the describes section This aiainadeauto okta r rsne in presented are that work evaluation and validation hpe 6 Chapter h niiulsbytm fCollabReview. of summarizes subsystems 6.1.2 individual Section the organization. an of tructure ( Technology Information it the how into and integrates prototype, CollabReview the of overview an Architecture 6.1 prototype. the design of architecture, implementation the and presents chapter This the 7. for Chapter used was and concept, the that alongside software prototype evolved a The is concept. CollabReview It the implements prototype. CollabReview the by 1 ie fe aakn(1964) Barankin after cited HAPTER oilcneto otaedvlpetproject the development in software solution of technical context social a of system technical ecie olbeiwa socio- a as CollabReview described 5 nin aaeepoem Japanese Ancient — ri ttemeigo h w htrings? that two the of meeting the it is Or aet hmk oa anaru. ga ai no shumoku to Kane hmk anr ka. naru ga Shumoku si h amrta rings, that hammer the it rings, Is that bell the it Is aeg auk ya. ka naru ga Kane IT infras- ) 1 h oeo h olbeiwpooyeisl scalled is itself prototype CollabReview the of core The application web-based a is prototype CollabReview The eomn raiain hl tcnb sdt improve to used be can it While organization. velopment .. vriwo h rttp’ archi- prototype’s the of Overview 6.1.1 evrwt aa2Pafr,Etrrs dto ( Edition Enterprise Platform, 2 Java Windows a or with Linux server dedicated a on runs server neces- labReview the company’s access the prototype through the information lets sary and develop- team, all the to of accessible ers easily it makes which cation, the infras- IT into prototype the of Integration 6.1.1.1 decision. suboptimal of a kinds different alike treating documentation makes reward- which the — to scheme own respect ing its with has instance documentation for of — kind peculiarities due every is that This fact articles. the wiki to or code docu- source of type e.g. speciﬁc mentation, one only handles instance CollabReview one of documentation, internal of kinds different the into integrates that olbeiwserver CollabReview tecture tructure IT ti aabsdwbappli- web Java-based a is It . nrsrcueo otaede- software a of infrastructure LAN h Col- The . J2EE 125 )

VI — THE COLLABREVIEW PROTOTYPE , flag name and the ) and when password a ( – which are p disabled ) 0 = flag that grants 0 a, a a created (name, branch, revi- ∆( who created the artifact artifact – so that artifacts can be of the artifact’s content – that is used in the database distance identifier administrator author parent markup obsolete class holds information about the objects, which can be mapped trans- primary key class models agents that contribute to class represents the authors’ reviews of string to enable fast access for internal that have been imported from the revision of the review’s author to connect the re- A ∈ ROTOTYPE name a content Author P Artifact Author Review address so that an agent can receive notifications schema to relate it to other entities, the are used by some responsibility metrics, the dates when the artifactit was was marked filtered by specifying a time, e.g. only artifacts that the authorship a numerical the artifact’s unique sion) as given by the revision repository, its computations of CollabReview, a reference to the according similarity needed in timeliness calculations, a reference to the by committing it to the revision repository – which view to its author, were “current” at a certain time, and which is the basic input of responsibility metrics. • The The • • The • • • • • the project. An agent isstring, uniquely which identified usually by is its theagent user is name also under which known in the tion the to revision repository. this, In an addi- agent can have an encrypted email from CollabReview, an extended rights to theand user changing like configuration settings, viewing and serverthat a status revokes log-in rightsdeveloper from has real left the persons, team. e.g. if a artifacts including the quality information. A review has in the main program andare the database the (an exception here parently to the revision repositorymanagement). if it These has are its the ownthe user elements data that model implement that was presented in Section 5.2.1. artifacts repository. It consists of which is needed for authentication when logging in, an EVIEW R , OLLAB ) in the C plug-in for HE Artifact RAM in Java Server IDE for the database. GB VI — T data persistence frame- KLOC 3 . The server front end, which . Normally they are present both KLOC with InnoDB engine by default because as container. Review 4 2 s). To run smoothly, it needs about 2Giga ) of Random Access Memory ( and JSP GB http://tomcat.apache.org/ http://www.hibernate.org/ http://www.mysql.com/ 2 3 4 In addition to the core application, the prototype also The CollabReview server consists of about 400 Java CollabReview needs full read access to the revision CollabReview requires access to a SQL database to opers’ machine. For example, there is an their reviews of source code backHowever, these to features the were core not application. used in field trials. 6.1.1.2 Core data structures of the prototype Updating the internal software processmodel artifact with evolution new artifacts cansive be but computationally does inten- not require special hardware. consists of separate program parts that run on the devel- classes with about 32 is a web interfacethe to server, the adds most an importantPagess additional functions ( 5 of Byte ( to be analyzed because itfacts. will Access need information to to the analyzethrough repositories all the is arti- configuration provided data. Forof example, source in code, the it case cangranted use access credentials to of the a Subversion useralso for that read this was its purpose. data It by can a accessing wiki, the directly. database, e.g. that of that is free for opengine source supports projects, transactions and to its preserve InnoDB thedatabase. en- integrity However, of Hibernate the supports othermanagement database systems and their SQL dialects as well. repository which contains the software process artifacts facilitated through the Hibernate jects into the relational schemamanagement of mainstream systems. database uses The MySQL CollabReview prototype MySQL is a widely used database management system application container. All instancesused Tomcat in the field trials manage its internal data. The database operations are virtual machine, and several tens of Author The key data structures in CollabReview are VisualStudio, which developers can use to easily report work, which allows simplifies the mapping of Java ob- 126

VI — THE COLLABREVIEW PROTOTYPE n e riat on r ucsieyipre in imported successively are found artifacts new Any seas 6.1.2.3). also (see AuthorManager h re nwihte eecmitdt h revision the to committed were they which in order the model. internal the updating for used be mech- could notification that a anism provide revi- or all support not repositories because sion necessary is This software the artifacts. of process revisions updated for repository revision repository original their from artifacts importer copy through that realized classes is access The perfor- reasons. for mance authorship like information precom- additional with puted augmented and key contents, artifact like contains information that of history model repository a revision creates the It evolution. its artifact build of to model repository internal revision the from artifacts CollabReview all reads artifacts. process kind software each project’s for a required of is CollabReview of instance One CollabReview into artifacts Importing 6.1.1.3 object. singleton the to reference back struc- the through data realized key for is these Access to tures. The access application. facilitates whole singleton the of together links modules that individual element the core its as singleton run-time olbeiwrglrypls(vr e iue)the minutes) few (every polls regularly CollabReview the has server CollabReview The • • • • • Artifacts iwn n authoring. and viewing atrcmuain fkrasoe eae ore- to related scores karma of computations faster cached the and rejection, the the fend of string and description rejection, a the potentially for responsible re- was to who and member calculations quality artifact from clusion was review the who be should review the if instead, or should unrejectable review others, the to of revealed author be the either if is decided review the if flag a computations, score quality into quality the provided, description review’s the rating timeliness and for stems motn ingredient important most the as Author Reviews eetdby rejected anonymous au fterve oallow to review the of value text ,wihalmiti a maintain all which s, n hog the through and , htteato has author the that CollabReview omr tfrex- for it mark to ota tcnbe can it that so Repository reason I1—Architecture — VI-1 ode- to ii.I re oipoeuefeunyo h exten- the of frequency use improve to order In wikis. ihalwts oeaercieabdrating. bad a receive coverage test low a with h olbeiwsre rvdssvrlitrae for interfaces several provides server CollabReview The aaatfcsadte oncek hi tl.Check- style. their checks soon then and artifacts Java nterIEt rvd emesitgainwt their with integration seamless provide to IDE rea- their same in the for For plug-ins possible. are as there small sons as be should contexts use two of the between distance perceived the sions, e.g. applications, other of em- interfaces be web can the that into form bedded server review a the with Additionally, widgets web there. provides review a submit artifact, to process in- and software a web server’s to navigating The allows terface system. the into reviews feeding prototype the into reviews Inputting 6.1.1.4 needed. if algorithms differ- authorship by ent computed information authorship of with layers markup several maintain com- can CollabReview responsibility putations. the asfor also markup model, authorship internal signing the into integrated trans- are and they formed artifacts, the importing While repository. fafce riat satmtclyrdcd Depending reduced. automatically timeliness is the artifacts repository, affected the of to The added is are review user. artifacts same submitted new newly the a by it) of of timeliness version process older re- software (or potentially same artifact the and for database, review existing the an in places stored gets it ject, its submit then and build the instance, For Cobertura submit CollabReview. then to and report process, project other their some into or integrated build be nightly can the They server. the of independently the to submitted and Repository collected, are events audit style’s new of notified gets server, CollabReview imple- the arbiter in Checkstyle mented a example, As occurred. has artifacts updated the or in new for check regularly CollabReview They the server. inside process run software arbiters of Internal analysis artifacts. automatic an on informa- based review tion generate 5.2.4.2) Section (see arbiters (cf. server the 6.2). by Section provided are that reviewing web for the services use plug-ins Such environments. development hnarve ssbitdt the to submitted is review a When feedback, human for interfaces these to addition In 5 http://cobertura.sourceforge.net/ Repository 5 etcvrg olcudrnatrasuccessful a after run could tool coverage test beta eiw.Etra riesrun arbiters External reviews. as object rgtntfidb twe change a when it by notified get or , XML IDE ,wihuescninstall can users which s, eot ota artifacts that so report, Repository 1 . 0 When . 127 ob-

VI — THE COLLABREVIEW PROTOTYPE anonymous class diagram of candidate can be UML class, which holds the HotW : A nested structure of : Creates and deletes user ac- to prevent unauthorized submis- Review CollabReview and ROTOTYPE labReview prototype P key and value pairsthe to server, stored configure for all run-time (re-)configuration. elements of AuthorManager counts for developers, reviewerstors. and administra- ConfigurationData Figure 6.1 shows a simplified A user may be an administrator. Administrators are Some of the read-only services that a CollabReview When a user authenticates to the CollabReview server, • • ated with the user. Aact user with who a wishes protected to service directlyneeds in inter- the to CollabReview present server, a validtions session are, ticket. for example, Directthe accessing interac- server the with web a interface web of browser, or accessing application 6.1.2 The major subsystems of the Col- perspective of its internal structure. the CollabReview server. Abstract classes/interfacesprinted are in italics. The diagram only includestioned in classes other chapters, andods only some of fields the and actual meth- graph implementation. is the At thekey components root together: of the normal users but in addition toto this, the they get administrator access area rights ofadministrator a area CollabReview server. is The a set of web pages in thefiguration CollabRe- options during serverlabReview server. run-time of the Col- server offers, are unprotected. Priornecessary authentication to is access not these services. For example, the ser- accessed freely. However, most services likereview entering into a CollabReview, require authentication eventhe if review is stored into theflag. system using The the reason is that there has to besion a of large link numbers between of reviews.only Every have one reviewer should (possibly weighted) vote per artifact. an internal session object is created and becomes associ- view server, where administrators can change some con- vice which outputs the current Author This section describes the CollabReview server from the web services through an IDE plug-in. which are either central to the architecture or are men- EVIEW R OLLAB C HE can also imple- . AuthorManager VI — T AuthorManager AuthorManager A user does not necessarily have to be a natural per- Normally, the user data of all authors is stored in Typical components that can be configured as plug- In addition, there can beto) implicit directly users interact who with do the notone CollabReview (need server. who Any has contributed toautomatically the get revision repository, an will implicit user account in CollabRe- CollabReview, karma scores for them willcally be computed automati- without further administrative efforts. and users, as they onlyin need one to system maintain (see user also accounts and Dencheva Agaraj (2010) (2012) for for MediaWiki GForge user management). son. It can besoftware metrics any running agent, inside the i.e. CollabReview server. including daemons and the database. But an ment transparent access to the user management of the re- can map users throughment. to This reduces MediaWiki’s the user effort manage- for system administrators User management and authentication ofdled users through are the han- ins are the internal datafilters, model the and importer, storage user engine, management, input score reputation/karma metrics, service daemons, weband interfaces, algorithms arbiters, for responsibility, similarity, qualitysessment, as- and weighting of artifacts. 6.1.1.6 User management and authentication enables the prototype toent be environments. flexibly The adapted plug-ins to toconfigured use differ- through are the defined master configuration and bundled file with that the is server atfile build can time. later The also configuration bebeen modified deployed when to the the application server. has change notifications. 6.1.1.5 Plug-in architecture Most of CollabReview’s subsystems andcan be components configured and exchanged using plug-ins. This on the amount of changesince the that last happened review, to new (versions thedates of) artifact to reviews the and timeliness up- automaticallyto propagate the through karma score metrics that have registered to receive view. While such users cannot automatically log on to vision repository. For example, the 128

VI — THE COLLABREVIEW PROTOTYPE ... getAuthorRank(author):integer getAuthorScore(author):float getName() endVisiting() visit(artifact) beginVisiting() filter rejectionReason:string rejectedByName:string isAnonymous:boolean timeliness:float rating:integer text:string authorName:string artifactIdentifier MetricReviewAgedCount metric:list MetricTechnicalDebt WorkerDaemon MetricMetaMetric MetricQuantity ReputationMetric MetricQuality Review {Hookable} 0..n 1 0..n 1 1 1 workerDaemon reputationMetricManager repository measurementsManager mailManager lifecycleManager importer configurationData authorManager listReputationMetrics() findReputationMetric(key) addMetric(metric) getDefaultFilter() setDefaultFilter(filter) reputationMetrics:list ReputationMetricManager CollabReview 1 1..n 1 iue61 ipie ls iga fClaRve server CollabReview of diagram class Simpliﬁed 6.1: Figure revision branch name 1 ArtifactIdentifier 1 1 getReviewCount() listReviewsBy(author) listReviews(artifactName,branch) setReview() getReview() registerSetReviewHook(hook) registerPreAddHook(hook) registerPostAddHook(hook) size() listFilteredArtifacts() listNonObsoleteArtifacts() listAllArtifacts() getEmptyArtifact() getArtifact(id) commit() removeGrouping() getGrouping() createGrouping() hooks:list artifacts:list reviews:list 1 1 BasicRepository 1 1 1 1 1 1 1 {Hookable} Repository Markup 1 artifactWeight artifactSimilarity artifactQualityAssessor artifactResponsibility I1—Architecture — VI-1 MeasurementsManager 1 1 MailManager measure(artifact) ArtifactWeight 1 1 1 LifecycleManager hooks:list obsoleteDate:date createDate:date distance:integer ancestorId creatorName:string markup content:string artifactIdentifier 1 artifact,string,maxEditCost) findTransformScript( HibernateRepository {Hookable} Artifact LineCountArtifactWeight TokenEditSimilarity 1 0..n WikiArtifactWeight SizeArtifactWeight ArtifactSimilarity configurationData ConfigurationData UnityWeight 1 1 1 1 importWorker 1 1 DefaultArtifactQualityAssessor Importer reviews) computeQualityFromReviews( assessQuality(artifactId) ArtifactQualityAssessor 1 HibernateAuthorManager 0..n 1 WikiAuthorManager getPriority() listResponsibilities(artifact):list ofAuthor assessResponsibility(artifact,author) AuthorshipArtifactResponsibility LatestContributorResponsibility 1 1 EditorArtifactResponsibility 1 ArtifactInstantiatedHook Hook ArtifactChangeHook commit() listAuthorNames() storeAuthor(author) getAuthor(name) createAuthor(name) contains(author) authors:list SetReviewHook PostAddHook ArtifactResponsibility PreAddHook authorManager 0..n 1 disabled:boolean administrator:boolean encryptedPassword:string email:string name:string work() nextPollDelay() getProgressInfo() cancel() 1 SubversionImportWorker MediawikiImportWorker ImportWorker removeHook(hook) addHook(hook) hooks:list 1 Author Hookable 1 WikiAuthor ...... 0..n 129

VI — THE COLLABREVIEW PROTOTYPE . s in Author ImportWorker AuthorManager component stores its object is usually cre- . The is an interface which transparently maps requests objects. A sub-tree of the full SubversionImportWorker component interface, which can be configured is responsible for retrieving all relevant Importer ConfigurationData ROTOTYPE AuthorManager P ConfigurationData ConfigurationData Importer A Currently, there is a The code artifacts) can be left out by defining a filter.are In operations Sub- that affect anfacts entire that directory, i.e. are contained all in arti- this group. The import worker plicability”). Almost all components takefiguration their information. own Some con- components cancustomized be by fully configuring a differentfor implementation an interface. ated by readinghas a a configuration name file andConfigurationData value at itself, start andconfiguration up. tree can is contain passed to It nested its a behavior. component to configure 6.1.2.3 software process artifacts fromrepository. the The project’s actual revision a work revision of repository-dependent implementation importing of isImportWorker the done by using implementation is started in regularrevision intervals repository to for poll new the artifacts and revisions. that crawls a Subversion repositorycess using information. configured ac- Unwanted artifacts (e.g. non-source must be implementedare by two a implementations class.HibernateAuthorManager of the Currently,the there server’s database using the Hibernate framework. to a MediaWiki databaseerations. but only They supportscomes work read directly as from op- expected thecreated MediaWiki but or database: use any a data changeimmediately user visible. made that in Write MediaWiki operationsnew becomes like user creating are a notintegrity from permitted/supported concurrent to modifications protect inand MediaWiki data CollabReview. 6.1.2.2 makes it flexible and adaptable to different kinds of soft- version, artifacts are grouped into directories, and there WikiAuthorManager The The CollabReview server is highly configurable. This ware process artifacts (see Requirement-3: “Flexible ap- EVIEW R im- object OLLAB C HE : Container for Author VI — T : Provides metrics for ImportWorker component : Forwards application life- s can be saved, and later be s) can be created by the maintains a list of all agents : A background worker thread : Sends emails to users and admin- by just specifying a new unique : Access to the data persistence . This gives the prototype the flexi- s can be marked as disabled which Author : Manages the Author entry must exist for data integrity reasons. Author AuthorManager for recurring tasks like pollingtory the for revision reposi- updates (not detailed further). layer where artifacts and reviews are stored. ReputationMetricManager all the reputation/karma metrics that are computed. WorkerDaemon istrators, e.g. if anerror artifact occurred, was respectively reviewed, (not or detailed if further). an MeasurementsManager measuring artifacts with regard to variouslike aspects their importance in relation to other artifacts. Repository bility to add to different kinds of artifacts. LifecycleManager cycle events (start up, shutdown) tocycle registered listeners life- (not detailed further). MailManager plementation that fetches software processfrom artifacts the revision repositoryRepository to put them into the Importer AuthorManager Author New users ( • • • • • • • name. Other propertiesetc.) of the can user be (email,itself. password, set After and that, changedlisted on or the loaded. Depending ondatabase whether the supports underlying transactions,committed explicitly. changes have to be ing However, prohibits them from interacting directly with CollabRe- the revision repository. 6.1.2.1 that are known to thetributed server. by modifying Known software agents process haverevision artifacts con- repository in or the have written oneor or they perform more only reviews, readany accesses. traces As of long contributions as left there in are the system, an accord- view any more; they can still indirectly interact through AuthorManager The 130

VI — THE COLLABREVIEW PROTOTYPE tuk,adatfcsi rnhit sub-directory a into branch a in artifacts and “trunk”, egtdb h eiw’rsetv ieiesas timeliness respective reviews’ the by weighted (cf. MediaWiki a of database the analyzes which “branches”. in name branch the with The AuthorshipArtifactResponsibility ArtifactResponsibility lityAssessor sibility 6.1.2.4 wiki GForge the (2012) for Agaraj worker plug-in. import operations. an such implemented support no to and does need branches worker import not no the artifacts, are of there manipulations mass As 2010). Dencheva, named directory a into trunk the in convention artifacts popular putting the of to according branch derive information to tag structure and directory the analyzes also It the in groups deletes Repository and maintains creates, therefore dtoeain aeacs hti rprinlt the to proportional is tokens. the that of cost length a have The operations character. sepa- edit newline specific the a instance by for character, separated rator are operations which for allows tokens, only full It on distance. edit means string by the string of another and string content artifact’s an milarity Similarity 5.2.4.4. artifact Section this in defined for reviews all of rating reviews. review its average at looking by the only artifact is an there Currently, of quality the 7.3.2.4. Section see derivatives with implemented comparison further a and algorithms authorship different ments The gives authors 5.2.3). of Section Sum- ratios artifact. responsibility an all for up has ming author an that responsibility of exchangeable. and configurable fully interface, are respective and their implement arti- metrics process All software facts. of aspects certain measuring for ieie hr sol h default the only is there Likewise, The ArtifactResponsibility a is there Additionally, MeasurementsManager A ArtifactQualityAssessor UTH MeasurementsManager erc tdtrie h iiaiybetween similarity the determines It metric. mlmnsthe implements o oedtie xlntoso the of explanations detailed more For . orflc asmnpltoso artifacts. of manipulations mass reflect to mlmnainfrthe for implementation mlmnain hc ae the takes which implementation, LatestContributorRespon- MediwikiImportWorker DefaultArtifactQua- C ONT implements otisbscmetrics basic contains esrsteratio the measures metric, component ArtifactSi- TokenEdit- determines E Editor- I1—Architecture — VI-1 DIT imple- 1 and , (see hc osdr riatsz n iwcut.See counts. view and size artifact considers which and artifacts, all for weight hr nytetuki ob nlzd h manager The analyzed. be to is trunk the only where ( uls-usrb pattern Publish-subscribe The The branch name, artifact’s an encapsulates identifier The Artifact version. tifier them. to subscribe that listeners to events certain lishes is It 6.1.2.5 different are there why weights. of artifact discussion a for 5.2.2 Section lines, characters, more of with artifacts for SizeArtifactWeight weight higher a implementa- assigns Exemplary are weight impor- higher tions more computations. a that various receive so the can artifact, artifacts process tant software a of etw h le pnalmtismngdb tunless it it. by override managed explicitly metrics they all upon filter the bestows project software a in branches exclude to exam- allows For it to scores. ple, filter the a from has artifacts It unwanted metrics. exclude karma and reputation of lection 6.1.2.6 revisions. two Numbers between successive. out be left to be need may of not revisions does different but for artifact increasing the be to revision has The number. number revision artifact’s the and name, Sub- like system control revision a reflect in better files to to changes able the be deleted. to be introduced can they were and Groupings altered, be can groupings in ings, artifacts process software the to deleting or renaming new new a a after when generated are events such example, For Review Review like manipulations mass apply to allow Groupings ArtifactWeight Repository ReputationMetricManager Hookable Repository orfrt eti otaepoesartifacts. process software certain to refer to s Repository nent ReputationMetricManager n otaepoesatfcs( artifacts process software and ) Artifact and s LineCountArtifactWeight saott eaddt h eoioy or repository, the to added be to about is UnityWeight hc en hti mlmnsa implements it that means which , Artifact riat a eptit group- into put be can Artifacts . stedt tr o reviews for store data the is a enaddt h repository. the to added been has component eemnsteimportance the determines The . hc onstenumber the counts which WikiArtifactWeight use s hc eun h same the returns which Repository ArtifactIden- anan col- a maintains Artifact w compo- ( which a ) pub- 131 in ).

VI — THE COLLABREVIEW PROTOTYPE s. Additionally, JSP s can be configured in the con- ROTOTYPE faces P like the user’s best or worstber rated of artifact, reviews, which the reviews num- have been rejected, quality of one’s own artifacts etc., browse the library of non-obsolete softwareartifacts process (see Figure 6.3), see a detailed informationincluding page authorship for and responsibility each information artifact, quality rating, submitted reviews,submit and own reviews, users and view can or reject other ones view their personal page which shows information (colors in the source window), its ancestors and (see Figure 6.4), what the timeliness of their reviews is, the average • • • if a reviewing widget iscle directly page. embedded The in the setbe arti- of extended available through functions servlets canExtensionServlet therefore and figuration files so that theyinstallations are of only the CollabReview available server. in some 6.2.1.1 CollabReview web portal interface that is available in all installations of CollabRe- user information (see Figure 6.2),all the the user basic has functions access of to CollabReview. Users can 6.2 Application and UserCollabReview Inter- provides a set ofapplications. interfaces They to integrate users into and variousto environments show user and reputationsubmission information, of facilitate reviews, and the allow genericlabReview. access to Col- 6.2.1 Web user interfaces of interfaces that allow alltions kinds through of web accesses browsers. to its Theface func- web allows portal user access inter- toprovides, all making functions it that independent from CollabReview thement. actual However, the environ- web portalfor may every not application be well context. suited Forhaving example, to instead log on of tocle a in different web the page wiki, to it rate is an better arti- from a usability perspective view. After logging on at its starting page with valid The CollabReview web portal is the default web user As a web-based application, CollabReview has a range EVIEW R . Repu- OLLAB C HE Visitor pattern which is the sum VI — T for an estimate of the which combines other . This hooking allows the for an agents’ amount of con- which computes the average hooks (see Section 6.1.2.5) following the s which are not all detailed here. The of the timelinesses of all of an agent’s reviews, and metrics into a new oneoperations using like basic mathematical addingSection or 5.2.5). multiplying (see also cost of technical debt ansource agent code has in brought Euro, into the MetricReviewAgedCount MetricMetaMetric quality of an agent’s contributions tocess software artifacts, pro- MetricQuantity tributions to artifacts, MetricTechnicalDebt MetricQuality • • • • At the moment there is a dozen different • When the server starts, the manager ensures that all concepts, and how a CollabReview serverthe integrates infrastructure into of a software project.anisms make Its it plug-in adaptable mech- toallow different for environments, future and extensibility (seeible Requirement-3: applicability”). “Flex- In the secondinner part, workings it of explained the the servering in on more a detail class while diagram build- of the most important classes. 6.1.3 Summary of architecture CollabReview prototype. First, it introduced overarching always available from the respective metric. tationMetric most important ones are because each metric cachesthe the agents. respective scores AfterReputationMetric for the initialization isinto complete, events each that can influence its scores followingmetric the to immediately update cachedrespective scores change whenever occurs. a Up-to-date karma scores are artifacts that are notReputationMetric filtered out, are visited by each This section gave an overview of the architecture of the Publish-subscribe pattern This visiting serves to initialize the karma scores, 132

VI — THE COLLABREVIEW PROTOTYPE iue63 rwigatfcsi olbeiwportal CollabReview in artifacts Browsing 6.3: Figure • rgeso h otaepoesatfc import the artifact repository. revision process the e.g. from software states, the internal of check progress can admins and iue62 olbeiwpra o npage in log portal CollabReview 6.2: Figure I2—ApiainadUe Interfaces User and Application — VI-2 igt r ml plctosadicmlt web incomplete and applications small are Widgets igt.Teidvda igt r ecie below. described are widgets individual The widgets. hc o o nnmu rnnaoyossubmis- non-anonymous a or comment, anonymous review for the box for field check text for a buttons article, radio the with rating widget page a MediaWiki is a 6.6a into Figure embedded MediaWiki. for widgets such screenshots of shows 6.6 Figure functionality. submission widgets Reviewing main embedded diverse the with shows wiki Moknowpedia 6.5 the Figure of example, page For wiki, GForge. Moknowpedia the or like software inte- existing it into made grate that CollabReview for developed widgets been Several have application. web existing an to extending analogous are Widgets browser. IDE user’s the web in surrounding their page to add They functionality new pages. or web extended other into embedded are that pages Widgets 6.2.1.2 igetprtdatfc ol aeatpqaiyscore, quality top a a have to would contributed artifact only rated who top user single A aspects. irregardless other score any certain quality of the average a list highest would to the rank with regard quality users with a example, users For of score. karma ranks the are ranks rewards and ranks domain Moknowpedia directly. listed or rejected be cannot representa- reviews numerical and have tions, artifact average the and for reviews rating of quality number The GForge. in bedded reason. rejection a enter to The field text user. a rejecting offers the the form of rejecting identity that the reveal message will warning review a contains Figure It in shown review; 6.6c. respective the for form jection non- submitted was review fifth anonymously. The comment. text without a and anonymously third submitted been The have rejected. review fourth been has review second the “Zur that that artifact. rating) the the (quality of to stars gave Left the review counts the 6.6b. that Figure number in a reviews is text of list the to leads light). (traffic review of own timeliness the ar- and (stars), panel), quality (accelerator artifact importance tifact average for scales and sion, lcigte“nnm iklast h eiwre- review the to leads link “anonym” the Clicking link, reviews) (all Bewertungen!” “Alle the Clicking iue66 hw ipie eiwn igtem- widget reviewing simplified a shows 6.6d Figure lgisfrwbbsdtosadaea aywyof way easy an are and tools web-based for plug-ins eiwn igt d review add widgets Reviewing cgwee”means uckgewiesen” ¨ Domain 133

VI — THE COLLABREVIEW PROTOTYPE ROTOTYPE P also interacts with userstionForge through system emails. is not The detailed Reputa- here any further. integrates into the GForge platformit in provides two simple ways: widgets Firstly, assubmitting presented reviews above, in e.g. the for GForge wiki.consists of But a secondly, it fat widgetits that surrounding does web not interact page or butits is augment own a that full runs web in applicationtab. its of ReputationForge own is tab an in extensive the socialmented GForge game in plug-ins imple- an interactive andtion. asynchronous web It applica- would not haveit been into feasible the to GForge web fullyframe interface; integrate inside instead, one it of has the itsFigure tabs own of 6.9 the is GForge a web screenshot interface. tionForge of running the in starting its page own of tab Reputa- in the GForge project view. Apart from its web interfaces, ReputationForge EVIEW R OLLAB aspirant C HE HotW VI — T Figure 6.4: Viewing artifact details in CollabReview server web portal Next to the photo of a user of the wiki, his level is pre- nism to motivate users to achieve higher karma scores. It sented (see Figure 6.8). Thehis level karma directly score. results from The levelsmechanism are to intended motivate as users a to rewarding Section increase 5.4.2.6). their score The (see ranking tableas and shown in Figure 6.5 serve the same purpose. 6.2.1.3 ReputationForge 6.7 shows some widgets that present a domain rank. The ReputationForge extension presented by Agaraj (2012) is a social game that acts as a rewarding mecha- whereas someone who contributedworse much rated more artifacts would but receive to a lower score. Figure 134

VI — THE COLLABREVIEW PROTOTYPE a onwei widget Moknowpedia (a) o digareview a adding for iue65 onwei anpg umne ihsvrlwidgets several with augmented page main Moknowpedia 6.5: Figure b onwei widget Moknowpedia (b) itn reviews listing I2—ApiainadUe Interfaces User and Application — VI-2 iue66 eiwn widgets Reviewing 6.6: Figure c onwei widget Moknowpedia (c) o eetn review a rejecting for d Frewde osbi re- a submit to widget GForge (d) view 135

VI — THE COLLABREVIEW PROTOTYPE (b) Who is writing the most positive reviews? ROTOTYPE P (d) Special feature: write mail to low scoring user EVIEW R OLLAB C HE VI — T Figure 6.7: Moknowpedia domain ranks and special features Figure 6.8: Moknowpedia avatars augmented with levels resulting from karma scores (a) Who has the highest quantity? (c) Who has the most up-to-date reviews? 136

VI — THE COLLABREVIEW PROTOTYPE //[CollabReviewServer]/accounting/ ihteotu ftekrasoe e evc for service web scores karma the of output the with h am crswbsriegenerates service web scores karma The inter- service web provides server CollabReview The xml?method=getOverallStats epnefrtasot hsi o hw here. shown not is This transport. for response ( a Protocol in Transfer wrapped Hypertext course, standard-compliant of is, response individual The Moknowpedia. the listing monitor a contains to 6.10 eval- Figure research during progresses. used developers’ this mostly of was service uations The instance. URL at accessible is It restrictions. access be could history afterward. development analyzed their developer field that so the on scores in data karma used collect was periodically service to web experiments The computed scores. the to karma access full application external grants respect with configured user every all of to scores karma the contains It overview scores Karma 6.2.2.1 plug-ins. like IDE applications or extensions other external with communicating for faces services Web 6.2.2 h e evc sara nysriewithout service only read a is service web The ReputationMetric iue69 anpg fRpttoFrewti Freproject GForge a within ReputationForge of page Main 6.9: Figure I2—ApiainadUe Interfaces User and Application — VI-2 neeyserver every in .I therefore It s. XML HTTP http: output. ) //[CollabReviewServer]/extension/ Kn ta. 97.Jvdcvoain eecounted were violations Javadoc 1997). al., et (King ssuigCektl ihteJv oeConventions Code Java the with Checkstyle using ysis ... ult eottext report Quality 6.2.2.2 o oedtisrfrt eto 7.1.6.7. Section to refer details more For members. public of case the with (marked twice anal- static The through automatically reason. generated the were and reasons repro- rating are quality artifact the the including for duced, reviews all that in After printed brackets. percentage con- responsibility “main her (called and person tributor”), responsible main revision. the current is and Next the name artifact’s artifact, the Per with starts artifact. short report a non-obsolete by each followed for for, responsible report are they files the of developer’s each score. their improve to from, how come on them scores hint their to where and them explain to the score, during used were reports Its ScrumLab only. read is and input a by ProjectReport generated is that report a configurable shows 6.11 Figure h eotcnit farnigtbeta presents that table ranking a of consists report The 6 suoy o tdn otaepoetue nteevaluation. the in used project software student a for pseudonym A 6 vlaint nomdvlpr ftheir of developers inform to evaluation ExtensionServlet quality h e evc osnttake not does service web The . ! ,o hie(akdwith (marked thrice or ), crs ..teaeaequality average the i.e. scores, tURL at !! http: in ) 137

VI — THE COLLABREVIEW PROTOTYPE text response with key, which is ei- HTTP response additionally a boolean string specify- status . The ﬁrst line of the re- load ” character. The content of the # anonymous text/plain loads a stored review from CollabReview. Fig- ROTOTYPE P All responses contain the Load contains the rating value, whetheras the review anonymous, is and marked the reviewcontains text. a An description error of response theanalysis error of and the a cause. stack trace for an integer number string withits the comment, review’s rating, and ing if the review should be submitted anonymously. ure 6.12 shows the responsesponse to is a wrapped load in request.content a The standard type response is the version numberments of the start web with service. a Com- response “ is encoded as key-value pairs. ther “ok” or “error”. A EVIEW "> "> R is "> "> . The ***** ***** *** *** OLLAB C Rating load HE . or plug-ins access this VI — T save IDE Figure 6.10: XML output of karma scores web service anonymous and ... ="quality">1.95.2 type="quantity 9.8 type="score 10.0 type="busyreview 7.9 type="bias" 9.0 type="agedcount 0.0 type="loneranger 18.8 type="silfwiki 0.9 type="averagereviewage >1.0 >moknowpedia encoding rating The command string is either The service expects several input parameters: a com- closely as possible. This specialas handling the is full necessary artifact nameplug-in might because not it be only known to knowsoper’s its the local ﬁle IDE hard name disc. on the devel- save command takesstrings the three additional parameter mand string (as explained below), thepassword, user’s and account and a targetartifact software can process either artifact. be the The fragment of full its name name. of In thetry the artifact to latter or case, guess the a an service will artifact which resembles the name as and to a CollabReview server. service to provide integration with the server. 6.2.2.3 Review web service The review web service reads and writes reviews from 138

VI — THE COLLABREVIEW PROTOTYPE ------+------+------+------+ +------+------+------+ +------+------+------+ ****** f b x i a | 7 a e | 6 | | 5 | Score | | 4 | | | 3 | | 2 | 1 | | | Developer | Rank | /Android/G /Android/G ie1:’’i o olwdb whitespace. by followed whitespace. not with is comment. ... preceded ’=’ Javadoc truncated] pattern not a files match is Missing and must ’=’ (!!): violations ’ic_launcher’ 16 [more Name 14: Line ’ˆ[A-Z][a-zA-Z0-9] pattern comment. Line match Javadoc 14: must a ’ˆ[A-Z][a-zA-Z0-9] ’drawable’ Missing 14: Line pattern comment. Name (!!): newline. match Javadoc 14 Line a must a (!): with ’attr’ Missing 13 Line end Name (!!): not comment. 13 Line does Javadoc (!): File comment. a 11 Line Javadoc Missing \ a (!!): Line Missing 11 30 (!!): Lines: 10 Line 0: Line statistics: file Line Source -7): (rated f Review contributor: Main eiw(ae 0:Suc iesaitc:Lns 5 \ 156 Lines: statistics: file Source 10): (rated b Review contributor: Main eeoe ult cr overview: score quality 08:00:01) Developer 2012-06-29 (generated: overview performance ======quality Project iltos 5(4x,5x,1 3 iltosprLn:3.2 Line: per Violations x3) 17 x2, 5 x1, (34 95 Violations: iltos 0x,0x,0x)Voain e ie 0.0 Line: per Violations x3) 0 x2, 0 x1, (0 0 Violations: ******************************** ******************************* ************* ************ *********** ********** ********* ******** ********* ********* ******** iue61:Qaiyrpr o aaproject Java a for report Quality 6.11: Figure I2—ApiainadUe Interfaces User and Application — VI-2 (90.4%) (59.1%) . | 9.3 | 9.5 | | 10.0 | | 10.0 | | | 10.0 | 10.0 | | 10.0 | | Wyon.aa(e.371) (rev. /Waypoint.java Rjv rv 115) (rev. /R.java * $’. * $’. 139

VI — THE COLLABREVIEW PROTOTYPE Mail- that gets invoked will receive the email. Con- 5% SetReviewHook award was part of Moknowpedia. To an- . The advantage of emails is that they can ad- ROTOTYPE P HotW The email in Figure 6.14 says that the user’s article Sending of mails is facilitated through the nounce the winner, an email was sent to the work group’s Manager Review notifications are sent wheneverdated a review or is when up- aprocess new artifact. one Only is contributors who submittedsponsibility are for threshold above of a a re- software tributors below the threshold aremade not a considered relevant to contribution have to thenot artifact, be and bothered should unnecessarily. Theplemented notifications with are a im- automatically when a review is added or updated. has been reviewed with acomment implies rating that of the reviewed “very articlescreenshot. good”. is an Review awesome notifications The were onlyknowpedia. used In in programming Mo- environments where source code was analyzed, review notifications wereunnecessary considered because a review based oncurred static analysis immediately oc- after a changeever was changed submitted. source Who- files received immediatebecause feedback summaries with the new karmasent scores to were all then developers. 6.2.4.2 Hero of the week 6.2.4 Email notifications on various occasions. For example,ted if a for review one is of submit- thean developer’s email artifacts, with he the can reviewnot rating receive and every instance comment. of However, thenotification server emails. has Instead, to notifications make havefigured use to separately of be depending all con- on the project context. scores. Emails push information torequiring the him user to instead pull of theuser information receives himself. too much But of if suchhim a information, and it make him can protect annoy himselfinformation from by such installing inadvertent a filterquently, for caution such must emails. be Conse- exercised whatto events emails. are tied A varietyimplemented and of used email in notifications CollabReview. have been 6.2.4.1 Review notifications vertise and attract users to CollabReview and the karma The The CollabReview server can send emails to developers EVIEW R . 7 OLLAB C HE VI — T plug-in has been developed so far. De- IDE Figure 6.13: Review plug-in for VisualStudio plug-ins are provided to reduce the perceived dis- http://www.microsoft.com/visualstudio/ 1.0 #CollabReview WebService #Tue response Jul status=ok 03 13:39:05 rating=7 CEST 2012 anonymous=true text=das bin ja ! ich 7 Only one Figure 6.12: Loading a review from the web service planned but discarded in favor staticopments analysis for and wikis. devel- Figure 6.13User presents Interface the (GUI) Graphical of this plug-in for VisualStudio forms an opinion of it.a review, When he he can then select wants “Submittext to Review” menu from submit and the be con- presentedupdate a his review review dialog (Prause to and submit Apelt, or 2008). 6.2.3 IDE plug-ins IDE tance between the developer’s usualment, working and the environ- reviewing platform.ers They to enable swiftly develop- change intothe the distance reviewer is role. crucial for Reducing increasing the chance that development of plug-ins for IntelliJ and Eclipse IDEs was velopers will submit reviews. Whenever a developer is working with a piece of source code, he automatically 140

VI — THE COLLABREVIEW PROTOTYPE J [some-group]@fit.fraunhofer.de To: m To: urMoknowpedianer ;-) Euer d nehmen! fleißig, und Beispiel Seid die staunen ein nur ruhig um k sich Da anderen gemacht. die verdient Gruppe Wissensaustausch unserer den in sehr und Moknowpedia Punkten zusammen, Hallo MoknowPedia From: f URL Diese PS: Gruß, Bester J Hallo MoknowPedia From: e aewscniee sabte a.Ised a Instead, way. better a as considered was consult page them having web of a instead them to information ing push- Moreover, Col- to policies. access organizational have for not labReview did developers AgileLab, the In report score quality Daily 6.2.4.3 should others that example. and his week, follow the during earned the shows he who 6.15 points states Figure It week. email. a such once list mailing internal *** urMoKnowPedianer Euer nnmmt_ergt bewertet! gut_ Bewertung: _sehr der Kommentar mit Optionaler " anonym Artikel dein wiki/index.php5/ http://moknowpedia/media Artikel: itdrgr der M bist Screenshot, Großartiger iue61:Atfc eiwdntfiainmail notification reviewed Artifact 6.14: Figure iue61:Hr fteWe announcement Week the of Hero 6.15: Figure a ihdeeWcemt6,1 mit Woche diese sich hat *********** ****** , oßte!. ¨ htdc ietzum direkt dich uhrt ¨ ******* @fit.fraunhofer.de *** HotW ud gerade wurde " I2—ApiainadUe Interfaces User and Application — VI-2 sadhwmany how and is *** Du . urfen ¨ onnen ¨ iia alwssn otegnrlmiigls fthe of list mailing general the to sent was mail similar A 6.16. Figure in seen be can email reprinted A aao euainrnigtable: ranking reputation Javadoc [some-list]@ To: ie htyuhv otiue to. contributed have you you in that classes/ by classes/methods/variables files for all Javadoc committed but only not the methods/variables score on reputation depends your Note, reputation your writing improve consider to scores order In about you ranks informs reputation Javadoc mail current daily the this the of participant Dear server CollabReview From: oti uota twudcs ofi h oigstyle coding the fix to cost would it that the Euro expresses each in it that cost i.e. debt project, the technical into of brought amount developer the of contained estimate It an scores. their of awareness increase developers’ to the was intention The TornBond. of developers ranks debt Technical 6.2.4.4 scores. current their the of in overview list an mailing them general giving their evening, to sent was email daily * * * * .V 9. P 8. U 7. K 6. K 5. S 4. L 3. S 2. S 1. prm rtr,ec asfor tags etc. @return, and @param, descriptions, Variable (public descriptions Method (important) descriptions Class iue61:Diyealwt ult ranks quality with email Daily 6.16: Figure methods. than ones) important private more methods ************** **************** ************** ************* ************** *********** ******* ********** ************ 50 points) (5.00 53 points) (5.37 49 points) (4.96 ***** 53 points) (5.39 44 points) (4.41 42 points) (4.26 points) (4.83 30 points) (3.00 38 points) (3.80 .uni-bonn.de *** lab, 141

VI — THE COLLABREVIEW PROTOTYPE of Java . 8 KLOC ROTOTYPE P http://sourceforge.net/projects/ 8 Its modular architecture and far-reaching configura- The heart of the CollabReview prototype is the Col- collabreview/ of about 400 classes withcode. a The total prototype of is 32 availablethe as Apache open 2.0 source license on under SourceForge.net tion capabilities gave it the requiredit flexibility to to different adapt test environments.date The that next the chapters concept vali- andfunction implementation are as sound required, and andlabReview evaluate for the software benefit projectsexperiments. of based Col- on a series of 6.3 Summary of the prototype of CollabReview, its development process, andceptual the and con- technical requirements. Theconcept CollabReview is a socio-technical systemfrom that software requires tools. support Therefore, thisthe chapter above completes concept by presentingtotype. the CollabReview It pro- is the softwareparts implementation of of the the technical system, andality. provides In all its necessary final function- form,tion it that is evolved the alongside result the ofduring concepts an their to implementa- several support iterations. them labReview server. It extracts software processfrom artifacts their repositories, estimates artifacton quality reviews based and quality metrics, assignsto responsibilities developers, computes karmaous scores, kinds provides of vari- userinterfaces interfaces, through and web-services. exposes It application currently consists The previous chapter elaborated the theoretical concepts EVIEW R OLLAB C HE VI — T plug-ins were necessary. Latter ones made type interfaces IDE Originally, the CollabReview server only provided a An email would be sent whenever there was no com- functionality, and (b) increase theof the reach reputation and system. presence Thereforeand email notifications collecting data. as web pages and widgets. Additionally, the prototype closer integration with the user’s workingnecessary contexts to was (a) increase the usability of the provided applications and for human users.plug-in infrastructure Besides that its allows flexible componentsserted to be directly in- into the server, interfaces are also pro- new revision of the sourcetory. code However, to these emails the were revision not reposi- the well-received developers by (see also Section 7.8). 6.2.5 Summary of CollabReview proto- problems made by the respective6.17). developer (see Figure mit for 15 minutes after the developer had committed a vided through network access, either as web services or The CollabReview server provides interfaces for other web services necessary, which also proved useful for web front-end for users for major interactions. However, 142 “interfaces” with users by sending emails.

VI — THE COLLABREVIEW PROTOTYPE .(ie4,ciiaiy rbe) xetd@aa a o ’parts’. for tag @param Expected problem): criticality: 40, (line 4. .(ie2,ciiaiy rbe) ae’DNIYMR utmatch must ’IDENTITY_MGR’ Name problem): criticality: 26, (line 1. Reputations-Rangliste: (Ver Javadoc 437200EUR insgesamt: Schuld Technische .(ie4,ciiaiy rbe) ty sntfloe ywhitespace. by followed not is ’try’ problem): criticality: 43, (line 5. .(ie4,ciiaiy rbe) aaee at hudb final. be should parts Parameter problem): criticality: 40, (line 3. .(ie4,ciiaiy rbe) ehd’createHIDForAttributes’ Method problem): criticality: 40, (line 2. Die euch dieser informiert nach Mail Konformit Diese auf anschließend eine Entwickler, Liebe ae:/ 0 Checkstyle Erkl durch: Bewertet Bewertung: Datei: a 11. P 10. .t 9. p 8. h 7. f 6. s 5. l 4. h 3. a 2. p 1. ore try{ Source: create HID public Source: create HID public Source: ore ulcHDcreate HID public Source: = IDENTITY_MGR String static private Source: Uberpr ¨ neugwrei a oreog repository SourceForge das in wurde Anderung ¨ rn:Ihv oepolm ihti file this with problems some have I arung: ¨ ******* ******* ******* ******* ******* ******* ******* ******* ******* ******* ******* fn ie e aee im Dateien der einer ufung ¨ Anderung. ¨ 87 ke eh cud 10ER(Ver +1200EUR (Ver Schuld: +400EUR Tech. Schuld: Pkte, (Ver Tech. -8.73 +6800EUR Pkte, : (Ver Schuld: -7.31 +418400EUR Tech. : (Ver Schuld: Pkte, +3000EUR Tech. -6.64 (Ver Schuld: Pkte, : +3000EUR Tech. -6.40 (Ver Schuld: Pkte, : +1500EUR Tech. -6.11 (Ver Schuld: Pkte, : +500EUR Tech. -6.05 Schuld: Pkte, : (Ver Tech. -6.05 +1500EUR Pkte, : (Ver Schuld: -5.04 +800EUR Tech. : (Ver Schuld: Pkte, +100EUR Tech. -4.82 Schuld: Pkte, : Tech. -4.35 Pkte, : -3.28 : h otciia nsare: warning(s)). ones 1 critical problem(s), most 26 The show-stopper(s), (0 *** / * CryptoImpl.java iue61:Tcncldb ak mail ranks debt Technical 6.17: Figure tz e aaCdn Conventions Coding Java den zu at ¨ I3—Smayo h prototype the of Summary — VI-3 brer kuleCode-Qualit aktuelle eure uber ¨ *** *** *** Pr[ at){ parts) (Part[] { parts) (Part[] Pr[ at){ parts) (Part[] sntdsge o extension. for designed not is ’ˆ[a-z][a-zA-Z0-9] pattern neugst ra (beispielhaft): ergab Anderungssatz ¨ neug -500EUR). anderung: ¨ *** brrgnund ubertragen ¨ .class neug ---) anderung: ¨ ---) anderung: ¨ ---) anderung: ¨ ---) anderung: ¨ neug ---) anderung: ¨ -200EUR) anderung: ¨ ---) anderung: ¨ ---) anderung: ¨ -100EUR) anderung: ¨ -300EUR) anderung: ¨ neug ---) anderung: ¨ ats-Reputation ¨ uberpr ¨ * $’. uft. ¨ 143

VI — THE COLLABREVIEW PROTOTYPE ROTOTYPE P EVIEW R OLLAB C HE VI — T 144

VI — THE COLLABREVIEW PROTOTYPE T a iainadEvaluation and Validation aiaindtrie hte rntterequirements the not or whether determines Validation auto n interpretation. and valuation processes. different being still while linked, much very hpe 7 Chapter hrceitc fClaRve oalwdifferentiated allow to CollabReview of key characteristics about information quantitative surveys Evaluation Evaluation – 59 Definition evaluation. of process selec- the for is data comparison, this tive preparing and criteria, various count requirement). of type respective the to proto- applicable or as concept type, (either CollabReview by fulfilled are Validation – 58 Definition met: are requirements process whether rigorous checking the of is validation described (2011), as Marwedel design by system in definitions the Following are and other each to contribute Valida- can concept. evaluation of and the image tion of realistic disadvantages a and create benefits to the also is goal the The development evaluates process. software and the for 5.1.2, CollabReview of Section benefits in defined were that uvyn aaaotadsg hl aigit ac- into taking while design a about data Surveying HIS etadpooyeaanttesto requirements of set the con- against CollabReview prototype the and validates cept It work. this of hpe at oase h eerhquestion research the answer to wants chapter dadV Berard V. Edward — akn nwtraddvlpn otaefo pcfiainare specification a from software developing and water on Walking ayi ohaefrozen. are both if easy eecnutda ato h aiain n r summa- are and validation, the of part as conducted were 7.1.6). Section (see place took studies various the which h olo h vlaini oaaye siae discuss estimate, analyze, to is evaluation the of goal The that studies the describe each sections following The ainbt otiuet ahohri h evaluation the in other each to contribute both uation aet eadesdi eaaestudies. separate in addressed the be to of have architecture however, requirements, the Other at prototype. looking CollabReview ones by some validated example, be For can obvious. is answer the cause of itself. evaluation by an characteristics no require relevant or may yes answer a this basically giving is answer, requirement a of While validation 7.14). the Section a (see through results concept validation the of evaluate discussion then from to requirements and of 5.1, set Section the against con- CollabReview prototype the and validate cept first gen- to The is is. procedure CollabReview eral of value the what show and Procedure 7.1 of evaluation concluding CollabReview. the to valida- contributes the result Subsequently, tion 7.13. that Section requirements in the reported of are all fulfill prototype and CollabReview the concept collected whether data validates The studies 7.12. these Section from in overview an in rized in environments the evaluation of the description details a 7.1 including Section procedure large. at concept a of osqety h rcse fvldto n eval- and validation of processes the Consequently, h aiaino oerqieet ssml be- simple is requirements some of validation The 145

VII — VALIDATION AND EVALUATION are an analytical evaluation method VALUATION E The advantages of concept evaluations are two-fold. Expert feedback (including developers that have ac- Concept evaluations provide two kinds of feedback: Other methods like expert interviews withof actual the users prototype are possible, too. shows if experts deem ancan approach valuable, imagine and using if it they also themselves. evaluate Second, the the CollabReview prototype, experts aspects and like partial the rewarding system, or its architecture. First, they provide feedback earlyprocess before in too the much effort development is spentdirection. in an unpromising Second, theyenvironments where give experiments cost-effective are access not (yet)or to possible advisable, because e.g. theyfunctional are prototypes not easily are accessible, notrisks available, of or the to effects mitigate thatthe a environment. field While experiment expert can opinions havecolored, are on they subjectively exploit the experiences ofgained experts through that many each years of practice. tually worked with CollabReview) complements theformation in- necessary for assessment ofcourse, the analytic concept. evaluation Of cannot replace empiricaluation. eval- 7.1.1.2 Experiment developed so far and partialby aspects of users it and were other reviewed are stakeholders. done by Usually, experts evaluations inbased the on field descriptions, who prototypes judge or thein other concepts a presentations controlled environment. Expertorganized workshops as can personal be one-on-one interviews,cus or groups as where fo- aa small group group discussion about of the expertsDaalhuizen, concept engage 2010). (van in Boeijen and Definition 63 – Concept evaluation to them in a suitable way. First, experts assess theCollabReview concept. practical It reveals significance flaws of in the the concepts, The main empirical evaluation method is the experiment. This can help to better realize the underlying concepts. Concept evaluations where experts judge concepts that have been presented em- and ALIDATION AND analytic VII — V means evaluation methods that means evaluation methods that trial is a detailed scientific investigation and analysis evaluation in usability engineering. Analytic Analytical evaluation methods include architecture Several studies were conducted for this research. The study 7.1.1.1 Concept Evaluation Concept evaluations are an analytical evaluation method. cluding very early phases of development. The concept Definition 62 – Empirical evaluation the CollabReview prototype (or parts thereof) in a real examine features and characteristics oftheoretically, i.e. CollabReview without actually using the prototype. analysis, logical deduction and expertexperts interviews who with have not used the prototype themselves. to this research, both evaluation methodsfollows: are defined as Definition 61 – Analytical evaluation pirical evaluation is somewhat analogous to theof static software, analysis while empirical evaluation isanalysis like (see dynamic Section 3.3.1): Thetures first and one characteristics examines of fea- theits subject conclusions, of while study the to second draw subject one to actually gather trials data the under real conditions. With regard sented in Section 7.12. Itpage revolves 209, around which Table 7.7 summarizes on the most important facts. 7.1.1 Evaluation methods Rosson and Carroll (2002) distinguish ones presented first are moreaspects, concerned with e.g. technical validity ofployed CollabReview karma in scores. natural environments Later tosocial study ones effects. de- An overview of conducted studies is pre- Definition 60 – Study of a situation (Oxford Dictionary, 1994). They took place throughout the research process, also in- Empirical evaluation world setting (potentially with real users) to gather data. A Analytical evaluation 146

VII — VALIDATION AND EVALUATION td sitral ai fisrsl sunambiguously is result its if valid internally is study A Ginl ta. 2011). al., et (Grinnell CollabReview the through represented thereof) parts (or Experiment aiiyo tde ecie o uhtutteei in is there trust much how describes studies of validity lohv oeohroii Ginl ta. 2011). al., et (Grinnell origin could other dependence some observed have the also that doubt a the of without shadow variables both have between variables relationship The causal dependent the well. as that changed show then to able additionally users. may as purely subjects it human be or include may data, real-world that with setting technical empirical an in prototype Experiment – 64 Definition n la niomn.Teol as o hnein change a for cause only The environment. clean a ing the in change a for potheses 2005). (Bortz, variables to confounding due explanations uncontrolled alternative plausible of increas- number an ing with declines validity Internal interpretable. validity Internal – 65 Definition de- between research distinguishes a sign of even Evaluation can results. They research certainty. Uncon- invalidate that to threats variable. are dependent factors trolled the and between independent relationship the the about certainty experi- in the ideal results of An ment result investigate. the to actually ought are they study phenomenon the of results the that experiments of validity external and Internal experiment. the of improve validity to external validity field the internal of Often bit a behavior. sacrifice human experiments which studying context for natural essential a of is advantage big the experi- have field ments experimenter, the the of limit control experiments experimental laboratory field between While line experiments. sharp field no and is They there in. often operate that subjects note the environment sub- the human or of jects, pool the like dimensions conditions several as “natural” regarding experiment under place field taking a experiment define an (2004) List and Harrison nenlvalidity Internal naturalness. of degrees varying have can Experiments vari- independent the alters artificially experiment An stetsigo h olbeiwconcept CollabReview the of testing the is satie yrln u n ia hy- rival any out ruling by attained is da experiment ideal internal eedn variable dependent and external salse a establishes I- Procedure — VII-1 ycreat- by validity The td setral ai fisrsl a egeneralized be can result its if valid externally is study A aibe ih olne ersn elt,o te nat- other or reality, represent longer no might variables aibe napeet h xeietlgopreceives group experimental the pretest, a in variables (Grinnell situations other be to to likely generalizable most and therefore valid are results Their variable. ru eerhdsgso experiments of designs research Group 2005). surveyed (Bortz, the samples of decreas- representativeness with and naturalness declines ing validity External situation subjects. research the and of conditions specific the beyond validity External – 66 Definition al., et (Grinnell situation to 2011). research generalize the longer no outside then settings would results experiment The the forgotten. of been have might influences ural natural of substitutes Controlled natural. less is it that means variables environment the controlling However, the otaporaemto obidkoldeaota 2011). al., about et knowledge (Manuel build topic to method appropriate most quasi- dard the below by lim- considered provided is other design Evidence or experimental ethical 2011). practical, (Plante, to itations experi- due and groups control mental into randomly subjects distribute 2011). due al., not et and (Grinnell hypotheses variable rival independent to the due to be control to the likely for is measurements group the in group manifests experimental also the that for difference hypothe- measured rival a for control ses: to The helps design again. group measured control are the variables At dependent not. the does end, group control the while intervention, the dependent the measuring After to respect variables. with important equivalent all es- are thereby that other groups the two or tablishing group one to randomly assigned mental 2011). al., et dependent and independent a between establish connection to the causal aim of they level because highest continuum the knowledge most on results experiment create ideal They an closely. resemble designs research nqaieprmna eerhi sntpsil to possible not is it research quasi-experimental In an are there design, experimental classical In eedn variable dependent ftu xeietldsg u hyaeotnthe often are they but design experimental true of n a and control ru fsbet.Tesbet are subjects The subjects. of group a eteidpnetvariable. independent the be may odstan- gold experi- Group 147

VII — VALIDATION AND EVALUATION . It 05 . = 0 significance p weak, moderate 4 As a final remark, . , the stronger the rela- Statistical = 0 r because of the immediate . r r 5% ≤ constitutes a strong or weak re- Strength No association Trivial Low or moderate Moderate to substantial Substantial to very strong Very strong Near perfect r indicate “moderate to substantial” rela- 5 . 0 Coefficient 0.00 0.01-0.09 0.10-0.29 0.30-0.49 0.50-0.69 0.70-0.89 0.90+ p < VALUATION E ≤ Significance is undermined by low correlation Furthermore, differences between correlation coeffi- The greater the coefficient 3 . correlations are preferable to measurements of precision to be very strongly relatedtherefore (de follows Vaus, 2002). the This nomenclature work insocial Table and 7.1 human when factors are0 involved. Correlations of tionships in such situations . Significance of correlations means that one can betion confident is that a not detected found correla- by chance but is actuallymeans there. the statistical This chance forthough finding it a is correlation not al- there is strengths and small sample sizes.found If to a correlation be is significant not duethat to does small not sample mean size, that then the correlation doescannot not be exist confident in the result and that probably a test cients are not easy totests interpret for statistically. the Significance difference have tofidence be in done results. to establish con- Relation to precision and recall strengths (de Vaus, 2002) tionship. But what lationship? For example, is or strong? Interpreting therelative, strength depending of on a the correlationtypically is context. find high Physical values for sciences influence of simple natural laws.have However, more if than outcomes a single cause, no two variables are likely Table 7.1: Nomenclature of social science correlation work sets the significance threshold at with exactly that same strength. It only means that one with a larger sample size is needed. . x rank mean . How- r Pearson } meaning n ALIDATION AND 0 denotes an for some 0 y The (Grinnell et al., denote perfect, , ..., y Spearman 2 | → r r < , y | VII — V 1 = 1 y { | r | = Y is a numerical expression of design. Whereas the compari- +1 and } ≤ of a correlation is usually its squared n r . design (Plante, 2011). ≤ s r , ..., x 1 2 strength − within-group , x 1 . x 2 one-group pretest-posttest design { r = Another important correlation is the Pretest and posttest results of the experimental group Strength of a correlation maximum strength relations, with that a relation is weakerfound. to The non-existent, or has not been correlation. This correlation isthe computed values by of both bringing variablesdiscarding into exact rank values order and only first, keeping thereby theirSpearman order. correlation The can be used tobut test non-linear for relationships. monotonous Its correlationgiven coefficient here is as lation. It finds a linear relationship betweenear function the that two can predictUnless the noted value otherwise, of correlation coefficients a Pearson correlation. coefficient the strength of theinverse relation, relation. where Pearson and Spearmancorrelation correlations is the most commonly encountered corre- X ever, it may not be confusedthe with variables causation, are but only somehow that associated. The correlation 7.1.2 Statistical tools discussed here shortly. 7.1.2.1 Correlation called 2011), or son between control and experimentalbetween-group groups is called from a classical experiment canout also data be from compared the with- controlthen group. be The control left group out. can Such an experimental design is then value variables, and allows regression analysis to derive a lin- ysis of the data collected in the experiments. They are There are two important statistical tools used in the anal- A correlation describes a relationship between variables 148

VII — VALIDATION AND EVALUATION ..nnoslt otaepoesatfcs—into — artifacts process software non-obsolete i.e. — ihpednm o rvc reasons. privacy for pseudonyms with Cross-validations h ae fsfwr rjcs(xetfrsm public some for (except projects software of names The ovd eedn ntecnet ifrn em are terms different context, the on Depending volved. recombines merely but data new collect not does view for value ftedt,rsrigters o validation. for subset rest a the with reserving trained data, be the can of it system, generalizability prediction show a the still train of to to order used In was system. data available prediction all is when This case to hypotheses. the impossible those often is testing it for and data data, further of collect set original an from ated Cross-validation k-fold 7.1.2.2 discussion a for this. 3.4.5.1 on Section expres- to higher back its Refer of siveness. because possible where recall and sd hsscincaie h sdterminology. used the clarifies section This used. in- “people” different of number large a was is CollabReview there which trialed, in studies diverse the to Due people of Terminology 7.1.4 critical. as considered not therefore gathered is CollabReview data The by wikis. in involved users that experiments by voluntarily the provided are are that statement reviews previous manual the to exception An data. existing CollabRe- anyways. devel- available a have of would member team other ac- opment any have not not that does information it to artifacts, cess to analysis static when applies Also, it information. software authorship or e.g. artifacts process repositories, revision from derived data exchanged been have or well, as experiments out or left surveys been of either part have were that subjects all names the of Additionally, out. left of been names have and organizations changed, no been where have but place) studied took only experiment were which ones, source open Ethics 7.1.3 After data. other as all once used while then set, is test partitions these of Each partitions. h aaue yClaRve osssmsl of mostly consists CollabReview by used data The A -odcross-validation k-fold k is k k onsterslsaeaeae.Agood A averaged. are results the rounds 10 = r sdwe yohsswr gener- were hypotheses when used are MLclne l,2004). al., et (McLachlan k − iie h hl e fdata of set whole the divides 1 esaeue straining as used are sets I- Procedure — VII-1 k A A A seDfiiin8– 8 Definition (see hyueteue nefcs—adfrti eev input receive this for and — interfaces user the use They iaigwt h olbeiwprototype. are commu- CollabReview Users indirectly the or with directly nicating interfaces. either user are they other that aware or web on emails, output in its read pages, also users And user software artifacts. its to process using contributing by or it reviews, to submit input interfaces, provide Users users. its are CollabReview. in est Stakeholder – 67 Definition on. so and interest, scientific software the of affects because it product, because to change, want make they to because it it, CollabReview,employ by affected in is interest work their of because form some have holders t u a xedt eae eiwn rdocumentation or reviewing activities. related to extend activ- can prominent but most ity their is Program- software, writing project. software i.e. ming, the in developed software being the is to that directly contributing are They back- engineering ground. software or science computer strong only. purpose this for receives output necessary its if who and prototype, CollabReview Contributor – 69 Definition only purpose. — this necessary for if prototype CollabReview the from reviews. submit or artifacts process con- Contributors software it. to to tribute input send CollabReview only the they from output prototype; receive to need not do tors output, its receives both. or or prototype CollabReview the to put User – 68 Definition stakeholder user contributor epewoitrc ihteClaRve prototype CollabReview the with interact who People Stake- stakeholders. are people of group broadest The otiuosaefrhrseilzdit developers into specialized further are Contributors Contribu- user. of form specialized a is contributor A sasaeodrta osiul ihrsnsin- sends either consciously that stakeholder a is saue htpiaiysnsiptt the to input sends primarily that user a is saypro hthssm omo inter- of form some has that person any is “Developer” .Dvlpr aea have Developers ). 149

VII — VALIDATION AND EVALUATION developers more because cowboy VALUATION E Apart from that, favorable projects for CollabReview The second kind are users who have to work with Col- Still, it is important to better describe developers and the interest on technical debts.team should The probably atmosphere be in good the labReview and is friendly, not so that perceived as Col- tool a for threat bullying. or In misused summary, as CollabReview is a intended their social context that givessoftware distinction project to is them. not One likeferences. another. There CollabReview is are intended huge for dif- software projects, they use their time. Inries a developers project, from where the management implementation hur- ofto one the feature next, thereManagement may support be is a no prerequisite. room for CollabReview. are projects which are notmal too but strictly self-organizing organized because or CollabReview for- addresses the motivation of an individualcide that how has to freedom work. to Strict de- for quality CollabReview control as leaves a few light-weight room method.labReview Also, aims Col- to befew resources cost-effective for in quality improvement. projectsformalities It with and relies on adds few few overhead,for making agile it development projects. suitable Possibly, CollabReview helps inexperienced or they have not yet learnedtion to for value its internal long-term documenta- benefit. Inonly general, a developers short-term with perspective onattention the to software their need motivation more to carethan for developers internal who quality have acause long-term perspective, the be- latter ones know that they will have to pay managers of the software developing organization who development and maintenance costs, quality assurance personnel who are looking for a tool totheir pain support of their having to work on unmaintainable software. the following second one. labReview and who are affected incause their it daily attempts work to be- influencealso their quality work practice. assurance While personnelmanagers and can sometimes be even direct usersgeted of users CollabReview, the are tar- first and foremost the developers. The geted users in order to show that CollabReview actually employ CollabReview to further their goals,marginally important this for is the only evaluation of the concept. validation of CollabReview focuses on developers as tar- This first kind of users is not necessarily distinct from where developers have a high degree of freedom of how want to improve internal quality to reduce long-term work, or the developers themselves who want to reduce works. While it is interesting to see how other users ALIDATION AND VII — V . Subjects are not inevitably subjects is a person that is primarily characterized is a person that is primarily characterized by is a person that is participating in a scientific employee First, it is all the stakeholders who have an interest in Similarly students are persons, who instead of an em- Employees are special kinds of persons, who are often The persons participating in an experiment are tra- student subject ity in software projects. ThelabReview targeted is user twofold: group of Col- the internal quality of software.be These customers stakeholders who can realize thatthe internal external quality influences quality of the product they are acquiring, and not by the relation with the CollabReview prototype. 7.1.5 Targeted user group CollabReview is designed to improve the internal qual- ployment relationship, have an educational relationship Definition 72 – Student an educational relationship within the evaluation context, by an employment relationshipcontext, within and the not evaluation byprototype. the relation to the CollabReview if not their interaction withis the primarily CollabReview characterizing prototype their role butrelationship the within employment their work context. Definition 71 – Employee study. stakeholders, users or even contributors of a CollabRe- users but the control mayinteraction with lack the the CollabReview knowledge prototype. ofin Therefore, their the context ofsubjects. experimental design, they are called Definition 70 – Subject ditionally called users of the CollabReview prototype.between-group For experiment, example, the in experimental a group are view prototype. However, persons are called employees with their context. A An A 150

VII — VALIDATION AND EVALUATION hc nrdcdKrksfo h rbe analysis problem the from Karakos introduced which Col- of evaluation and validation the for conducted were cdmc o o eiu environment. serious too an not in academic, interests, short-term partly with with personnel structured, inexperienced less and that control, agile self-organizing, quality and are pressure few with projects for aRve rttp a o e ed o srtesting user for ready yet not Col- was for the prototype However, predestined labReview environments. naturally study is the in it inclusion such, As 2.3.4, perspective). Section to back Col- the refer of (also development concept the labReview of motivation re- the with to project important gard most the perhaps is Karakos Karakos 7.1.6.1 is them to here. introduction given and of overview An labReview. that studies the hosted environments different Several Environments Study 7.1.6 archetypes. these selected from projects in evaluated be should CollabReview • characterization: this fit projects of kinds Three • • n h ako irrh ae ult manage- quality makes hierarchy of lack the And EU ntedvlpet,adasrn ult oto is control quality available. strong not a and developments, interest the medium-term in to short- em- have most only work, ployees other most sim- as be self-organizing can ilarly institution research a at projects Internal develop. they software the short-term in a are interest only students with The developers results. inexperienced software the academic in mostly a interest and re- control, few quality has for department sources the Still, value add department. the that for results deliver should also univer- but at sities activities teaching are labs Programming difficult. ment partners. between distance physical the to due difficult are collaboration whole and the Communication of quality project. the harm thereby internal and neglect quality to might time partners few Some have do- that harmonize. partners the diverse of the and uncertainty main the of because organizing rjcshv mauepoessadaeself- are and processes immature have projects I- Procedure — VII-1 eepbihda pnsource. open as published were and 7.2) Section (see algorithm the of evaluation were exe- therefore were prototype CollabReview the with hnJv eeomn elcdmr n oeo its of more and more replaced development Java when hsmasta omt r n-rnlradfwwork few and fine-granular are commits that means This 00rvsos hsteaeaecmi ds2 SLOC. 21 adds commit average the Thus revisions. 7000 114 commits. year aacd hteovdoe eea er.Teeare There years. several over evolved that code Java hleg o h sdagrtm u load othe to a adds also poses but it algorithms as used the intentionally for chosen challenge Subversion was project’s It the of repository. use sub-optimal is ten about reason is This 214 the commit. of times each average with contained an added repository Hence, are SLOC the revisions. study, 1600 of than time more the At 350,000 code. encompasses It months project. 30 the after into repository development Karakos the in sources project, the of end the various At from countries. partners European scientific and developers commercial 70 from about involved project The are: evaluation 7.3). Section (see consistency its experiments of The benchmarking base. code Karakos the with cuted experiments technical Only ended. project the before srte o at low rather is specialization Developer commits. two between happens ≈ year-over- increasing with active develop- and the large is and team well-established, ment and mature is Source- Ohloh base at month 2007. the February of in project Forge was It code. C original in shown is screenshot a game, 7.1. illustration the Figure For of maturity continent. the new Source- of a at colonizing project is player game source open Forge.net an is FreeCol FreeCol 7.1.6.2 use repository. better Subversion made its developers of where project conducted FreeCol were the experiments in same the Karakos, for part at high very is (see 5.2.3.5) Section specialization Developer experiments. of realism h vlainrslsaebsdo h orecorpus source the on based are results evaluation The the for relevant are that Karakos of facts key The 7dvlpr aecnrbtdt h aesne2002 since game the to contributed have developers 57 2 1 600 https://www.ohloh.net/ http://sourceforge.net/projects/freecol/ aafie.Terpstr otismr than more contains repository The files. Java 1 SLOC eebigasrtg aecascweethe where classic game strategy a resembling i ds e omtta hto reo.The FreeCol. of that than commit per 0 = KLOC . 47 . fisttl166 total its of i ds 2 0 = eot httecode the that reports . 97 SLOC sacounter- a As . KLOC fJava of 151 are

VII — VALIDATION AND EVALUATION of C# code). De- KLOC decided that they needed a 3 ) LARI of Java code (plus 40 VALUATION E it is the living successor of Karakos, many developers should have a self-interesting in keep- technical debt low, management has a long-term interest in the project, its development is organized inbeing an open agile source way, while at the same time, it has a medium size andan number active of development developers, team, and and because it enables easy access tobetter the observe development effects team of to interventions. pseudonym used for anonymization 3 The source code of TornBond was selected as an ex- • • Most of the developers belong to a single unit of a • • • • umentation and knowledge management.ics Typical of top- articles are news, projects and persons, technical technical debt inherited from themajor previous topic in project discussions is between a developers. perimentation environment (see Section 7.8) because 7.1.6.4 Moknowpedia Research Institute ( new form of managingdedicated their discussion, project it knowledge. was decidedknowpedia In to based a set on up the the MediaWiki Mo- software for doc- 7.1.6.3 TornBond bug fixing and refactoring of the old sources.KLOC About platform Sourceforge.net. The repository contains about 2012). research organization, where developmentas is an handled internal agile project.technology used TornBond in is other a projects future includingcommercial scientific base and ones, and hence isof in the the organization. long-term Some interest ofresearchers the who developers are plan young to buildthe their software. own Consequently, researches they also on interest have a in long-term it. However, at the moment the overwhelming velopment is taking place on the open source hosting TornBond is the open source release of parts of the soft- A few years ago, two work groups at Large Anonymous ware developed in Karakos. Current work mostly covers 10 developers are contributing to a code base of 150 1200 Java files and 400 revisions (Prause and Eisenhauer, ) FLOSS ALIDATION AND VII — V it is a Free/Libre Openproject, Source Software and ( consequently isble inherently to compara- an agile developmentGabriel, project 2005). (Goldman and a good mixture ofdevelopers with not many only commits, or a manyfrom single commits only a one few or developers, two it has a reasonably sizedtoo code small base nor that too is large, neither and the entire development history beginning fromfirst the revision is available, which reducesskewed results the due risk to of missing information, a medium number of developers are involved, i.e. Figure 7.1: Screenshot from the FreeCol game In addition to FreeCol, also the source code of the • • • FreeCol was selected as the code base for the evalua- • open source project Log4j is usedLog4j in is one used study. in However, thatand case has among not several been other chosen projects, forbut any particular just characteristics to increase thesource size code. of the It corpushere. is of therefore available not further characterized tion of the authorship algorithman (see environment Section for 7.2) testing and the as computations consistency (see of Section reputation 7.3) because 152

VII — VALIDATION AND EVALUATION Yusol rt natceaotti! a fe heard often was this!” about article an write should “You ihvlal otiuin.Freape rtn high writing example, For contributions. valuable with well- and central a through pro-actively provided was for requests explicit or conversations hallway like ways iwfeunyi nlddt comdt h atthat fact the accommodate to included is frequency view fact the represent to included is Size week. per views ouetto.Ised h olwsta information that was goal the Instead, old documentation. the in conveyed still was information knowl- Most of exchange articles edge. limited of a number to total leading the low; end, remained the In done. rarely but produce. to willing those than of consume number to higher willing a people was There equally contribute. seemed to everybody motivated groups Not the of a members participated. is few actively This a Only hoped. . as problem active common as not was contribution participation and that showed ex- Moknowpedia first with However, periences infrastructure. IT or processes, links, interesting planning, publication workshops, and sem- exhibitions, inars fair trade tutorials, and documentation in n srfrne oeotni te rils It articles. other in often more referenced is and informa- interesting tion, more contains probably article the page The contributions. more aggregate they information more because contain potentially articles article larger of that number average the and characters, article of in function size logarithmic a articles: viewed often The weight artifact authorship. through determined is Responsibility earning points. of karma ways two al., implements et Moknowpedia (Dencheva 2011). points more them collect motivates systematically and some- efforts, to users their gives for exchange This in thing peers. their resulting by published, appreciation are in karma points of karma users’ amount The certain points. a users earns group articles work quality their serving by points karma earn Moknowpedia users in scores Karma assistants. and employees research student as working are Moknowpedia scientists computer of users the being of While most 2012). interdisciplinary, Eisenhauer, reviews and 800 (Prause than articles more for and articles, 1000 over in sions 2011). with al., experiments et (Dencheva field CollabReview for environment Moknow- ideal made an situation pedia This improved. quantity be and quality to articles’ needed the ex- groups, knowledge the improve in to change order In repository. known h rtwyi ocnrbt owl ae iiarticles. wiki rated well to contribute to is way first The revi- 7000 wrote users 50 years, four last the During w ( a ) ucinpeesbge n more and bigger prefers function nMoknowpedia, In I- Procedure — VII-1 yyhi an is Vyrygh iw.Aue ilgtbnspit nhssoebased score his on points bonus get will user A views. eso eoioyadawk ae naGog server. GForge a on based wiki a and repository version sstreknso ead Dnhv ta. 2011) described al., as patterns et earlier: rewarding (Dencheva the of rewards some of combining kinds three Moknowpedia uses in Rewarding are scores how of Refer description reviews. a computed. his for of 5.2.6 all Section of to timeliness back the of sum the on honored. any be that should is wiki rationale the the to articles work, contribution create their to of have part not essential do as users As ratings negative possible. and not only are users Reviews by manually points. created any ar- are contributors an its case, award worst not the does in ticle So, scores. quantity and quality weight article’s an in the results Multiplying numbers weight. two higher a have therefore should ainevrneto olbeiw eadn the Regarding CollabReview. of environment uation Sub- a partner including One infrastructure Euro- development five consortium. the provided from project the partners in nine countries are pean There years. four is Vyrygh 7.1.6.5 h eodwyt anpit sb umtigre- submitting by is points earn to way second The user’s a multiplying by computed are scores Karma hnVrg tre,i a nedda ao eval- major a as intended was it started, Vyrygh When • • • ihacooaebr etoigi h weekly the in mentioning a bar, chocolate a with week past the during points karma most earned who profile their to next icon level a them earns which seScin5427.Wnigtettei rewarded is title the Winning 5.4.2.7). Section (see user the to awarded is title award) week the of (Hero indi h ekymeeting. men- weekly longer the no in is tioned not winner is the chocolate and anymore experiment, awarded the list. of mailing end groups’ the the Since to email an and meeting, Woche” der Held “Moknowpedia the week per Once the of top the 5.4.2.1). at Section users (see top table with displayed table, is ranking user a each in of points of number total The 136). page on 6.8 Figure (see picture 5.4.2.6) Section (see level new a they points to of promoted amount are certain a collect users When EU rjc iia oKrks t duration Its Karakos. to similar project h Moknowpedia The w ( a ) . 153

VII — VALIDATION AND EVALUATION . Devel- ) and the th 30 API until th were written in Java. 5 IDE in special lectures and seminars. Solid XP VALUATION E The topic of the lab was to improve a static analysis The development team consisted of instructors and for four weeks from September opment took place in a5 large pm office up space to from 6 9in pm am the every till day. project’s Subversion 725 repository revisions during were this created time, the software engineering department of a local university. development, familiarized them with agilemethodology, development and generated software artifacts thatintended were to be actually used in the department’s work. field experiment. Also refer tothe Prause study. et al. (2012) for toolkit for Java software. The analysistoolkit components of were the written in Prolog,its while the Application toolkit Programming library, Interfaceintegration ( with the Eclipse However, only the development ofdeemed the suitable for Java a parts study. was ten student developers following an agilemethodology development around a kanbanProgramming board was (see done Figure in 7.3). pairs.before During the the lab, semesters the studentstopics had software received engineering training in on general the ologies and like agile method- programming experience was expected. The project ran Participants choose tasks from aand set complete of them available tasks bytasks contributing are bound to to the differentby reputation wiki. CollabReview, scores e.g. computed The rating articles,quality improving one’s score, or contributingbasic objective to is to articles. achieve anchange absolute The to score, the task’s a current relative score, or to improvea one’s steady rank. contribution for ahave period of to time. be Selected fulfilled tasks be one started. by The one successful before completionparticipants a of a a new task task medal awards can andare overall visible karma points, to which otheris users a screenshot (Agaraj, from 2012). ReputationForgeof showing tasks Figure the that 7.2 set are currentlybox available overlies the to selection a screen, user.information giving An of the the user info task further that he has just completed. 7.1.6.6 AgileLab The AgileLab was a post-graduate teaching activity of The lab took students into the daily work of software The lab’s realism made it a favorable environment for a Additionally, tasks can have a time limit, or necessitate ALIDATION AND The Reputa- VII — V ’s project officer. Harsh sanctions had even been EC ReputationForge promotes a soft competition between So it was decided to not use the source code of Vyrygh However, it turned out that the software development havioral studies and existing reputation-based systems. creating an online community and socialdevelopers network, can where earn points for personalself-fulfillment enjoyment (Agaraj, and 2012). users, who are divided into newbies and advanced users. tionForge system implements a persuasivebased reputation- social game. It aimsengagement at increasing in the a participants’ extensive team’s conceptual wiki. investigation into It a range is of the be- result of an as much as desirable butenvironment; just that as made much its as wiki Moknowpedia.of a The Vyrygh wiki is suitable used inSection the 7.9. field Scores experiment are presented computed in analogouspedia to (see Moknow- Section 7.1.6.4). YetReputationForge the as rewarding presented scheme by is Agaraj (2012). ReputationForge rewarding system ship and responsibility. Lastly, theand available source its code developers were furtherC# cluttered fractions, into Java further and degrading theability source for data code’s gathering. suit- in the evaluation. But the project’s wiki was used. Not these issues to the technicalargued committee, that the such consortium data couldnot not reflect be the used way since the it consortium would works. Most partners own repository as long asplace. the The development valuable work commit took bereaving history CollabReview is of effectively determining lost, precise author- developers had contributed to thethose source seven, code. two had Out not of contributedat to all, the and core software one hadarchitecture contributed specification less than foresaw ten twelve lines.but components only The three were available intory. When the the development project’s reposi- software quality manager raised high-quality software which was alsothe demanded from formulated in the consortium agreement. process would be unsuitable forand evaluation. a After half years, one theremits were to still the only Subversion about repository fifteen per com- month. Only seven development of source code, theconditions originally were agreed favorable upon for the50 evaluation: developers more working than on a coderepository, and base with in an a official Subversion commitment to produce The main ingredients of the concept are aspects of would commit their components first to the partners’ 154

VII — VALIDATION AND EVALUATION ige l,19;Sie,20;Siels 01.See 2011). Spinellis, 2009; (e.g. Seibel, read 1997; to al., easier et gets reducing it King that by so code noise source style is of distracting rules understanding coding the of ease idea to The mod- be 2001). to 9126-1, software (ISO the ified” of capability “the means ability quality internal of definition pragmatic A 7.7). Section (see developers scores the karma for quantity and quality computed CollabReview prototype a experiment, field the Col- During for labReview. environment evaluation suitable a AgileLab the (2012). al. participants et non-lab Prause from contributions few a including xetfrispi rgamn,alo hsmade this of all programming, pair its for Except iue73 glLbkna board kanban AgileLab 7.3: Figure iue72 hoigtenx akfo h vial ak nReputationForge in tasks available the from task next the Choosing 7.2: Figure Maintain- I- Procedure — VII-1 os case. worst AIdcmnigcmet)wr ped ssmallest As upheld. were comments) documenting (API ilto.Tehge h est fvoain ie vi- (i.e. violations of density the higher The violation. eto ... o icsino oigsye and styles coding of discussion a for 3.2.2.3 Section ol eabtaiylw oi a u f at off cut was it so rating low, quality arbitrarily file’s be a could definition, this With ratings. quality lower a had comments quality of any its rating missing lower not the file file, A a in rating. code) of line rule per a olations was @throws) or @return according @param, or comment (like Javadoc tags a missing entity code source every documentation: Javadoc of completeness comments. Javadoc of use correct defined the be through would quality internal denominator, common Javadoc of use the regarding rules on- only was end, the development In while going. developers the with some discuss rules to wanted organizer’s counter- the not even Also, productive. indentation”-rule, “do for the characters like tab or, use constraining realization effective too the consid- functionality, hinder to of were not rules as much Some too ered discussants. all by sensible the that conventions code Java Checkstyle of source set of full the understandability Yet the code. was through quality defined that was be Consensus to lab. the to prior cussion quality. code source hcsyewshnecngrdt nycekfor check only to configured hence was Checkstyle dis- of matter a was quality internal measure to How 4 http://checkstyle.sourceforge.net +10 4 oli ol hc eentconsidered not were check could toolkit hl ihrvoaindniiscaused densities violation higher while , − 10 nthe in 155

VII — VALIDATION AND EVALUATION violations asymptoti- 1 . , or Javadoc. 6 10 0 − in the rating. Below 1 Quality assessment in , while every +10 , the rating would approach 0 Figure 7.4: ScrumLab Kanban board rithm VALUATION E That is, using a number in source code without explaining it; say, 6 As opposed to the AgileLab environment, Checkstyle using 42 without explaining whereor it 43. comes from, or why it is not 41 the ScrumLab was set upReviews similar would to be that created in automatically thestyle through AgileLab. Check- audits. Quality ratingsnumber depended of on violations the per average meant line a review of rating code. of per No line more violations meant aa decrease rating of of cally with the number of violations per line. used its complete set oflike default formatting rules, problems, i.e. magic including numbers rules the case of private class members,public and members. thrice in case of 7.2 Evaluation of Authorship Algo- algorithm with respect to itsagainst soundness copying and of robustness source code (Section 5.3). The eval- Definition of internal quality Javadoc violations, however, were weighted twice in This section describes the evaluation of the authorship until th revisions planning 13 ALIDATION AND (see Figure 5 600 ≈ VII — V and , and Team B twice as 300 ≈ KLOC 5 ≈ , the teams created at a local university. Students got first-hand th for estimating the efforts of story points to be real- 29 http://www.atlassian.com/software/ LARI 5 During the development time from April Many aspects of the ScrumLab were inherently favor- A review meeting with the customers was held once Prior to the lab, the instructors had provided a few able for the evaluation of CollabReview,rate e.g. development teams, two agile sepa- development, assize well and as duration of boththis, teams’ several projects. organizational In aspects addition could to bethe adapted needs to of the experimental design (see Section 7.11). greenhopper/ in their Subversion repositories. At the end of themuch. lab, toring and coordination was supportedban by board a based virtual on Kan- Jira and GreenHopper to organize their development activities asparticular, presence they at liked. a In specific place for development ized in the next sprint.held Daily within “stand-up” the meetings team were throughthe teleconferencing presence without of instructors. Organizingthe them team was and up its to Scrumtwo Master. The months, lab allowing lasted for everyMaster over student of his to team be for the at least Scrum one week. Progress moni- questions regarding the process. Additionally,tioned they as func- product owners anduser customers stories, who and prioritized reviewed deliverables. per week. In this meeting,poker the students played topics from which teams werestudents allowed then to received choose. training The on theogy Scrum and methodol- created user storiesof the to project. elaborate During on the the lab, scope the instructors answered experience with the Scrum developmentby methodology working on asoftware software team. project The in ScrumLab a featuredScrum self-organizing two teams independent with a totaltwo of seventeen teams developers. only The met in the first andmanagement the system, final and meeting. a geo-caching game on the 7.1.6.7 ScrumLab of June 7.4). Apart from these meetings, the students were free Team A had written The project topics were a web-based industrial robot The ScrumLab was a post-graduate teaching activity was not expected. web and for mobile devices. 156

VII — VALIDATION AND EVALUATION (2009). h uhrhpagrtmi ple oteeouinhis- evolution the to applied is algorithm authorship The prelimi- a provide to is experiment this of purpose The ...,rsetvl) h hre h itnebetween distance the shorter The respectively). 7.1.6.2, .. Method 7.2.2 CollabReview of algorithm shows authorship It the that correctness. algorithm’s the of validation nary Purpose 7.2.1 Prause on based is algorithm authorship the of uation o ml o o ag eas ml oebs would base code small a because neither large is too base nor code small The too trivial. be would authorship ing determin- otherwise contributed; Addition- developers available. several ally, is revision first history the development from entire beginning their that is validation this for algorithm. the of qualities the Subversion about sions of results the blame to compared are algorithm sections. following in and introduced 5.3.3.2 were Section heuristics The search collected. neighbor nearest is the heuristics of data process, less this In significantly effort. with nearest results the good finding deliver guarantee but not neighbor do They used. are heuristics expensive, too computationally is for predecessor search the exhaustive an as However, (e.g. steps detection). subsequent clone the during mistake reduces a This making scripts. of the edit chance string predecessor, are supposed smaller and its less and artifact current the Section Karakos and 7.1.6.1 projects Section software in the (introduced FreeCol of and code source of tory • nipratcaatrsi ftetopoet used projects two the of characteristic important An the with obtained results authorship the end, the At • • • Rqieet3 Feil applicability”), “Flexible (Requirement-3: eitneaantattacks”). against and resistance “Robustness and (Requirement-5: text copying of against moving robustness certain a reveals and ob- methods, results comparable the with than tained better are that results produces computations”), “Validityof (Requirement-7: flawed obviously not is data projects’ medium-sized to small- handle can h eut r ute nlzdt rwconclu- draw to analyzed further are results The . I- vlaino uhrhpAlgorithm Authorship of Evaluation — VII-2 hl.Teeprmn eeae nemdaeresults. intermediate generates experiment The while. ihtesm,otmldsac.Det h costliness the to Due distance. optimal same, the the with found heuristics which checked was it that, After Comps. h aacmae h eut fteagrtmwith algorithm the of results the compares data The and Karakos projects the for results the lists 7.2 Table ... nlsso ers egbrheuristics neighbor nearest of Analysis 7.2.3.1 data. authorship heuristics, actual search the neighbor and nearest of the kinds about data two data: generated experiment the of Conduction Data 7.2.3 in confidence results. the increase further will experiments, Later worth- deemed not were results validity rock-solid the data, increase the would of two data with more done While only were projects. they experiments, the for sary algo- the environments. that different show in to works nature rithm their in Finally, differ maturity. projects prototype product the a of of software scope require the would proof but could out are base that code problems large duce a and representative be not eut bandfo Subversion from obtained results data authorship the in Differences 7.2.3.2 weeks. all for add results to for possible waiting not without was artifacts it search, exhaustive the of one another or search, exhaustive the as artifact same distance. optimal the and artifact, new the nearest of true neighbor the delivered in search exhaustive run The were parallel. search exhaustive an and all neighbor, heuristics nearest different the re- for FreeCol, searching and When spectively. Karakos in artifacts process software and artifact. one, process optimal software the to distance the neighbor’s of nearest off suspected is the distance which by factor insertions, age of number total the by heuristic, the by found FreeCol: u osvrlhnrdhuso rcmuigneces- precomputing of hours hundred several to Due h nlsso h ifrn ersiscvrd7500 covered heuristics different the of analysis The steaeaenme fcmaiosdn per done comparisons of number average the is Hits stenme fcretnaetneighbors nearest correct of number the is Recall sti ubrdivided number this is blame Badness . steaver- the is 157

VII — VALIDATION AND EVALUATION - N I - exclud- 1.0 1.0 2.0 5.0 9.9 35 94.3 27.2 43.8 19.7 24.3 50.9 70.2 87.7 . 6 Comps. 3.1 1.1 9.6 2.3 2.2 1.2 1.1 1.1 1.1 1.1 1.1 1.1 132.9 161.2 excluding trivial lines) Badness contributors ( 03 . FreeCol 7 66 ( . 6 44 31% 17% 94% 95% 78% 88% 93% 95% 95% 95% 96% 96% 96% 99% . Recall 7 times. n for greater sample sizes. An investiga- 24 . Hits 2333 1307 7144 7242 5977 6713 7107 7182 7220 7253 7275 7285 7315 7588 4 NDEX - I VALUATION 1.0 2.0 7.1 1.0 2.0 4.9 9.8 18.1 19.0 48.0 70.4 86.3 E 101.5 Comps. Relying on the inverted indexthere is is most only promising. a But small increase in the accuracy of 33% a different author isor reported 31% (2,955,080 did characters not match).an The artifact average and distance its between parentfacts). is There are 1025 10581 (483 artifacts forare of non-obsolete, current which thus arti- 868 artifacts (or have been 8%) times. revised 690 several artifacts are notsor different (branches from and their renames) predeces- and 227 are not yet obsolete. ing trivial lines). When includingaverage obsolete artifact artifacts, the has contributors. This could mean thatorigins either was track lost of or some thatcode. a few Two individuals main reworked the developersand normally 18% contribute to an 65% artifactartifacts (59% only). and 21% The for average artifact non-obsolete or is branched moved, renamed 7.2.4 Results From the data, the followingnearest results are neighbor obtained heuristics for and the information. the mined authorship 7.2.4.1 Results for nearest neighbor heuristics VERTED The average artifact has 1.8 4.9 3.1 1.7 1.4 1.1 1.1 1.1 1.1 1.1 1.1 91.5 96.8 91.1 Badness Karakos ALIDATION AND 2% 83% 83% 20% 21% 57% 61% 66% 71% 74% 76% 79% 80% 81% Recall VII — V 149 Hits 6299 6307 1533 1612 4360 4628 5054 5414 5624 5790 5995 6092 6150 Table 7.2: Comparison of nearest neighbor heuristics 20 50 75 100 200 1 2 5 10 NDEX NDEX NDEX NDEX NDEX NDEX NDEX NDEX NDEX I I I I I I I I I authors per artifact, respectively. This 56 RDCSSR EARCH if the authors of trivial lines are not consid- . AME P S 3 All non-obsolete artifacts in Karakos total AME Non-obsolete software process artifacts in N 58 . N 3 REE M IM NGL and NVERTED NVERTED NVERTED NVERTED NVERTED NVERTED NVERTED NVERTED NVERTED I I I I I S S I I I I T S All combined Heuristic , or 37 . The average distance between two artifacts is 2172 4 49 . and 11% (considering only non-obsoleteand artifacts: 9%). 86% FreeCol FreeCol total 269,981 linescluding (9,603,912 white-space characters) and in- comments. For 88,848 lines or ered. If obsolete artifacts areis not considered, the result is as expected asand obsolete their artifacts history is are thereforemain early not contributors artifacts so for colorful. artifacts usually The contribute two 85% neighbor. This phenomenon is mainlytions due of to the reorganiza- repository andnew to artifact branching, names which in leads Subversion. to observation It that is the supported number by is the 21%) almost when cut only in non-obsolete half artifacts (949 are or considered. ferent author is reported (12characters million or 37% out did of not 33 match). million characters; or 2231 characters if only non-obsolete soft- facts are non-obsolete artifacts, whichartifacts means are that still many in their early forms. A high number of Karakos 927,305 lines including white-space andincluding comments, and branches. For 336,965 lines, or 36% a dif- The average number of different authors per artifact is 4 ware process artifacts are considered. 57% of all arti- 158 1763 artifacts (40%) have a distance of 0 to their nearest

VII — VALIDATION AND EVALUATION a o ute nlzdbcueo a run-time bad of because analyzed further not was (e.g. h aebsdhuitc aeasrn orlto of correlation strong a have heuristics based name The T VERTED ewe h w heuristics. two correlation the Matthews a between compute to possible now is ( hit ther artifacts which for they to regard com- with by heuristics explained two is paring increment This heuristics. ferent unfavor- gets soon. speed able and precision between tradeoff the the sense to make stick not to does it medium Hence, with misleading. lines are that certainties seems It sampling. random than n, that so required are n of sizes what into tion emt opeetec te ufiinl elto with combinations well Also parallel. sufficiently in other them running each justify complement to seem add would and results benefit. different heuristics, very to correlated return benefit negatively however, add Strongly would they other. if each prognosis a for allow not (e.g. predictions similar to make benefit they few because other add each do heuristics strongly correlated example, positively For results. better guarantee always about of correlations negative r are there FreeCol For no tic. was there between and correlation though, non-negative, are relations bigger for smaller for stronger are and about r r achieve the to similarity, chance their to good due a expected As with results. parallel better in run they be then can heuristics, correlated negatively two are there while cases, same the in r fail and succeed heuristics both S I DEX D D NVERTED m m m m I NGL I REE unn ifrn ersisi aallde not does parallel in heuristics different Running dif- combining by rates hit increase to possible is It NDEX I NDEX NVERTED = 0 = 0 = − → S n h aebsdheuristics. based name the and n I hit S IM r NVERTED P − m EARCH . . RDCSSR 95 5 0 I nldstecretrsl,fudta o great for that found result, correct the includes n N or 1 1 1 NDEX . 0 = frvr ifrn apeszs e.g. sizes, sample different very (for 2 I rms ( miss or ) AME and en htte opeetec te.If other. each complement they that means n NVERTED I between miss between NDEX . I ≥ I 45 NDEX NVERTED ersiscreaesrnl ihabout with strongly correlate heuristics n orltoswith Correlations . nrae h eal u h heuristic the But recall. the increased I I o ahatfc,ahuitccnei- can heuristic a artifact, each For . NVERTED and 50 NDEX between 100 T ( S eoe os,adee worse even and worse, becomes n r 0 S T I REE NGL m ,rsligi w aarw.It rows. data two in resulting ), NDEX M REE 100 and N I n ≈ NDEX S P S AME I ≤ S EARCH RDCSSR M and NDEX 0 EARCH I 100 r . N NVERTED 10 m 3 .Wa orltosdo correlations Weak ). AME ersi o ra as n great for heuristic n .FrKrksalcor- all Karakos For ). I → ( NVERTED I- vlaino uhrhpAlgorithm Authorship of Evaluation — VII-2 r and 200 n n te heuris- other any and I ≈ 1 and NVERTED or and ol enthat mean would to ) 0 S S I . NGL S 7 NGL NDEX I n weaker and ) NVERTED M I r N NDEX m I P P I AME NVERTE NVERTE RDCSSR RDCSSR I 200 0 = NDEX 200 and , or , . r I I 95 N N m n ). - - - - , ssonbfr,teeaetefaue httealgo- the that features the are these before, shown As to used was evolution software hand-crafted small, A a ot nFeClterslsaesmlr n major One similar. are results history the the FreeCol that In so lost. it was about Subversion telling without c.Cna 16)sLaw). (1968)’s Conway (cf. h iiaino loihsta eemn contribution determine that algorithms of limitation The Karakos in contributors major of presence strong The iin oie ydfeetatos egdwt con- with merged authors, different by modified vision, version. bk 07,ti ol etepiayrao o the for reason primary the be could Jarz- this and (Basit 2007), projects abek, 30% for to reported 20% been as have and But clones clones, high. detect quite not appear does may Subversion 36% of Karakos for ences Subversion different and to algorithm lead the should between this results Hence, handle. can rithm changed. was indentation and artifacts other from tents re- obsolete an from recovered another artifact, to process moved software was code crafted Source specifically manipulations: recognize text could algorithm the if test data authorship for Results 7.2.4.2 in overhead Karakos. access database resulting and characteristics yloiga hti hr,i htte anthonor cannot they that is there, is what at looking by limitations Known 7.2.4.3 few borders expectation to organizational due across the ownership development code collaborative with implicit strong concordant is there is that artifact each line. non-trivial for a of context the in not lines if trivial copied that not is are reason was The lines non-trivial identified. of or successfully cloning out that left indicates are out left lines not trivial when FreeCol and Karakos Sub- by separately handled not from is comes that difference reformatting biggest by code author the same But the to algorithm. attributed the is con- artifact information each license in multi-line tained the that is here factor around moved been had artifacts many checked this, who Besides author in. the it artifact to such it in attributes Subversion checked who first. author same the to tributed Karakos in ( Lan- code guages Description Services source Web the from results auto-generated of the are 14% of about sanity that for found check manual A differences. tfis,teaon frpre uhrhpdiffer- authorship reported of amount the first, At h ifrn ubro uhr e riatin artifact per authors of number different The WSDL )adta otaesaeteeoeat- therefore are areas most that and s) blame 159 .

VII — VALIDATION AND EVALUATION . A manual analysis into the possible ) of a software process artifact is prone blame RDCSSR P VALUATION E . As opposed to Subversion, CollabReview’s algo- However, a limitation with data from real projects is The next step was to trial the algorithm with the data First, it was tested how good the nearest neighbor Next, various key facts were measured in the source Though it is bad practice to commit artifacts that can NGL S results (Requirement-7: “Validity of computations”). that the true development history cannot be recovered and copied source code, andsoftware process the artifacts. right predecessors of from two real-world projects. It wastion applied history to of the projects evolu- withhundreds several of thousand thousands revisions, of lines, and several developers heuristics are. While a nearestthe neighbor true is not predecessor perforce thatmost the probable developer one. used, Finding itto a is reduce nearest the neighbor the number means ofpredecessor edit into operations its that successor, transform and a therebychance for reduces errors the in the transformingcontrol process. system that A only revision considers the single predecessor to make many mistakes. Whileis the hard true to nearest find, neighbor thein algorithm most finds cases a using very different good heuristics. neighbor code, and then checked forthe soundness. algorithm It made was authorship found attributions that in that over deviated 30% of theSubversion cases from the resultsorigins obtained of the by deviations supported the plausibility of the IDE rithm is able to copeences. with Yet it resulting also indentation has differ- problemsinto with two. lines that are split be auto-generated (Richardson andthe Karakos Gwaltney, repository 2005), contains several ofgenerated these. source code Auto- has aneous, tendency which to means look homoge- that therelines are but few lots very of distinctive semi-distinctive ones,search times. resulting in Furthermore, until long identified aslarge trivial parts code, of these software processcloned artifacts and will be appear attributed tothem the in developer who for checked the first time. 7.2.5 Summary orous testing using unit testssoftware and evolution a history. specifically It crafted correctly identified moved The algorithm and its implementation underwent rig- (Requirement-3: “Flexible applicability”). ( ALIDATION AND VII — V ” into an artifact should not always be recorded as } Starting at revision 1928 (in FreeCol) a new developer From looking at evolution snapshots (revisions) one The algorithm can be unfair to major contributors Subversion errs but there istrue no in guarantee all that cases this or holds that there is not a third authoraccount involved or pair programming). committed several revisions after reformatting with his mechanism. Yet it isformatted efficient source and passages like also comments. works for not cannot find out if codeor was copied if from it somewhere was else Subversion written and the anew. algorithm differ, Whenever it results is probable between that author. For example, the firstline author “ that introduces the the author of this line.as So trivial. the On line the has other toas hand, be trivial considering recognized too would many not lines from detect one code artifact areas to the thatissue other. are with The an moved algorithm occurrence solves statistic this which is only a crude ior is, is difficult: Theest algorithm neighbor efficiently but finds it a cannot near- the guarantee that developer based this his is the newtion version one is on. irrecoverably This lost informa- duesion to control the systems way work common (cf. revi- hand, Section trivial 3.4). lines On should the not one be attributed to the same a test repository with chosenartifacts, features copying (like of renaming substrings, of restoringfrom of outdated substrings artifacts, indentation changes)ally and studied manu- results for twoKarakos natural and project FreeCol. repositories But defining what correct behav- of Alice with just aeffected few by changes, the Alice bad would rating be scoresthe most because most she characters contributed and lines. 7.2.4.4 Discussion contribution. Developers cannotcode. own Likewise, deleted having source madedeleted contributions by that another developer, were cannot be honored. process artifacts. For instance, if Bob breaks source code contribution for things that areif not there there. was For a example, misleadingthat comment a or developer dead removed source to code improvealgorithms quality, will then have such no way of honoring that kind of To verify that the implementation of the algorithm does which was not identified (e.g. using someone else’s what it should do, extensive testing was done: unit tests, when someone else contributes just a bit to their software 160

VII — VALIDATION AND EVALUATION ilso htteagrtmadrptto crscalcu- scores reputation and algorithm the that show will obtained results The too. similar, is code source wiki char- This characters. line new of use the to regard with eso hslmtto,terslsfo hssuysupport study is this algorithm from the results that the limitation, Regard- this artifacts. of their less of if quality contributors the major degrades to someone as well by as artifacts something, process deleting to software unfair up be clean may who correct- algorithm developers the The verifying results. the of obtained way is of no it ness is if there So and source. predecessor right identified actually was an code from source if copied unknown is it i.e. anymore, andfo t n h eutn am crs t pur- Its that show scores. to karma is resulting pose the and it, ob- from information tained further authorship the is analyzing algorithm by authorship evaluated the evaluation, this In Purpose 7.3.1 consis- code developers. source with tently associate can importantly, CollabReview more the and that code, regarding source their styles of cleanliness constant rather have that are developers implications The consistent. are CollabReview fairness and consistent. predictability are of scores matter karma a that also is algo- It authorship the rithm. of it correctness then the supports consistent, further is algorithm obtained authorship information the authorship through the and scores karma If Scores Karma of Consistency 7.3 sound. and consistent are it by lated experiments Further languages. other well those therefore to should transfer here presented were that Java for most particular, In MediaWiki. systems or Subversion control revision including most by exploited is acteristic • • hsscinsosta am crscmue by computed scores karma that shows section This similarity their share languages programming Many on and sound orecd Rqieet5 Rbsns and of “Robustness attacks”). copying against resistance against (Requirement-5: resistance code its source to due robust I- ossec fKraScores Karma of Consistency — VII-3 ihatosi nomto odrv am scores. karma derive to information authorship with that, determines algorithm authorship The E. on worked h ehdi nrdcdwt neapebfr the before example an with introduced is method The orecd r aiodbtpormesaecertainly are programmers but manifold are code into respectively) source debt, technical (or reasons bugs possible introducing that for is is aspect definition important quality An a provided. and procedure, test consistency Method 7.3.2 otiuint oryrtdD. their rated from poorly scores to low contribution receive Green while especially E, and to main contributions Blue the his is from score he an karma achieves data as Yellow average A. Red, enough artifact for well-rated is to karma contributor there good a example, compute the to score In karma the in calculations. out left are however, artifacts, combined Some are They artifacts. process software the of Green Red, Blue. from some code and with contains C Red while by only Green, from written Yellow help mostly while was E, A example, and for D, C B, C, A, to to Blue Green to and C, left and D, from A and E to contributed D, Red C, B, right). non-obsolete (A, five artifacts of consists process developers base software other code Their by well. edited as possibly are process that software to artifacts contributing collaboratively i.e. are code, right) developing to left Green, (from Red, Yellow developers and The Blue conducted: is evaluation concept the how the of of example an visualizes 7.5 Figure procedure the of Example 7.3.2.1 2010). al., (Schr et constant rather are differences interpersonal that are There factor. important an • • nadto oti,qaiyrtnsaeaalbefrall for available are ratings quality this, to addition In • Rqieet7 Vldt fcomputations”), of “Validity (Requirement-7: tivsiae h nuneo h egtn func- weighting tion the of influence the investigates it and computations”) of “Validity informa- (Requirement-7: authorship tion, obtaining of ways compa- other least to at rable or better, is algorithm authorship the then well as repository, consistent be revision to software has a information across authorship in consistent artifacts are process scores karma if w htgvsdfeetwihst artifacts. to weights different gives that tre l,20;Brown 2006; al., et oter ¨ 161

VII — VALIDATION AND EVALUATION ) a ( 0 . q ) that were presented in Section 7.1.6. a ( 7 q VALUATION E formation is consistent and karma scores are sound. Assess the quality offacts some to software generate process a arti- setset). of quality This samples is (training Compute developers’ karma scores as thequality average of their contributions (training). Use authorship information andpredict karma the scores quality of to thediction). remaining This artifacts is (pre- Correlate predicted and assessed ratingsquality. of A artifact high correlation means that authorship in- An open source project used as an evaluation environment, see 7 The evaluation uses the evolution history of source To reduce the influence of such noise, quality is de- The simple four step procedure for checking the con- 4. 2. 3. 1. regard to this topic. Section 7.1.6.2 code from the projects Karakos,FreeCol TornBond, and AgileLab, Log4j 7.3.2.3 Quality assessment of artifacts for all software process artifactsabilities of and the dispositions project. do not While factors change affect easily, the a quality lot evidenced in of tion an 3.5.2.2). artifact (cf. Among Sec- them isbe that consistent reviewers in might rating not thejectivity quality in of what artifacts comprises due good todevelopers quality; sub- might or pick that the the most ambitious best tasks and (see bug-prone also Sectionan 3.5.2.2). unwanted noise These because deviations they are originate from factors sessments should be made throughand objective not measuring, be affected too much by task difficulty. fined here as the cleanlinessstyles and of the source absence code, of i.e.an code implementation smells. code task The should difficulty not of coding affect a style developers’ too much. Also see Section 3.2.2.3 with quality of the software process artifact. So, developers in better artifacts more often.have Consequently, a they higher should karma score becauseof quality their is abilities the and result dispositions. sistency of karma scores works as follows: The evaluation procedure requires quality assessments The quality of a developer’s contributions affects the which are not part of this analysis. Instead, quality as- who make high quality contributions should be involved ALIDATION AND ¨ oter et al., 2006). VII — V Next, a quality rating of the previously left-out soft- Developers are individuals with varying codingand abilities attitudes towards software andfactor quality. This will human reflect indevelopers are their involved contributions. inparts the If of the evolution different code of base,different different then quality. Although different other parts factors willthis play have relationship a a is role, no too, coincidence (Schr formation has to be (giventheir that coding authors cleanliness). are consistent in 7.3.2.2 Four-step evaluation procedure authors’ karma scores. Inquality the score best is case, equal this to predicted the actual assessed quality score. Figure 7.5: Predicting software processusing artifact developer quality karma scores The better both scores match, the better authorship in- ware process artifacts is predicted by combining their 162

VII — VALIDATION AND EVALUATION where ... oprsnwt te algorithms other with Comparison 7.3.2.4 score. quality sessed ρ ρ con- project-dependent a stant adding by projects different adjusted in therefore quality is lower or higher generally levels a quality at artifacts many too interval the over nicely distributed density weights with 0 critical) to irrelevant (from levels severity To code. source of line compute non-empty per problems of ber density problem the quality): review ity omn noae ahln fsuc oewt its with code source of line each annotates command is It changes. of coarse. type rather or thus measure amount This the up. consider add not shares does contribution his list, this in the tors of third one about has rest who ( one file next a the for by authorship followed the has of contributor third latest one about The authorship. sharing as utors contribu- own its go with The files each tor. source revisions, of of development, series During revision a through a is. of file contributor source a the who about metadata paragraphs. following the in scribed authorship several be there of functions Let that algorithms. to as other compared of therefore good that are as results least Its at algorithms. other is that should information prototype output CollabReview the in authorship The used 3.4.2). algorithm Section (see are literature information in authorship known obtaining of ways Several E C , 3 1 DIT ONT 1 o ahartifact each For > max( = , 3 c 2 3 2 3 0 and , q uhrhpalgorithms authorship x uhrhpalgorithm authorship then ρ 1 n au to value C soeo h loihsfo h aiisde- families the from algorithms the of one is ρ 1 × × he orcin r ple oke values keep to applied are corrections three , 1 ONT s ∞ − ahpolmi egtdacrigt four to according weighted is problem each , ( 3 1 ,a u, ρ 3 1 10 eoecmuigtevleof value the computing Before . 2 foecnrbtrapasmlil times multiple appears contributor one If . ≈ loih osdr h atfiecontrib- five last the considers algorithm = , : ) ρ min(10 0 1 . p r 3 222 eod ftedfc e ieratio line per defect the if Second, . A a x ( ( ρ → ∈ ,a u, n oo o l te contribu- other all for on so and , 1 ) , A [ 4 7 − ) otherwise , hcsyesbisaqual- a submits Checkstyle , − 10 → ρ ρ 1 , [0 2 +10] scluae stenum- the as calculated is )= )) uvrinmaintains Subversion , Subversion’s 1] [ − − I- ossec fKraScores Karma of Consistency — VII-3 fo a ogood to bad (from 10 q 10 ρ ( 2 a , +10] or ) = ie h as- the gives 3 1 ρ +10 1 q ≈ without , Finally, . ( a blame 0 First, : ) . 333 from ), aepoesatfcsoe ult lse,krascores, karma classes, quality over artifacts process ware h vlaingnrtddt ntedsrbto fsoft- of distribution the on data generated evaluation The eso oprsec eiino l ihisprevious its with file a of revision each compares version esosaeue here: used are versions aapcae like packages Java .. Data 7.3.3 packages imported to reputation binds that algorithm an P baseline: Comparison E to author. analogous original defined the to traces back better ownership code algorithm this Consequently, Subversion’s of algorithm. short-comings several improves It Section 5.3). (see prototype CollabReview the in used gorithm is algorithms thorship authorship. few only receive editors file. major the i.e. of ship, line one least at edited owned. lines of number the by to owned proportionally is authorship it revision. then the committed changed, who or developer new the is line a If revision. Sub- information, this get to order In author. respective esdqaiyo riat.Ti eto rsnsthat presents section This as- data. artifacts. and of predicted between quality correlation sessed the on data and what prescriptions no makes function weight the w of influence the also algorithms, ship functions weight Different comparison P for ob- provided as are results algorithm The this with packages. tained of reputation the computes imported for looks it Instead, need history. not evolution does file’s and a algorithm authorship an not is gorithm (Schr developers of instead E A DI ( UTH • • • a ACK ) U w w characters, in size their to w hudb netgtd hl h euainmodel reputation the While investigated. be should 1 l s uhrhpalgorithms authorship ie aeatosi oalpormeswho programmers all to authorship same gives onstelnsi riat and artifact, a in lines the counts ie otaepoesatfcsawih relative weight a artifacts process software gives . sin h aewih oalartifacts. all to weight same the assigns java.lang A DI UTH U. ACK tre l,20b.Teral- Their 2006b). al., et oter ¨ hc r ae nteal- the on based are which , w nadto oteauthor- the to addition In hudlo ie three like, look should ntesuc oe and code, source the in Schr h hr aiyo au- of family third The E DI tre l define al. et oter ¨ I net author- inverts E DIT A UT assigns U 163 is

VII — VALIDATION AND EVALUATION ) a ( q . For +10 ) UTH UTH ONT a ( 0 ,A ,C ,A q ) ) ) . The next f f f ( ( ( 05 s s 1 . 0 w w w +5 p 0 -5 of karma VALUATION E -10 of Spearman’s rank correlation betweenand assessed predicted software process artifact quality every combination of authorship algorithmthere and project are three values for each of the analyzed weight Figure 7.7: Quality karma scoresthorship by algorithms three in selected FreeCol au- alteration has only been applieddoes to not the affect the plotted computation data.the of It correlations correlations. are All signiﬁcant of at section will compare the results of more algorithms. 7.3.3.4 Comparison of prediction quality by means idea is that better algorithms shouldthorship give more data, precise which au- in turn shouldkarma result scores, in and more therefore precise in better predictions. Table 7.3 estimates the quality of different algorithms in terms The comparison of authorship algorithms is based on the ) a ( 28 10 was s . Fig- 9 q w 15 c 10 8 10 − ALIDATION AND 7 14 6 82 5 88 is used. 4 82 VII — V 3 68 ONT 2 45 1 33 instead. Cyan (upper) bars 0 22 ) a -1 17 ( 1 -2 11 w -3 16 -4 12 -5 14 5 -6 ments -7 10 7 algorithm. For violet (lower) bars, artifacts are -8 7 -9 UTH 28 -10 Figure 7.8 depicts thepredicted relation quality ratings between for assessed eachafter artifact and performing as a scatter 10-fold plots crossbeen validation. altered The for plot presentation has in afor way y that coordinates. ranks A are small used before random computing value ranks has to been make added them unique. But this differ from brown bars because C 7.3.3.3 Cross-validation of predictions and assess- example, karma scores for threeship combinations algorithms of and author- artifact weights areure displayed 7.7. in Each Fig- triplet offor bars one represents developer. the karma For scores process brown artifacts (middle) are bars, weighted software accordingand to authorship size information has been obtained with the other projects are similar. 7.3.3.2 Karma scores assessments of source code they contributed to. As an Before other data wasproject-dependent generated quality for adjustment each constant project,calibrated the ﬁrst. The goal wasfacts that were software approximately process normally arti- distributed overquality all classes without a tooure long 7.6 tail shows at the class distribution ofity artifacts classes over for the project qual- FreeCol. The distributions for the Figure 7.6: Number of FreeCol artifacts per quality class 7.3.3.1 Distribution of artifacts into quality classes A The developers’ karma scores are computed from quality weighted uniformly 164

VII — VALIDATION AND EVALUATION hc nsse oefi rrobust. or fair more seem ones which That significant. statistically not and small and are algorithms weights authorship between differences the Also, underlying the about conclusions draw to made the were After classes. quality onto artifacts process ware h hieo uhrhpagrtm n seilyar- especially and algorithms authorship of choice The soft- of distribution normal approximately an is There hsnudrcnieaino te hrceitc,e.g. characteristics, other of be consideration should under weight chosen and algorithms authorship that means fair. generally are results the authors, and breadth artifacts full all the of of Over predictions quality. the artifact for process matter software much dif- very the not however, do cases, ferences most In treated contributors. being it in of because unfair and problematic arbitrariness of be feelings might create This could users. scores few karma a different very for in result can weights tifact score karma from results of Discussion 7.3.4.1 algorithms. authorship scores karma of analyses further tested, was distribution discussion and Results 7.3.4 projects. all in artifacts number process high the of at cause significant are correlations the rithms, measurements 0 n -10 Predicted quality rank(q0(a)) sesdquality Assessed a C (a) 0.31 = r analysis -5 ONT w s authorship , 0 w l iue78 cte lt fpeitdadasse ult o FreeCol for quality assessed and predicted of plots Scatter 7.8: Figure and q 5 ( w n a 1 ) fnnoslt software non-obsolete of o l uhrhpalgo- authorship all For . 10 I- ossec fKraScores Karma of Consistency — VII-3 0 n -10 Predicted quality rank(q0(a)) p sesdquality Assessed 0 b E (b) 0.43 = r . -5 05 DIT be- authorship 0 hyaenihrmc etrnrmc os.This worse. much nor better much neither are They ... icsino eut rmcross-validation from results of Discussion 7.3.4.2 htaeqiedfeetfo h te loihs Yet results. algorithms. acceptable achieves other still the it from different quite reputations are computes that It contributed. recently who opers au- of comparison for results of Discussion 7.3.4.3 reasonable. are results the that indicates with obtained results code legacy to due contributions. history third-party poor and a weak in for lie may reasons predictions the Log4j, For pair its nature. from programming result weak probably The AgileLab TornBond. for original in correlations (bad) prediction the aid of to seems loss history The code. and source copied generated from suffers its Karakos to whereas due history, expected good much was FreeCol in predictability FreeCol r exists. definitely artifact process software and developer with significant large to Due C s ONT ngnrl h orltoscefiinsaesmlrto similar are coefficients correlations the general, In Karakos are projects the for values average The 0 = q . 5 srte rd eas tol osdr devel- considers only it because crude rather is ( 32 a r fqaiyprediction quality of hrhpalgorithms thorship ) s TornBond , 0 = n 10 ntedt es l orltosaehighly are correlations all sets, data the in . p 43 n Log4j and , 0 n

-10 0

P Predicted quality rank(q (a)) r 0 ACK s . 05 0 = h eainhpbetween relationship The . sesdquality Assessed c A (c) 0.39 = r ySchr by -5 . 38 UTH r AgileLab , s E 0 = tre l (2006b). al. et oter DI ¨ authorship 0 U . 27 ie slightly gives h good The . r q 5 s ( a 0 = ) 165 . 21 10 ,

VII — VALIDATION AND EVALUATION ¨ oter 0.30 0.34 0.32 0.32 0.30 0.35 0.29 Avg. as bound 1 w attitudes l w (.28) Log4j s w .30 .30.26 .32 .25.25 .26 .25.25 .29 .25.25 .23 .27.32 .26 .31 .30 1 w l w (.43) FreeCol s w .31 .31.43 .41 .44.39 .43 .39.46 .41 .46.43 .43 .45.46 .46 .46 .48 VALUATION 1 E ¨ oter et al. (2006) argument that package imports w Karma scores are based on observations of manifested Generally, there is a tendency that projects with more programming behaviors. But social andimpact other effects behavior can by e.g.tudes. superposing For instance, developers’ time atti- pressure canming influence behavior. program- Psychology refers to predictability. This higher diversity oftors reputation leads indica- to fewer smoothing,entiated and predictions allows that more lead differ- toet better al., results 2006b). (Schr While all correlations are significant, pair- sarily significant. Differences are only probable trends 7.3.4.4 Threats to validity Correlations express that a relationship betweenables two vari- exists. For prediction, anot cause-effect necessary. explanation is It is impossibleresponsible to for say both if observations. something else Forimports is example, package influence quality asSchr they indicate complexity. indicate task complexity. Some developersbad could have luck having to write more complexcode, source complexity code. could radiate ontoever, developers. the experiment How- was designed toof reduce such the confounding effects factors: Checkstyle measures how rather independent of task complexity. all projects and algorithmssoftware exhibit process artifacts that with weighting theable. same all weight Although is larger favor- software processa artifacts large contain sample of anot developer’s outweigh coding samples style, from it smaller should artifacts. developers per software process artifact show a higher l Through specialization on certain parts of the source wise differences between two correlations are not neces- without statistical confidence. “clean” source code is, and cleanliness of source code is w . (.09) l AgileLab w be- s DIT E w .09 .08.36 .39 .17.26 .38 .24.17 .41 .12.18 .38 .17.26 .18 .25 .25 ) over DIT 34 E . 1 ALIDATION AND w = 0 s l r ( w (.40) VII — V 1 w TornBond , too. However, in the s I w .34 .33.43 .39 .34.32 .39 .32.35 .39 .35.31 .44 .29.37 .38 .37 .41 DI E ) and 1 31 w . = 0 l algorithms report more developers w s (.36) r has an advantage because it only looks . ’s fuzziness and copy detection could ( Karakos 1 . this loss seems to be beneficial. On the s UTH s 0 w A w .33 .33.31 .33 .31.31 .32 .31.33 .31 .31.32 .30 .30.34 .31 .33 .33 U UTH ONT ), C A | ≥ UT ’s performance, although it is constructed in- φ 31 U . A r U I | Table 7.3: Prediction quality as Spearman correlations for different authorship algorithms and weights UTH UT ONT DI DI DIT ACK ACK = 0 P A E E A P C E The average correlations for the different weights Karakos’s poorer revision history conceals true author- s r thors and packages, only 16%6.4% (FreeCol), (Log4j), 2.2% or (Hydra), 3.4% (LinkSmart)ficient revealed a phi coef- diction. But not only byfor authorship algorithms, but also dependently from authorship information.be There an may implicit authorship dueof to developers a for possible certain preference packages. tasks that However, involve there respective wasa only direct few relation indication between of authors and package imports: be a bit too ambitious inthors, attributing leading text to to a its worse original performanceFreeCol’s au- compared good to history reflects in betterall predictions algorithms. by The results foras all good, other and projects especially are AgileLab not ming with development its methodology pair adversely program- affects pre- quently, more predictors are available per artifact. ship. Here at who touched a softwareseems process to artifact be recently. the This betteravailable. alternative if only poor history is relatively strong influence on sourceflects code; in this still effect good re- valuescase for of average, the per software process artifactbecome because associated a with developer an can artifactcopied just by into contributions the artifact by another developer. Conse- cause it loses some informationmajor by editors treating alike. minor and represented Minor in contributors proportional algorithms, who seem are to less have a ( Among all contingency tables for combinations of au- worse results than its proportional counter part 166

VII — VALIDATION AND EVALUATION ol aet eipoe uhmr.Nonetheless, more. much improved be to have would Cross-validations ratings. quality with associated were developers’ recognize to trained was system reputation A Esr 96.Sca fet fgopwr iethe like work group of effects Social 1986). (Eiser, .. Summary 7.3.5 modeling. better with better would become and less substantial, rather is are prediction correlations the then Still fashion, accurate. static a behaviors in coding modeled dynamic are If modeled system. not reputation are the effects in These up give standards. and higher resign own could his standards conversely, or, higher fix; with to one one the attitude other quality the lower for may the problems it with leaves one artifact, the process that software possible same be the in but function- with different ality developers on two working are attitudes quality there different if example, Wegge, For or 2001): 3.5.4.1 Section (see attitudes developer and rider attitude different dif- a a reveal For may developer time. a over object, changeable ferent and object target a to h crs ult ssfcetfrti eerhbecause research this for sufficient is quality scores’ the robustness and reliability scores’ The not attacks. are to they immune resistance, tampering authorship some investigated have the algorithms while Sec- Third, (see tasks 3.5.2.2). difficult tion more for affinity an have ones excellent might reli- because to programmers suitable “bad” be identify not ably may scores karma to Second, difficult to and model. due unpredictable also are but that algorithms influences the social this in First, shortcomings perfect. to from due far is but substantial are ratings ity the from Consequently, predicted scores. karma be contributors’ can quality artifact that showed that artifacts process software were data Training code. source of cleanliness regarding behavior programming h orltosbtenpeitdadasse qual- assessed and predicted between correlations The • • • • okcrety Rqieet7 Vldt fcom- of “Validity (Requirement-7: correctly, work developers, with eeoeshv ahredrn oigstyles. coding enduring rather have developers and putations”) algorithms authorship and sound is data authorship consistently code source associates CollabReview are scores karma but consistent, coincidence, no are results or sucker fet a ute fetperformance affect further can effects I- aiiyo optdKraScores Karma Computed of Validity — VII-4 free hl am crscmue yClaRve r con- are CollabReview by computed scores karma While el npriua,tepeiinof precision the particular, In well. hsisei drse nteeprmn nScin7.4. Section in experiment the in addressed is issue This according chosen be should algorithm authorship The h olbeiwcnetbid nkrasoe to scores karma on builds concept CollabReview The . aiiyo optdKarma Computed of Validity 7.4 reputation. social actual as same the are scores karma CollabReview. for choice good a is It mind. in ness algorithms outperforms It like algorithms. other the to pared as robustness expected and fairness like criteria other to small. rather Yet and insignificant predictions. statistically of are precision differences the and scores karma ences scores code. source karma and between exists relationship significant a n nenlqaiy(eurmn-:“aiiyo computations”). of “Validity (Requirement-7: quality documentation internal to and contribute developers something well measure how scores karma like this that of show purpose to is the supposed study Thus are In behavior. they contributor because thing. alter off bit same to a the be not may are scores they fact, per- peers, are their contributions by one’s ceived valuable how and internal to quality contributions is between there connection While a certainly quality. of internal value for the contributions to their according have scores should the developers but to that reputation, aim not social do developers’ scores compute Karma measure and things. They scores different exactly. karma match partially to But have not degree. do some reputation to reality in tion meaningful? something measure scores karma question Do The is: valid. are artifacts process collaboratively software from created derived scores karma the whether question is unanswered an However, well-behaving. foster Purpose 7.4.1 Prause (2012). and al. (2012) et Eisenhauer and par- Prause is on section based This tially team. social development actual the the what in measure measure reputation not actually might they They that should. clear they not is it sistent, h eut fti td ontso htcomputed that show not do study this of results The influ- weights and algorithms authorship of choice The am crssol eettedvlpr’reputa- developers’ the reflect should scores Karma C ONT Scores bv ht thsbe eindwt robust- with designed been has it that, Above . A UTH sfiecom- fine is 167

VII — VALIDATION AND EVALUATION ). = for ) is 19 s . 80% r 88 88 . . = 0 s = 0 r , and = 0 s r 49 s . r = 0 s r for LinkSmart. These 68 . ) (see Figure 7.9). Actual repu- = 0 35 . s r = 0 s r of users), and eight ones for TornBond ( VALUATION for quality scores (see Figure 7.10). Both ones E 28% The Spearman correlation between reputation in- In Moknowpedia, a deeper investigation revealed that In TornBond, the correlations between a developer’s 53 . ≈ reputation and quantity score0 is are weaker than the correlationcombined between score. reputation and But quality has a more important role data while capturing only a small portion of his daily quired from the team members,computed and the by combined the score reputationMoknowpedia, system and is results are an evidencescores. for The combined the scores reflect validityin the of the actual peer opinion reputation group well or very well, respectively. the correlation between the quantityopers’ score contributions of and the their devel- reputationmuch ( higher than the correlationand between reputation quality ( score tation in the community is mostlytity dependent of on the contributions. quan- The reasonkind may of be contributions that to almost the any authors wiki of are wiki welcomed. articles Indeed, veryfor often their receive good articles. ratings Additionally, thereobservable is only but a positive very dependency weak oper’s between quality score a and devel- his quantity score ( ter ratings because they are more elaborate. 7.4.3 Data and results other proxies is presented in the respective subsections. 7.4.3.1 Computed karma vs. social reputation Fourteen interviews were conducted for Moknowpedia of developers). Several participants of Moknowpedia anymore (new job, moved to different work group), or they would not be ablefew to persons make were a in qualified both statement. teams A and interviewed twice. challenging. One person rejected tointerviews, participate mentioning in a both feeling ofto lack assess of others’ competence contributions. Hescores perceived as reputation something which would generate performance The data of the two sub-studies for social reputation and They reported to find rating for TornBond a bit more ( work, and he had a bad feeling about this. An explanation is that bigger articles tend to receive bet- were not interviewed because they were not available were known to have used the wiki very rarely so that ALIDATION AND VII — V earn them an according socialpeers, reputation and with their lead to high contribution inumentation. alternate forms of doc- There are three different karma scores compared for The second part of this study compares developers’ The first part addresses social reputation. It builds • Karma scores measure contributions to documenta- • each developer: The quantitycontributions, the score quality as score the asthe amount the quality weighted of of mean artifacts of he contributedscore to, which multiplies and a a developer’s combined qualityscores and quantity to obtain a single measure of contribution. karma scores to manifestations ofas other documentation proxies. Itsource correlates code with karma (a) scores thelog obtained quality messages of from in Subversion commit AgileLab, anderal (b) documentation contributions in to Moknowpedia. gen- members and to then rank themall from the best ranking to worst. lists for From of a project, each mean developer were ranking positions computedaverage to ranking obtain list. the This team’s listlabReview’s is ranking then list compared using to the Col- Spearmanlation. rank corre- to back them up. Socialinterviews reputation with is developers surveyed of through Moknowpedia andBond. Torn- The interviewees werecontributions, supposed in their to opinion, say had whose to added the the most project, value considering both quantity and quality. it means that karma scoresfore are has valid. to This show that studysocial karma there- scores reputation indeed and correlate with with mentation. contribution It to is other divided docu- into two according parts: on quantitative data analysis and inferential statistics. tion, which are manifestationstribute. of They willingness should hence totation correlate con- and with with social further repu- documentation. contribution If to positive other correlations are forms found, of then in documentation. It should 7.4.2 Method If developers differ in their willingnessdocumentation, to then contribute their to willingness should manifest They were asked to first rate the reputation of their team The results are combined with qualitative observations 168

VII — VALIDATION AND EVALUATION eoe’ uniysoei dutdmnal obetter to manually adjusted is score quantity veloper’s eetteaon fhscontributions. his of amount the reﬂect ( score correlations ( quality The score combined and this. reputation by between Mainly affected team. was the developer by recognized one still is computa- which reputation but in tions considered not was which code contributions. author’s an of the quantity between the relationship and observable quality no is There here. 10 10 20 29 1 3 8 1 h onodcd aecnan oe.E source .NET some contains base code TornBond The Computed Comptd. 1 1 0.25 = r 0.48 = r a ult euainranks reputation Quality (a) ranks reputation Quality (a) 3 iue79 cte lt fcmue am crsadsca euaini Moknowpedia in reputation social and scores karma computed of plots Scatter 7.9: Figure iue71:Satrposo optdkrasoe n oilrptto nTornBond in reputation social and scores karma computed of plots Scatter 7.10: Figure 10 r s Social Social 0 = . 60 20 r togr hnti de- this when stronger, are ) 8 10 29 I- aiiyo optdKraScores Karma Computed of Validity — VII-4 10 29 10 20 1 3 8 1

r Computed Comptd. 1 1 s b uniyrptto ranks reputation Quantity (b) ranks reputation Quantity (b) 0.88 = r 0.57 = r 0 = . 3 66 10 or ) Social Social ihMkopdacnrbtos h orlto re- correlation The contributions. Moknowpedia with hspr ftesuycreae am crscomputed scores karma correlates study the of part This uigteAiea n hi codn o messages log according their created and revisions AgileLab all the during data, message log commit essary proxy message log Commit 7.11. Figure in presented are sults code source TornBond from for scores and karma messages, correlating commit AgileLab AgileLab from with scores code source karma correlating data of for Availability other proxy. allowed to a as contributions documentation with of forms quality code source from proxy other vs. score Karma 7.4.3.2 20 8 10 29 10 10 20 29 1 3 8 Computed 1 Comptd. 1 1 c obndrptto ranks reputation Combined (c) c obndrptto ranks reputation Combined (c) 0.90 = r 0.68 = r 3 nodrt bantenec- the obtain to order In 10 Social Social 20 8 10 169 29

VII — VALIDATION AND EVALUATION , it still 05 . 10 0 ≮ 11 8 . was found. While the = 0 p signiﬁcance. Similarly, 41 . 05 . 0 = 0 s TornBond r p < 3 The developers of TornBond are a subset

r = 0.41

1 Moknowpedia 8 3 1 VALUATION 10 E The correlation for reputation in Moknowpedia is very the presented ranks were subjectteam before, to so discussions it in is the just probable repeat that scores interviewees did but not consideredpute. them in a critical dis- Wiki proxy of the users ofcompare Moknowpedia. their karma So scores it from was source code possible quality to positive correlation of correlation is not significant at seems probable that both are related. 7.4.3.3 Threats to validity styles) in TornBond is possibly not fully congruent with source code. Also, the developmentof teams a only consist few developers, whichteams. is But quite due to common the forare set’s agile small on cardinality, correlations the edgedifferences of between the presentedMoknowpedia correlations and within TornBond are hardlyboth significant teams. in strong. However, Moknowpedia includes someness aware- mechanisms. An influencetion on cannot the be users’ precluded,instructed percep- although to interviewees assign were ranks according toto their not opinion, consider and the presented ranking list. Furthermore, The automatic quality assessment (adherence to coding with their contributions to the Moknowpedia wiki. A what developers personally consider good quality of (b) TornBond quality karma vs. Moknowpedia combined score ALIDATION AND 10 between a devel- 8 VII — V 55 . . The result implies = 0 = 10 s , which is statistically sig- r 61 N . Karma for Figure 7.11: Scatter plots of quality karma scores versus other proxies = 0 s 05 3 r . 0 . From the remaining commit messages,

r = 0.61

p < IDE Commit 1 1 3 8 revisions were committed by other developers 10 (a) AgileLab quality karma vs. commit log quality 300 There is a correlation of during the second half of AgileLabthe were examined. correlation Here is nificant at that a developer’s disposition tomessages provide hints good at commit how likelyhis he source is code. to provide Javadoc in oper’s final quality scorecommit and messages the among amount all ofpossible his that non-bad messages. not It all developers maycommit were be message familiar concept with from the the start of the project. empty, “commit”, ...), okmuch information (some payload, effort e.g. “implemented madecepts”), the but or con- good not for messagesinformation. that According add to some this meaningful classification,mits 342 were com- considered as good, 83 as ok, and 46 as bad. not counted because they were submittedof within an seconds earlier commit withing an that identical the log developer message, just hint- my split experience, his it commit is into quite parts.to common In that usability this problems happens with due in the a Subversion major integration each one was classified as bad (completely useless, e.g. developers who were part≈ of the experiment (another Therefore, additionally only messages that were created were examined. 471 revisions were created by the 10 working on other sub-projects); some 60 revisions were 170

VII — VALIDATION AND EVALUATION eeiini agrsuiswt lae odtosis conditions cleaner with studies larger in repetition A per- to supposed not and significant, and strong were h ups fti td a ofidotwehrthe whether out find to was study this of purpose The niaoso otiuint documentation. with to correlated contribution of therefore internal indicators were to scores contribution measure Karma they quality. if i.e. valid, prototype CollabReview are the by computed scores karma Summary 7.4.4 interviewed. where be or could available, users was (software reviews) data and artifacts of envi- process amount only sufficient the were a it where But ronments factors. these to due optimal eial oipoecndnei h eut.However, results. the in confidence so. improve do to to desirable disposition their and quality internal to tion available. size sample due small rather the is however, to which, moderately significant, only not was and correlations strong but the of here One weaker there. were still Correlations by created developers. documentation these of forms other to compared different. bit a is team the in low, perception is the documentation although to contribution and her So score done, karma code. things source getting comprehensible and hardly hard produces but working as perceived peers is her lot, by a For contribute different. may developer be a may example, which in quality, team internal the of for terms contributions developer’s a of measure value they the Instead, reputation. social measure fectly Correlations developers. fellow their of contributions am crsapoiaetedvlpr’contribu- developers’ the approximate scores Karma • • and scores karma between correlations summary, In were scores karma developers’ the part, second the In the about interviewed were developers part, first the In sub- are study the for chosen environments The aiettosi te ouetto were documentation other in manifestations were reputation social – – – – rbtost Moknowpedia. to tributions ( moderate and AgileLab, in sages ( substantial ( strong very ( perfect near r s r r r 0 = s s s 0 = 0 = 0 = . 41 . . . 61 68 90 o infiat o con- for significant) not , o omtlgmes- log commit for ) nTrBn,and TornBond, in ) nMkopdaand Moknowpedia in ) I- nltclCnetEvaluation Concept Analytical — VII-5 okgop agrpnlfo eae eerhgroup, research related a from panel larger a group, work conferences. and workshops researchers, other with discussions included workshops iulrves(w otreaoyosinternational anonymous three to (two reviews vidual neprmnsi o osbedet ieconstraints, time to due possible not is experiments on based evaluation empirical allow where environments opinions to Expert access results. early obtain to communities was international that and feedback interdisciplinary on from based collected is concept the of Evaluation Purpose 7.5.1 development. at its of feedback stages expert different with evaluated was CollabReview Evaluation Concept Analytical 7.5 “Validity computations”). (Requirement-7: of support scores and karma of significant validity were the one fourth the correlations for all except studies, further without Even this during research. available not was environment appropriate an uiradsno utdsilnr cetssfo the from scientists multidisciplinary senior and with junior rounds discussion medium-sized four each), experts indi- paper-based software seven of collaboration, areas and the engineering in scientists international ten twenty with to workshops medium-sized five include cussed international at papers scientific on and based reviews discussions and department, the in colloquiums research The backgrounds. repeatedly different were with it audiences of to parts presented and concept whole con- the CollabReview cept, the evaluate analytically to order In Method 7.5.2 tech- on less and details. CollabReview, nical of aspects focused social mostly the workshops on the valida- requirements, the all to of contribute tion could opinions While the domains. principle other in about in learn applicability to and and feasibility concept, its the obtaining improve of to purpose was main feedback The research. this of frame through work experiments. ongoing the to risks potential or availability h vnsweeteClaRve ocp a dis- was concept CollabReview the where events The the in conducted were evaluations concept Several 171

VII — VALIDATION AND EVALUATION VALUATION E A potential problem identified in the concept evalu- Experts expressed the opinion that the concept was Potential for unfairness was recognized in the way Also on an individual level hazardous effects are pos- As a result of this, developers might try to attack the the bad points although itonly is approximates Alice’s scores. fault. Yet CollabReview ifbreak Alice source tends code, to she regularly will statistically end up with an attacks should not be too elaborate. ations was the rating biasability of is contributors. Understand- not anon objective human measure, perceptions”. “because it Athe contributor bases project who might is rate source newto code to skewed differently, scores. leading The usage of objective metrics like changed again” might be better.that Also, everyone experts thinks predicted that heeveryone wrote would good rate source himself code, so best.be But independent. reviews Therefore should arbiters were created. probably difficult to gethow to right, do and it has thatapproach not the yet was been best deemed found. way likely Athis, simple-minded to the present fail. research providespotential With a respect problems. telling to critique of Furthermore,that information the contained in experts review feared commentsdanger was of in getting overseen duekarma to scores, the which more do not influential reproduce the provided text. how responsibility is computed. Oneone expert developer noted (Alice) breaks that the if source code another one there is no perceivedone’s loss peer group, or since gain the of participantseach are standing other.” unknown among to If theyinfluence are each known others to scores, each however,different. the other Other situation experts and is feared can thatcur, a in rat which race developers might get oc- quality exhausted by ratings. chasing top Laterinvestigated experiments for will the occurrence be of cautiously able rat consequences races, (Requirement-4: and undesir- “No hazardteam”). to the sible. One expert feareduse that a the “smart results manager to would situation, fire CollabReview the would be worst perceived developer.” asand In a face threat such extreme opposition. a reputation system, e.g. through theor responsibility the metric quality metric.some So defense CollabReview mechanisms. needs But to as have is long as not CollabReview associated with too high potential win or loss, (Bob) with just a few changes, then Bob would get all “bugs in file”, “time to fix a bug” or “time until code is ALIDATION AND VII — V The experts saw the possibility of hazardous social Integration into the development environment was tive is missing. A psychologisttry recommended it! to Reactions “Just [of people]situation can to be situation.” very different from effects. One expert pointed outnities that it in is online usually “commu- clear how the reputation score works; they know that they will havecode to forever. In work such with a their situation, source own a interest developer in has his keeping very sourceproblem code is in good that shape. this is The because not it always creates possible high and dependence wanted on the individual de- suitable for environments where the long-term perspec- the “approach will be acceptedder by the a pressure software of team real un- industry development experts conditions”. shared Many the viewnot that suited CollabReview was for commercial development (Prausedik, and 2012). Dur- One expert proposedopers that to motivating create devel- high quality source code by making sure seen as a potential weakness.labReview would It make was sense doubted in that combinationmal Col- with reviews. for- However, ifits support CollabReview as emphasized a “post-review”argued approach, that both other could experts co-exist. CollabReviewbe might equally not well suited forenvironments. commercial and It open may source be too optimistic to assume that calculating personal reputation scores”. Anotherstated expert that attempts with ratings were already made 30 expected that there would beand some that resistance many to would its not use, feel comfortable with it. between the front lines ofexpert’s two opinion opposing best factions. summarizes One this,that when the he concept noted wasgenerate dangerous some attention”. “but Especially it the will karma scores probably it “may be possible to make something of it without Discussions of the concept tendedamong to the create experts controversy that reviewed it.discussions arose Several with times the heated presenter being in the middle countless individual reviews with close colleaguesinternational and experts, and a series of almostat 40 a inter- major software engineering event. 7.5.3 Results velopers. Still, the CollabReview concept may be more years ago and that it “did not work out formerly”. It was views led with international practitioners and researchers were subject to criticism, so it was recommended that 172

VII — VALIDATION AND EVALUATION a on ob qieinvtv” n xetexplic- expert One innovative”. “quite be to found was .. Summary 7.5.4 to want I finished, it’s it.” “When have discussions: after the said company of software one Russian example, a For effective.” of potentially manager new is the is and clever, least, is at tool me quality to a as of score idea reputation “The a explained: using expert as another quality and improve strength, to a reputation at looking named itly in concept 40 CollabReview of The out 2012). 34 Durdik, and (e.g. Prause quality source improve documentation fact and in code can they system reputation that a expressed that experts think of majority the hemently, is she because Bob code. broken of in that involved always than worse rating overall fteClaRve prototype. feasibility CollabReview the the addressed of and concept, hints the improving provided for also feedback develop- The software environments. different ment for suited is well concept equally CollabReview the not But that well. agree as to projects seem software experts of trans- kinds are other results to obtained ferable the that indicate can described This experiments later. field the in eventuated actually work. can it that thought and out, tried ing be- worthwhile CollabReview considered experts most can it because difficult be work might real environments into dis- CollabReview heated bringing into While got cussions. often disputants and Proponents controversial. were concept CollabReview the of Receptions • h al ocpuleauto eeldise that issues revealed evaluation conceptual early The • • • ve- opinion their expressed often opponents While eprevda unfair, as perceived be and reviews, of non-objectivity from suffer system, reputation its against attacks to exposed be on effects hazardous have – – niiullevels, individual and group I- onwei il Experiment Field Moknowpedia — VII-6 h rvossuisvldtdol eti set fthe of aspects certain only validated studies previous The experiment first the was experiment Moknowpedia The h edeprmn rnsteClaRve prototype CollabReview the brings experiment field The fMkopdaehne ihCollabReview with enhanced clarifies Moknowpedia and matter, if this Moknowpe- addresses The experiment before. field studied dia as been team not the have on whole prototype a the of effects The computed scores. of karma validity the like prototype CollabReview Purpose 7.6.1 for changed been have subjects Sec- anonymity. of in names described As 7.1.3, (2011). tion al. et Dencheva mostly on is description based Its users. real with CollabReview of Experi- Field Moknowpedia 7.6 leta h oto nyasalnme fpoad and probands of number significance. small statistical a limited only of level, cost personal the a at albeit on mechanisms the reputation of the impression of good impact a obtain should It and data aspects. qualitative ex- human on the relies of data and focus explorative, gathers The is and periment months. several use, of productive period in the is over that wiki a into Method 7.6.2 • • • • • • Fwoeha o users”), for overhead “Few Rqieet1 Ipoeitra otaequal- software internal “Improve (Requirement-1: ae rblyn Rqieet4 N aadto hazard team”). “No the rat (Requirement-4: bullying like or phenomena races destructive cause not avoids and motivation”), individuals’ to “Ap- peal (Requirement-2: contribution force not does (Requirement-8: low operation of cost the keeps dependence”), expert without contributors unique “Involve (Requirement-6: the everyone of preserve knowledge to deniers involves ity”), artifacts process software of quality the improves documentation, internal of amount the increases ment 173

VII — VALIDATION AND EVALUATION , 2010. th 18 is no longer mentioned in official HotW Moknowpedia and acceptance VALUATION E Finally, data regarding use of the review tool was Even after the test phase is over, Moknowpedia’s re- The number of new article revisions created during In addition to an increased number of contributions, statements to which users shouldof express agreement on their a degree Likert scale: “fully agree”, “agree”, concerned the effect that reviewing extensions had on the MediaWiki counted 176 page views inbefore, the and four 258 months views duringincrement the of test 47%. phase. The This number isthan of an that page of views changes is because lower page MediaWiki does views not that count recur inarticle a does short not period. necessarily increment So,every the editing time. read an counter surveyed. After the test phase,the there database. were 237 155 reviews reviews in (moreadditionally than contained half sensible of comments. all In reviews) total,reviews these were submitted by 16user different users. who had Only participated one fornot the submitted test’s a full single duration review. had ing added continuously. However, no moreawarded and chocolate the is meetings. 7.6.3.2 Quantitative interview results: Effects on 7.6.3.1 Measured results By analyzing the data infour databases and months server before logs, the the revieware extensions compared were to the added fourof months the of test the period test was period. February Start the measured period raised froman 320 increase of to about 517, 62%. whichspan As is Christmas the time, four the pre-test test months to period was the also same compared period in 2009. Hereaverage number 363 of new characters revisions perdia article grew in from Moknowpe- 5324 toin 6134. size. So, I articles the tended same tocharacters period grow per one article year was earlier, 5703, theduring which number the is of test. also less than also an increased number of article views was recorded. view facilities keep being used, and new reviews are be- The pre- and post-test questionnaires contained twelve were created, which still means an increase by 42%. The “neutral”, “disagree”, “absolutely disagree”. Items 1 to 8 ALIDATION AND VII — V The target user group in the field experiment are self- The test phase itself — between the two interviews — It was not feasible to divide the whole group of users 7.6.3 Data and results and 16 members, respectively.personnel, only Due 15 employees to were fluctuationfor able the in to full participate test period. knowledge. organizing project teams at ahierarchies. research institute Individuals with have far-reaching flat freedomautonomy and in organizing their work.or Explicit punishment instruction are, at most, rare. Intrinsic motivation is one third working students, and two executive managers. data was collected, andmonths compared right to before data the ofof test. about the Although four four the monthsresults test of might productive phase still use be was influencedholidays. long, by seasonal the events like swers. Before and after theinterviewed with test questionnaires. period, the users were ran for four months. Duringobservations this regarding time, the any modified incidents Moknowpedia and were recorded. In addition to the interviews, actual usage terviews could then be compared.know Yet that probands there did would not be asocial second desirability interview. bias A would possible both then interviews be and in cancel effect itself during time out. of Additionally, the several months betweenprobably the too two long interviews was for probands to remember earlier an- interviewed and filled out questionnaires.capture This the was initial to spectrum of opinions as a baseimproved, line. and whether the reviewingaccepted, extensions and were to gather future design recommendations. phase with similar questionnaires. The results of both in- into two groups. A within-group designMoknowpedia, experiment all members of the work groups were valued very much. Staff members are mostly researchers, The vast majority has a computer science background The goal of the questionnaires was to survey if contents At the start and end of the test, the work group had 18 with a few exceptions that still have good computer A second round of interviews was executed after the test was therefore devised. Before adding CollabReview to 174

VII — VALIDATION AND EVALUATION Number of answers Number of answers Number of answers Number of answers 10 11 12 10 11 12 10 11 12 10 11 12 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 tm4 ril interestingness Article 4: Item tm1 vrl attractiveness Overall 1: Item iue71:Rslso onwei r-adps-vlainitriw(eceae l,2011) al., et (Dencheva interview post-evaluation and pre- Moknowpedia of Results 7.12: Figure tm7 ril diversity Article 7: Item ++ ++ ++ ++ tm1:Mkssense Makes 10: Item + + + + 0 0 0 0 2 1 2 1 2 1 2 1 nd st nd st nd st nd st interview interview interview interview - - - - interview interview interview interview — — — — I- onwei il Experiment Field Moknowpedia — VII-6 10 11 12 10 11 12 10 11 12 10 11 12 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 tm8 onwei value Moknowpedia 8: Item tm1:Ivs eiwtime review Invest 11: Item tm5 ril importance Article 5: Item tm2 ril volume Article 2: Item ++ ++ ++ ++ + + + + 0 0 0 0 2 1 2 1 2 1 2 1 nd st nd st nd st nd st interview interview interview interview - - - - interview interview interview interview — — — — 10 11 12 10 11 12 10 11 12 10 11 12 tm6 ril understandability Article 6: Item 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 tm9 eadmotivates Reward 9: Item epneba indicator bias Response ++ ++ ++ ++ tm3 ril count Article 3: Item + + + + 0 0 0 0 2 1 2 1 2 1 2 1 nd nd nd st nd st st st interview interview interview interview - - - - interview interview interview interview — — — — 175

VII — VALIDATION AND EVALUATION ). The Gina went to one of the adminis- Lou, who was registered in Moknow- During the evaluation, Izzy became Hero Trudy had missed the meeting in which the VALUATION E other users. They complained thatsomething there wrong must with have Moknowpedia. been As aof consequence this, Izzy deliberately choseothers to a contribute chance less to to earn give the title, too. curious and went to hercould colleagues try to the ask extensions. them As howinstalled she the in extensions Moknowpedia, were she already could start using them Missing comment trators of Moknowpedia and complainedgave that her somebody article a badcomment. review The administrators rating themselves without could writinganything not about a do it, but after this incident, Moknowpedia reviews. Yet there wereone. no further incidents like this First activity pedia but had never written anregularly article write before, and started review to articles.he made Within it one from week, zerorankings. points He to stated the that fifth thestart position extensions contributing in motivated to him the Moknowpedia. to Frustration of the Week several times in a row. This frustrated of a reward systemintention into was the that wiki it could ismore reveal interesting. a positive feedback bias in towards giving theserved second interview. only We a ob- smallpoints, change +0.2), in so feedback it behavior seemsability (3.7 that bias, there thus is supporting few that social observedin improvements desir- the other items are real. 7.6.3.3 Anecdotes integrated into theprobands, natural and could work observe environment first-handduring what of the happened test the period. Thisof section anecdotal presents stories a from collection the test. Curiosity Moknowpedia extensions were introduced. She was The experimenters and designated confederates were without problems. was updated to allow rejecting of seemingly unjustified ): = 06 . 11 0 − Item ALIDATION AND r = 10 The integration Item VII — V r , 14 . = 0 9 Item r for Item 11 ( The last depicted item was mixed into the statements An interesting question was if acceptance of the re- Four statements regard the reviewing extensions them- The answers substantiate that Moknowpedia and its Assigning one (“absolutely disagree”) to five points 25 . 0 correlations is significant. Itrelation could between be profiting that from there theand is advocating review them. no extensions as an indicator of a possibly changing response bias. It But there were only weakbetween Spearman the rank items correlations 9 toranking 11, list. and the The user’s position strongest in correlation the was Profiteers might be more willingthan to do others. reviewing work Aing more review probable work explanation leads to is profiteering. that Yet do- none of the agreeing users grew from fourusers to decreased ten, from six and to disagreeing of two. users Although agreed the from majority the startsense, that after reviewing the would test make they even more agreed (Item 10). so that afterward a slightreviewing extensions. majority was However, the in statement favor9 of of was Item the still the oneSimilarly, confronted most with the users most initially objection. did not thinkhas that clearly they changed towards acceptance: the number of before. Many users that were previouslyagreed undecided, to later positive statements. selves (see Table 7.4). Before theusers test, that there were were not several sure of whether rewards would moti- to agreement. Also, the number of skeptics decreased tween first and second interview — are listed in Table contents improved during the test. In the second inter- Moknowpedia in only seven cases, as compared to 22 agreement score is computed. Thefirst average and agreement second of interview canhow then agreement be has compared changed. to see Thestatements, texts translated of the into different English, andagreement the scores associated — absolute and relative change be- reviewing extensions themselves. Detailed agreementpresented is in Figure 7.12. viewing system was especially high among profiteers. vate them (Item 9). Through the test, these users changed 7.4. view, users disagreed with the positive statements about (“fully agree”) to each user’s agreement, an average was intentionally formulated diffusely ( would invest time in writing reviews (Item 11). This Wiki contents. The remaining four items related to the 176

VII — VALIDATION AND EVALUATION a detdt h epaeta srcmeddfor recommended is that template the to adverted was Improvement users. to attraction certain a that. have like with — associated merit not directly positive did not a She if even — instead). positions review Unique to bad article back a her get her rated to who give reviewer do the could making So (unless she points reviews. that the nothing for points was then, get there But would she back. told, position was old she her get her to to rate came down could articles she she own if So asked and review. office experimenter’s good an a older received her had of had one articles she because because but not recently, something points; done of number a however, the day, received in One she one points. last zero the with being ranking of with reputation proud her was leaving She review, points. a zero none received time, had long articles a her for of and review a to. submitted requested never been She explicitly had she what contributing Anti-hero com- review valuable ments. contained and points, fair the were get reviews Al- his to primarily colleague. articles that reviewed rank- of he the ahead though on remain eye to an watching kept he ings, weeks, col- next said overtaken the For again was had reviewedleague. points he Larry until some hours, articles few get of a dozens to Within way reviews. easy write to an him that gave advice They points. the he more how some on gather hints quickly for could went them Larry asking rankings, colleagues, several the to in him overtaken had league competition Playful al .:Ue gemn oiesi onwei otts usinar Dnhv ta. 2011) al., et (Dencheva questionnaire post-test Moknowpedia in items to agreement User 7.4: Table ati a enamnrcnrbtr only contributor, minor a been had Caitlin 11 10 hog eiwcmet Martin comment, review a Through 9 8 7 6 5 4 3 2 1 # Statement tmkssnet nettm nrveigadcommenting and reviewing in time valuable invest is to articles sense of makes quality It the actively rate more to wiki mechanism the A use to me motivates reward A Moknowpedia using worth is It diverse find are to topics easy treated and The comprehensible is information information core useful Article and important contains wiki read The to interesting are satisfactory articles is The articles of number substantial total are The wiki the in articles The attractive generally is Moknowpedia nie ytefc htacol- a that fact the by Incited I- onwei il Experiment Field Moknowpedia — VII-6 0 ie nuhicnie—atcsaanttesystem the against attacks — incentive enough given — go” lhuhh meitl eevdlt fpoints, of lots received immediately he Although “good”. adn ehnsswr o ecie svr effec- very as perceived not were mechanisms warding rmterveswshlflt i n o improving for and him to Feedback helpful page. was profile reviews and his the profile change from to the him about help colleagues them his had be- asked template he the used so not fore, had He profiles. user creating iea oiaigues nteaeae hyalwere all they The average, weakly. the rejected On users. motivating at tive re- three All mechanisms. rewarding the state- additional about three ments were there interview, second the In recommendations design Future 7.6.3.4 advertising. viral of form a to leading other, each Moknowpedia tease used to users and The better rating. a appropriate to review more the a changed Yet Rico ‘poor’!” later minutes minutes few meeting last your “Hey rated exclaimed: I Sonny, jokingly and laughed he hallway, the Teasing an such attempt would one anymore. attack no The that attack. negotiated the through group created were that in database reviews the all deleted Administrators probable. become that shows incident authors The articles’ him. the than because points title more the received win not did still he articles all rated that the script hacked a Stan writing Week, by the Moknowpedia of Hero become and points of Manipulation quality. article’s the rmhsofie ioswSnya h n of end the at Sonny saw Rico office, his From oqikygi osdrbeamount considerable a gain quickly To akn list ranking Pts. 3.6 3.9 3.0 4.1 4.0 3.9 4.5 3.5 3.3 3.6 4.0 Chng. a ecie smost as perceived was +0.80 +0.53 +0.47 +0.47 +0.60 +0.53 +0.20 +0.60 +0.27 +0.53 +0.73 177

VII — VALIDATION AND EVALUATION VALUATION E The evaluation shows that CollabReview increased The CollabReview extension also drew more users to The additional cost of operation through CollabRe- is no threat or punishment involved.no Similarly, record there of was events that wouldundesirable suggest phenomena the like existence rat of races,stead, the or reputation bullying. system In- was perceived as fun. the volume of contentsthat in the Moknowpedia. number of Userscontent articles size feel increased. was measurable, too. Anfor The increase quality, same in where holds true users thinkunderstandability that increased. interestingness Additionally, reviews and pro- quality. The Moknowpedia wiki becamefor more users attractive through this. Moknowpedia. Except for a veryproved few the users review concept that outright disap- (onequite of active them contributor was of a articles),that there did was not no at user leastone user contribute admitted some to reviews. have startedof At contributing the least because rewarding system. Althoughto it achieve was active not contribution possible byceptions, everyone CollabReview without was ex- successful attional drawing addi- users to Moknowpedia. computing power, and a small reward for the Hero of the users what to do. No one is forced to comply and there initial quality assessments. The valuebe of increased rewards (e.g. should an additionalcertain day amount off), of and reputation getting points a compulsory. per Even week punishment should like be payingthank-you money box into was a proposed. Anotheruser wish groups was so to that have teamsother. could compete against each 7.6.4 Discussion and threats to validity time to make their contributions and reviews, or even CollabReview takes a different approach. Itif assumes users that have enough motivation tothemselves contribute, will then find they the timeintrinsic to motivation do would so. be Although preferable, purely CollabReview has some extrinsically motivating effects thatdesired lead effects. to the vide confidence in article content, further adding to their view is negligible. All that was needed was a little more The users originally expressed the wish to have more wanted to be called upon to do it by their managers. Week. By its design, CollabReview does not prescribe ALIDATION AND VII — V (2.7 points), and finally the level Hero of the Week Wishes for future improvements were to harden Mo- One user testified that the rewards had motivated him Positive remarks were that the design was pleasant, A few users said that they would not feel motivated The questionnaire invited users to provide further re- they also said that theabout rewards the made wiki. them think more knowpedia against manipulation, to providehints a how user to with earn hisrespond next points, to a way reviews, to and immediately to have computer-generated and answered that itnoted had that the changed motivation their wasnot not behavior, sustainable, withstand but and other could pressing tasks. The lackarticles as of well time as notthey reviewing. would A not few users care said forusing that the the rewards wiki and, because in they fact, were were convinced by it, but to start contributing to Moknowpedia.more Others browsed vigilantly to bearticles more able carefully. to A give majorsaid that reviews, contributor, he however, or had reduced wrote a his chance. contributions to A give couple others of users felt appealed by the rewards tend to more articles. Reviewsof also trust. gave them But a also feeling theappreciated. anonymity of Several writing users reviews was valuedtheir that own activity, they or could compare see Some it users to noted the a others’ positive activity. mechanisms, fun factor which in they regarded the as rewarding entertaining. geted people. and the integration fitted wellpedia. into It the was existing simple Moknow- to access,stand. and The easy review to comments use were andful regarded under- because as they very motivated use- and helpedarticles. with Users improving felt that the reviews made them read and considered reviewing articles as distracting andof a time that waste they couldSome rather spend users on bore writing concern articles. thatmanipulated the and be extensions attacked could systematically be to harm tar- marks of what had orthey would had have liked motivated them, or what disliked,ior. and if The it major changed design their recommendationsbelow: behav- are presented by the rewards in any way. Instead, they responded it effective rewarding mechanism (2.9 points), followedthe by hierarchy (2.6 points). They had a distrust against the reviewing system, and was given as the major reason for not contributing to would have required much larger rewards, or coercion. 178

VII — VALIDATION AND EVALUATION ihsc ytm h ou a nqaiaiefeed- qualitative on was focus The system. a such with groups. work two with months several for evaluated use was its Moknowpedia, in CollabReview installing After have might results then scientists, computer not were high a attain to invest will users efforts more the worth, Rqieet2 Apa oidvdas motivation”). individuals’ to “Appeal (Requirement-2: h ra lae nta er httesse ih be might system the that fears initial allayed trial The experiences first gather to was evaluation the of goal The niomn eemr opttv,o ftemajority the if or competitive, built the more if were not motivation, environment were intrinsic groups and work cooperation friendly the on if re- different example, or very For to environment sults. lead the could in themselves, factor people the minor unpredictable. a be only to tend Changing that beings human involved the like effects destructive races. risking rat of cost positive the greater at to but effects lead will rewards on higher relying or example, coercion For 3.5.2.2). Section (see reputation is reputation more The individual. the for reputation of aeterfis otiuin eas fCollabReview of because contributions even first users their made Some “Involve dependence”). (Requirement-6: them without for contributors care or was use user to no however, forced ex- them; using CollabReview enjoyed the and accepted tensions users Most field quality”). the software after internal scale “Improve Likert (Requirement-1: 5-point experiment a about on as better perceived point was a content half the in- size and average 20%, their by 60%, creased to 40% new by of increased number articles the increased: process Moknowpedia software in the artifacts of contri- quantity more and made quality and The but articles, butions. often, more more read wiki also have the they accessed users the have only too. collected, were numbers with usage interviews Hard and field users. the in observations from back Summary 7.6.5 due effect. not the other is some test, that to the precluded during observed fully behavior be in Col- change cannot to it due Yet probably is time labReview. had this worked contribution it improve That to failed. exper- attempts the had Several before behavior stabilize and Usage iment. establish to old. time years its had two Moknow- about the was started, test pedia the when Also, different. been l nal h vlainrslswr rmsn.Not promising. were results evaluation the all, in All to due complex highly are systems Socio-technical value subjective the to related closely are effects All I- glLbFedExperiment Field AgileLab — VII-7 hl h xeietfie oaheetedsrdout- desired the achieve to failed experiment the While nopruiyfrsuyn olbeiwi software a in CollabReview studying for opportunity An not was negative, or positive either effect, verifiable A Pas ta. 02.Nt htalpeetdnmsare names presented all that Note 2012). al., et (Prause h glLbwstefis edeprmn eln with dealing experiment field first the was AgileLab The h ups ftefil xeietwst r CollabRe- try to was experiment field the of purpose The iwcnet h vlainifrstedsg fsimi- of design the informs evaluation the concept, view aiaetesm eurmnsa nteMoknowpedia the to in and as code, requirements source same of the validate quality the improving for view mrvn orecd ult nasfwr project. software a in quality code source improving Experiment Field AgileLab 7.7 interviews. final improvement the for through hints gathered by complemented the are during These occurred that test. processes vivid social a the provide of stories picture anecdotal The systems. future lar for overhead users”). “Few (Requirement-8: obtained be could contributions improvements considerable value reputation, exchange for small a offering and Moknowpedia, into the prototype CollabReview effort the little integrating a in with invested that showed team”). experiment the the to to- Instead, hazard with “No confronted be (Requirement-4: or rejection bullying, tal for tool a as misused eeomn rjc pndu ihteAiea.The AgileLab. the with up opened project development Method 7.7.2 systems. similar and experiments future of the design informs information This effects CollabReview. and of interplay valuable socio-technical teaches the It about failing. lessons for reasons the about mation found. failed. experiment the However, users”. for “Few overhead Requirement-8: and dependence”, contrib- without “Involve utors Requirement-6: team”, the “No to hazard Requirement-4: motivation”, individuals’ to “Ap- peal Requirement-2: “Im- quality”, software Requirement-1: internal primarily prove i.e. experiment, field Purpose 7.7.1 7.1.3. Section in explained as pseudonyms on based is description This critical learned. aspects were to success regard for with lessons valuable come, eie hwn noealsceso h CollabRe- the of success overall an showing Besides nta,teeprmn a dpe odlvrinfor- deliver to adapted was experiment the Instead, 179

VII — VALIDATION AND EVALUATION ), 43 . 0 − = r well anyways. The API ) and end ( 70 . 0 − = r the experiment VALUATION E scores for you? understandable — How wellthe did way you reputation understand scores were computed? present — How present werein the your reputation consciousness? scores fair — How fair were reputation scores suited to important — How important were the reputation your work? • • Although the lab was graded, students were instructed • • activities when documenting. 7.7.2.2 Postmortem interviews after completion of not have the intended effects,designed. a structured It interview would was be donecompletion with of the the experiment. students The afteron interview the was a based questionnaire, andture design aimed of at similar interventions. learningcontained The for Likert questionnaire items the for fu- quantitativelyimportance capturing the of certain factors,questions and to additional qualitatively free investigate text deeperhappened. into The what questionnaire asked about eight human tervention was certain functionality to a hardondary deadline, goal (supported while through the CollabReview) sec- wasobtain to high quality sourceconflicting code. to some These degree two in goalsing the are quality short can term temporarily as neglect- speedSection up development 2.2.2.2). (see In particular,tion a quality developer’s and contribu- quantity wereintervention negatively start correlated at ( respectively, i.e. more quality means less quantity and that neither their quality karmakarma scores nor scores their as quantity determinedaffect their by lab CollabReview grades. would Consequently,quality even if would improving cost developersnot their affect their time, grade this inDevelopers were neither would free a to good invest nor (almost)time any a into amount bad writing of way. Javadocexpected comments. to document And the public theyonly were trade off between implementing and documenting vice versa. was that of having less time for other implementation When it became foreseeable that the intervention would “feelings” about CollabReview to learn whether the in- control ALIDATION AND VII — V at the end of each day to improve phase. First, comparison data was 8 The day when the intervention started was experimental Standup meetings are used in agile development to discuss current 8 There were two partly conflicting goals set for the CollabReview then sent all developers a daily email The arbitrary re-combination of developers into pro- the standup meeting project: the primary goal was to deliver software with issues and to coordinate work between team members. of all developers (including theirrepeated individual scores), a and short explanationimprove of how their developers quality could ratingsaddition by to the writing published Javadoc. ranking list, a In 30EUR Amazon lab. A short information on the top scores was given in informed all developers that theyexperiment. were taking It part explained in that an personalon scores Javadoc depended in their source code, and that measurements in the late afternoon. This email contained a ranking list measure the effect of CollabReview. CollabReview and thebased on it experimental intervention group running in parallel.lowed Instead, a the within-group experiment design fol- andand was an split into a collected during the earlier phaseknowing. without the In developers the secondproject phase, duration, after the about intervention halfmental started. of phase the lasted The for experi- 8 working days and served to karma scores served for data gathering asadditionally well, be but published would to developers tobehavior. influence their gramming pairs and their smallsible number to made split it them infea- into a control and an experimental During the lab, CollabReviewquantity karma computed scores of quality developers. and Thescores quantity served karma for data gathering purposesnot only, disclosed and to were participants of the experiment. Quality study started as a fieldpostmortem experiment analysis but of later the turned experiment intostand to a what better had under- happened. 7.7.2.1 The field experiment part of the study visibility. voucher was announced as a prize for the developer who would have the highest karma score at the end of the were not totally accurate but reflected trends. 180 “Day 0”. It was signified by an introductory email, which

VII — VALIDATION AND EVALUATION eecmue.Drn h okn or,teteaching the hours, working the During computed. were h gr trsa a 1,wihwstefis a of day first the was which -11, day at starts figure The quantity of history the of consists data collected The uigtediysadpmeig,sotyatrthe ex- after students the shortly begun, had meetings, intervention CollabReview standup daily the experiment During the of conduction the on Notes 7.7.2.3 oe a ,wihmrstesato h nevnin is intervention, the of start the marks which 0, Day fore, developed, increase. is contributions code developers’ more the any and contain more not As does figure results: to the surprising commit general, first In their base. making code early the when the project during the of appear days developers More project. the time. over evolved developers individual ( by code source contributed (mea- of lines code non-empty source hecto of in amount sured the how shows 7.13 Figure scores karma quantity of History 7.7.3.1 the of results and questionnaire. scores, karma quality scores, karma Data 7.7.3 when fortunate be to results. out the turned interpreting later cases. both mishap in the Millhouse fact, was In who prize, affect the of not winner did the the mishap in this published however, Luckily, one email. the daily than rank had different users Some a actually email. via published being scores karma computed. were the they about how questions or explicit the scores, many Still ask scores. not did the students on murmur further observed staff scores karma their how about confusion some pressed h gr osntcutdy nweed.There- weekends. on days count not does figure The wrong to led algorithm computation rank the in bug A • • • • o you? scores for reputation are acceptable How — acceptable situation? scores lab reputation the think into you fitted do well How — fitting that scores? you reputation for were it there was interesting How — interesting all? reputation at that computed like were you scores did much How — likable I- glLbFedExperiment Field AgileLab — VII-7 HSLOC )) egtwe htdvlprhscnrbtdmr code. more contributed has developer that when weight h vrg cr steeoeeuvln otecode the to equivalent therefore is score average The eoes uniy ..advlprssoehsahigher a has score developer’s a i.e. quantity, velopers’ oe hl ult am crswr optdfrom computed were scores karma quality source that few while is only code, reason contributed The had developers down. the and initially up jumped beginning, scores the these in Especially time. over for evolved scores quality karma It individual study. developers’ the the to how regard shows with important more is 7.14 Figure scores karma quality of History 7.7.3.2 by edited or removed later developers. is other which code base, of code amount the phe- huge into This a Sanjay. copying from for results level nomenon normal a to decrease rapid base. code the in code source of amount total analo- the is to which developers, gous the the of average score after the karma weeks shows quantity half line a dashed The and one started. The intervention 7, project. day the at into ended weeks two project than more little a after feto nraigtecd ult sntsupported not is quality code the intended The increasing quality. of code effect on influence small 0) a day at only (starting had intervention the that shows 7.14 Figure Discussion and Results 7.7.4 problematic. less (2.67) were likability (2.89) presence while and areas, problematic most the (2.11) are understandability (2.44), Fitting especially and outright: rejected (2.33) of not fairness reception is overall it that The but shows improvement also 5). for = potential reveals high intervention very the ..., 1, (very = 7.5 Table low in presented are responses Likert-scaled results questionnaire Quantitative 7.7.3.3 effect. less have going will code change source single a more average, with the that into too, simple here, The is score. reason karma to quality leads average also stabilizing base a code larger A quality. overall base’s de- their by weighted scores with developers all of score average. have the will on line influence each less code increases, source developer of a amount by the authored When quality. average the hr sasde nraet ihpa n hna then and peak high a to increase sudden a is There h ahdln hw h vrg ult karma quality average the shows line dashed The 181

VII — VALIDATION AND EVALUATION 7 7 6 6 5 5 Bart Millhouse Bart Millhouse 4 4 3 3 2 2 1 1 Ned Rainier Ned Rainier 14 9 day 0 day 0 VALUATION -1 -1 E -2 -2 Quality rating -3 -3 Lenny Mel Lenny Mel Quantity in HSLOC -4 -4 ALIDATION AND -5 -5 VII — V -6 -6 -7 -7 Krusty Seymour Krusty Seymour -8 -8 -9 -9 Figure 7.13: Amount of source code contributed by the developers (quantity karma scores) -10 -10 Figure 7.14: Average quality of source code written by each developer (quality karma scores) -11 -11 Kent Kent Sanjay Sanjay 182

VII — VALIDATION AND EVALUATION ( h ocuini htteitreto i o aean have not did intervention the that is conclusion The o ffntoaiyt opeealtertssbfr the before tasks their all complete to functionality of vor ult am crsa h n fteeauto period evaluation the of end the at scores karma quality the non-negligible. that chances are de- the not that but do mean exist not necessarily do not correlations small does tected the That to due size. probably sample is which significant, correlations not Most are correlations. rank Spearman coefficients denote correlation all otherwise, stated Unless quality and opinions between Relations 7.7.4.1 interpretation speculative. this remains But happening. have from then that could prevented intervention Our delivered. is software fa- in quality that sacrificed be have may normally It would developers deadline. con- end-of-project be might an scores by quality founded significant that any argue show could not One did effect. just It too. effect, adverse peak. quality decrease a small with a coincided actually 0 day is because there quality fact, in In data. the by r s h vrg esnloiin orlt ekywith weakly correlate opinions personal average The ( Q l end al .:Lkr-cldoiin bu h nevninadcreainwt cr ak nAgileLab in ranks score with correlation and intervention the about opinions Likert-scaled 7.5: Table = ) am scores karma . 25 Average r r r r r Seymour Sanjay Reinier Ned Millhouse Mel Lenny Krusty Kent Bart s s s s s hl nyatiiln orlto was correlation trivial/no a only while ) ( ( ( ( ( Q Q Q Q mail t start t end l start l end ) ) ) ) ) -0.40 -0.50 0.20 0.28 0.53 2.89 - - + + + 0 0 0 - - present -0.14 -0.42 -0.01 -0.05 0.04 2.33 I- glLbFedExperiment Field AgileLab — VII-7 - - - - + 0 0 0 - - - fair -0.04 -0.01 -0.23 0.41 0.44 2.56 - - - - + + 0 0 0 - - important n edakreceived) feedback (no r -0.67 -0.13 0.20 0.28 0.36 2.11 s ------+ 0 0 - - - under- ihrnspbihdtruheal( email through published ranks with eto rbbyhdhdsm fet h relationship The effect. some had had probably vention ontv isnneter a xli hsobserva- this explain can karma. theory of presence dissonance perceived Cognitive low low to a leads having rank that quality account Instead, into this. take support should not designs does future data the pub- Yet actually scores. the lished of irregardless quality, in investing to stronger even or strong similarly correlation a show should ranks pres- trial correct perceived the The stronger a karma.) to of being leads ence (that ranks around the way other in the higher is reason- it following that the suggests But ing code. there- quality and higher themselves, write ranks fore achiev- and in scores interested karma more high are ing presence high a has karma ( scores . karma of presence perceived the and of reception quality. in overall investing better more a to that leads karma effect reception, the better to a also to leads but profiteering that effect attributed be the might to strength increased This stronger. bit a inter- the that indicates It start. the at scores with found

53 standable hr sasrn eainhpbtenpbihdranks published between relationship strong a is There .Oeepaaincudb httoewofe that feel who those that be could explanation One ). -0.03 -0.37 0.52 0.49 0.58 2.67 - - + 0 0 0 0 0 - - likable r -0.39 0.06 0.18 0.27 2.56 0.01 s - - - - + 0 0 0 0 0 ( -

Q interesting l end -0.11 -0.02 0 = ) 0.28 0.38 0.41 2.44 - - - - + 0 0 0 0 - - ﬁtting . 28 -0.41 -0.15 -0.10 0.13 2.56 0.06 - - - - ≯ + 0 0 0 0 0 - acceptable 0 . 53 Q r -.53 -.18 .08 .25 .35 2.5 2.8 2.3 1.3 2.9 3.1 2.9 3.0 2.5 2.0 fpeec leads presence if , s l end (

mail Average tteedof end the at r s = ) ( mail . 35 183 = ) is )

VII — VALIDATION AND EVALUATION ), 67 ) = . − end t Q ( ) = s . A simple r end t 53 ) or interesting . Q 0 ( 41 s . − r 0 − ), have problems under- ) = ) = 42 . end t 0 ). end t Q − ( Q 44 s . ( r s r ) = ). But karma scores often mattered ) = 0 39 . end t 0 Q end t ( − s Q r ( s naires karma scores ) = r VALUATION ), acceptable ( end t E Q 50 . ( Fairness was one of the more problematic aspects of Major contributors have a stronger influence on total 0 s r rithm was strange and thatcould they achieve did good not ranks. know how They they ing said to that source by contribut- code whichers’ was contributions, high one quality could due also to oth- earn quality karma intervention. But acceptance of thethem intervention is among especially low. Theybecause might they give feel the treated software unfair a lot of functionality, but fix or spoil few source code. 7.7.4.3 Qualitative feedback from theSome developers question- perceived the presence ofas karma high scores due to theirdaily competitive emails effect, sent and due outdevelopers to noted to that the developers. this single emaildowns However, in created some presence. ups and It couldmails even happen were that sent — in because gotten the about evening it — until developers the hadnored next the day. for- emails Another altogether developer ig- becausehis he work did (on not the consider externaltion. API) relevant for documenta- the intervention. Onechanged developer when stated he added that a hisit comment, score and was so fair. he thought Others, however, criticized that the algo- 7.7.4.2 Relations between opinions and quantity For work quantity, mostof the correlations correlations are for quality, theare and inverse stronger. in many cases Forjor they negative example, relationship between there quantity seems karmaand score to average be opinion aexplanation ma- for this inversionrelationship is the between medium a negative developer’stity quality karma and scores. quan- have More a than fairness problem: that, they massare tend contributors unfair to ( think that scores standing how scores are computedand ( do not feel that reputation is present ( to them ( code quality because theytherefore contribute of more high code. importance to It also is reach them with the − very low quality karma scores because they only need to (like Millhouse) it is much easier to achieve very high or ( which seems not valued enough. For a minor contributor ) = mail ALIDATION AND ( s r environment, ). Neither high XP 05 . VII — V 0 | ≤ ) ). 27 . mail ( s ). This finding is consistent with r environment. The only negatively | ) = ). One might think that acceptance 36 . 13 XP . ). It may be that profiteers plainly enjoy mail ( ) = s 58 ) = . r mail ( ) = mail s ( r s : those who rank high do not consider ranking r (Festinger, 1957). It can also explain why the re- 9 mail 23 Humans want to always maintain a positive self-image. If they . ( 9 The factors that are most strongly correlating with Developers who achieve high karma ranks have a sub- Those who understood well how karma scores were There is a weak negative correlation between per- The perception of fairness seems unrelated to actual 0 s r reputation into the correlated factor is the perceived importancetion. of reputa- make observations, which are notthen congruent such with observations positive are self-image, ignored. could explain why there istween a moderate finding relationship karma be- scoresgood scores interesting ( and attaining high karma ranks are thecomputed, liking the of presence having of karma reputation, scores and the fitting of receiving the benefit of reputation; asno described such above, evidence was found.that developers who Analogously, care it for may source be code quality — and tool that supports source code quality. This reasoning computed had a slightly betterranks chance ( to achieve higher the expectation that understanding isproving necessary performance. for im- stantial tendency to like the computation of karma scores importance to ranking. Thisoverall could team result motivation in as a the threat positiveis effect to weak, on while winners the negative effect on losers is strong. rank ( is tied to profiteering andevidence a for good this. rank, but there was no ceived importance and− the karma rank ranks from quality karma scoresrelations as were only found very ( weak cor- nor low ranked developers consider karmaor scores less more fair. Similarly, there isbetween only acceptability a of minor karma relationship scores and a developer’s tion lationship between quality karma scoreopinion ranks, that and karma the scores fitis into stronger the for published ranks than for the correct ones. very important but those who rank low give higher ( who therefore have high karma scores — welcome any 184

VII — VALIDATION AND EVALUATION js aeecs fntadn omns”Among comments.” adding not of excuse lame a “just a o rcue hti tl mgtfi” apyn it “applying fit”, “might still it that precluded not was for motivation a “as code”. liked documented and good problem writing a as seen not was as scores karma the considered he So important. not was unnecessary. description a make would which pair- by affected negatively was fairness Additionally, eoe etoe htfrhskn fwr,Javadoc work, of kind his for that mentioned veloper onls n nmotn.As am crswr per- were scores karma Also unimportant. and pointless de- one Again surprise. a as in made come it but scores methodology karma research reason the the the was course, Of delay this lab. the for in late up showed scores lab expressive”, be the should including naming in well, “the reality instead very that The reflected ideology it. not for was credited lab is the one only code, responsible the are for people two although and on account, commit one’s might else someone because programming arbitrariness. of impression the create could This points. oorlbwsntbnfiilbcueo u a nature.” lab our of because beneficial not was lab our it to Although environment. evaluation the into poorly scores karma interesting. made highly competition the But pri- and conflicts. meaninglessness ority of feelings by impacted tively itself idea the Still it. for dislike keenly”, diffuse that a score had the just care following or not “not “did were first”, not it”, second about be relevant. to not “bad was felt it ones that Some concern some was there though it. and understand it” to about try care not not did “did so clear” just developer not One was score some. the to calculating of missing process was “the algorithm and the on information the understanding. detailed with was More problems it caused comments”, that score of karma Although idea correct. the was “knew it that developers hope even the but expressed clear”, only was he explanation devel- “the one that a Only thought had ranking. oper time the same on the influence at high and relatively trial, the in problem severe with competition the developers. again other and “im- work” were my important of scores provement karma made that aspects the be could conflict this ap- noticed running developer a One implement plication.” “to stuff”, priority other the much especially “too and with conflict in being as ceived h euaincnetwscniee sfitting as considered was concept reputation The nega- was scores karma of importance perceived The al- well, comparably scores karma liked Developers most the as rated was scores karma Understanding karma that was problem one importance, Regarding I- glLbFedExperiment Field AgileLab — VII-7 hc ae orlto eut adysaitclysig- statistically hardly results correlation makes which how and] [... is score my what with agreement “in were scores. karma with of because pair-programming was problem major A ol aebe eddfrClaRve oufl its unfold to CollabReview for needed been have would the instance, For experiment. the for control of out were possible. been subjects have of hardly numbers would greater sub-project, or project ware h edeprmn nlddol e developers, few a only included experiment field The icn.Saitclsgicne(%trsod o N=9 for threshold) (5% significance Statistical nificant. validity to Threats 7.7.4.4 assessment done”. long grades/payment/other [are] as not work idea us, of “nice for a only is is it it that as found or computed”, was it others However, in tendency everything. the measure like to not life does commenting, modern he few that to said led another which and peers his among tance conflict in being as feature perceived of was teaching contributor development the of driven also loss But the commits. and in work information pair of effects group yCollabReview. computed by responsibility be the not pair- blurs did could which developers programming, project that the was factor of Another length influenced. the Yet effect. full time more Perhaps short. to been have might experiment of realism the to 7.1.1.2). adds Section this (see studies time, field same the experimental At of that cost control. natural the is at comes It realism experimental 2004). List, and their (Harrison of with education domain the students in graduate working was experience development subject it the students, while featured example, For pool possible. setting as natural closely a part resemble as as to but, tried specifically activities, experiment teaching the of for it up artificial, set was not here is presented experiment field the of soft- agile an of size team But normal the most significant. is then developers statistically ten bigger, been been have had would size results team the moderate a If reveal results strength. be many to Still, need care. them with of treated most significant, almost are results about at starts nificance at reached is n eeoe etoe hth oie o accep- low noticed he that mentioned developer One hrfr oesbotmlevrnetlinfluences environmental sub-optimal some Therefore environment the that degree some to true is it While r ≈ 0 . 6 o an For . N ≥ 18 r ujcs hl some While subjects. ≤ 0 . 4 ttsia sig- statistical , 185

VII — VALIDATION AND EVALUATION to IDE VALUATION E affect their score, responsibility assignments are less fuzzying by pair avoid- programming and scores are more under developers need a better understanding of how to Understanding has one of the highest correlations with Reputation gaming can still work when the experimen- • The reputation idea was mostly accepted. But sub- Major contributors have a strong influence on overall Low ranks result in higher perceived importance of • Even more so, as whilebad being to second be is second good, not itto first”. “feels low Additionally, low perceived ranks presence, lead sodevelopers need especially more poor-scoring presence of reputationunderstand and that must scores measureportant something to them. that Contrariwise, is major im- the contributors worst have feelings about quality scores. But it isprocess them artifacts the most. the email rank.Hauser This and Katz, is 1998). congruent Perhapshave with developers needed would literature more just time (cf. to get acquainted to CollabRe- reputation could have been improvedup-front by providing training, more and morefeedback on immediate developer and actions. elaborate Anplify alternative the is algorithm to for sim- measuringier quality to to understand. make it eas- tal setup is amended inbetween such investing a in way that quality the andfirmer relationship a as, desirable e.g. reward in is the VIE theory by Vroom (1964), i.e. results could be integratedachieve higher in and the continuous presence. developer’s jects expressed that acceptance dependsare on used how scores and that theyEspecially should developers retain who their already light care character. come for reputation. quality, wel- However, there wasit no is indication profiteering that which leads to a better reception. quality. But acceptance of theis intervention especially among low. them They mightthey feel give treated the unfair software because a lotfeel of that functionality, this and contribution might isbe not explained valued to enough. major It contributors that must intervention the purpose is of not the assessing theirthe performance. goal Instead, is tohere improve they the have an quality even higher ofcontributors responsibility (cf. the than Section project, minor 3.5.3.2). and karma scores. This is a danger for team motivation. view. But also in the short time, the understanding of who are very important because they influence software ALIDATION AND VII — V of the reputation system environment Money seemed adequate for such a short, one-time Extreme programming favors expressive naming of and Katz, 1998). Perhaps thereprizes. should For be some more developers, or competition is better motivator. an For important instance, Krusty mentioned competition as important factor andhigh enjoyed presence it. is needed. Something And withreception in a with general, regard a to better the overall more different investment feelings in leads to scores. Instead of a daily email, and Katz, 1998). Consequently, therea should lot have been of (potentially senseless)ever, Javadoc it comments. did How- not happen here. The question is why? project. Possibly winning theor one too prize much was out too of risky control for the individual (Hauser Classical management theory states thatsures what is one mea- what one getsmight (see not measure Section the 3.5.3.3). rightbehavior, A things, but leading metric one to should undesired get what one measures (Hauser Management must clarify that thereand is that no indeed goal naming conflict, andRaskin, comments 2005). are Full important support (cf. forments the is employed necessary. measure- 7.7.4.6 Lessons learned regarding the effectiveness standing and perceptions of fairness in the team. identifiers over comments. Much inlosophy, project line management with had this ambivalent phi- opinions regarding comments and Javadoc duringiment. the field Some exper- students perceived Javadoc as senseless. to the project. Thelog analyzed and code did some not Java parts. includecontributed Pro- to For example, prolog as parts, Ned it mostly seems that heresponsibility mappings did and only possibly leadsor to skewed incorrect karma scores. In turn, this reduces the under- ming because there is onlythe one revision committer control recorded system in whiledevelopers. code is The written social by dynamics two tween that the are changing in pairs of effectintransparent developers, be- and to which the are reputationunderestimated. system, Also, the should metrics not werefor be computed a only part of the whole code base, which was assigned 7.7.4.5 Lessons learned regarding the development CollabReview is not very well suited for pair program- very little, which is wrong. It blurs contribution and 186

VII — VALIDATION AND EVALUATION hl h nevninpoal a a oeeffect, some had had probably intervention the While h am crswr o en sapromnemea- performance a as meant not were scores karma The ag- an into CollabReview brought experiment field This orecd niista r ouetdwt Javadoc. with documented of are ratio that the entities increase code of to source was quality objective internal The the code. improve source to project software ile Summary 7.7.5 esn learned: lessons is right. environment done project be must a and into difficult, system reputation integrat- a that shows ing it for Instead, unsuitable quality. is code system improving reputation a does that experiment evidence the But not discussed. were this for Variousreasons occurred. improvement quality measurable no winner. the for announced was prize a their and of scores, digest karma daily a receiving started developers lected, documentation. write not to do developers do: what to doing like of pain” compensation “endured a the as for but productivity developer’s a of sure • • • • • the therefore is study this of contribution major The col- was data comparison where phase initial an After • Isrmnaiy:bte score better “Instrumentality”: e hr sn niainta ti h profiteering the is it that indication no is there Yet nesadn em ob h otesnilfactor improvement. essential performance most factors, for the all be Of to seems quality. CollabReview understanding in of investing more reception to overall leads better a But reception. overall better a to leads that them. supports that CollabReview like tool wel- a quality come for care who developers Furthermore, bad a to leads place perception. second the Already negative strong effects. but positive weak high but i.e. presence importance, perceived low to leads rank low A team endanger not to spirit. and things, right the to measure carefully implemented be must Measurements “Valence”, prize). (i.e. achieve the to worthwhile is prize the (i.e. place first the rewarding probabil- only higher not a by with prize e.g. a ity, to leads score high a effort code “Expectancy”: in (i.e. individual the of control ⇒ etrscore) better I- onodDsg Experimentation Design TornBond — VII-8 ⇒ rz) and prize), iha mrvdpooye h rmr ou fthe of focus primary The prototype. improved an with purpose the experiment, field AgileLab failed the After h glLbfil xeiethdsonta nprinci- in that shown had experiment field AgileLab The mrv t ocp n design. and to concept necessary its deemed improve was experi- prototype Further the occur. with not mentation did effect intended experiment The the but failed. works concept CollabReview the ple Experimen- Design TornBond 7.8 ainaepeetdhr sms fte r technical interest. are scientific them lesser of of most details as experimen- here TornBond presented the are tation in obtained some results Only and validation. data for data was quantitative collect purpose to the not so secondary, were quality” software dependence”. without contributors “Involve Requirement- 6: and motivation” individuals’ to “Appeal Requirement- 2: requirements the on was experimentation experiments field further for road the pave thereby and to was experimentation TornBond the of Purpose 7.8.1 project. the into to brought cost developer would the it problems that the Euro fix in cost ap- approximate scores the karma praise debt Technical evolved. debt code technical source how in tracking was trialed the it Among modifications environment. development software a into hrfr,teClaRve rttp a brought was prototype CollabReview the Therefore, • • eurmnslk eurmn-:“mrv internal “Improve Requirement-1: like Requirements • • etnemsl eed nhwkrasoe are scores karma used. how on ac- depends and mostly rejection ceptance general few was there However, most, unfairness. the of them feelings to creating mattered they but scores about karma feelings bad had contributors mass Especially vletecnetaddesign and concept the evolve and learned, lessons collect tation 187

VII — VALIDATION AND EVALUATION Negative CollabReview did The developers in AgileLab only. However, the scores were all neg- +10] , VALUATION 10 E − [ To reduce the amount of emails but still preserve some Possibly, a social game addressing more of the intrin- and honor attempts to improvetypes them early. used Later a proto- modified quality assessment method that reported a low presencethe of presence, daily emails emails. wereevery first To commit. sent increase This immediately caused after adevelopers. lot One of developer emails complained being that sentabout he to to was configure just his emaila reader later to meeting, filter the them developer out.among made At his an colleagues on-the-fly who survey else installeddeveloper proposed a that filter. the Another emails be sent only once per presence, the prototype waited severalcommit minutes for after a a new commit.mit, If it there updated was its no karma furtherlist scores com- with and technical emailed debt a information ranking to developers. Negative karma scores should bescores avoided tend to demotivate. Initially,prototype the published CollabReview quality karma scoresof in the range ative in order toOne developer leave called much the prototype room frustratingthere because for were improvement. only negativethey seemed points. haphazard, and Another sarcastically proposed one that noted Rewarding system is important not have any of theknowpedia, other however, rewarding the mechanisms social of environment Mo- wassimilar. very All developers even were memberspedia. of Moknow- The hopeinterest was of that developers would peer-pressure arise. dueInstead, This the published did not scores self- happen. were notthing perceived worthwhile. as Karma some- scores alone doif not the work, team even is well-established,from and a knows karma different scores project.representing Additionally, management a should human be being involved in the re- is actually desired. sic motivation of developers couldsence compensate of the external ab- exchange value, and relieve CollabRe- Amount of informationshould pushed be tuned to right the developers view from the necessity of management representatives. was more likely to produce positive values. week. warding system to confirm that improvement of quality “points be squared”. Scores should be mostly positive, The de- ALIDATION AND VII — V ). Lines in other colors are the technical debt LOC type just started sending themmits. reports So the after developers their complained that com- liked they to would have be betterCollabReview informed is about introduced the in a changes.ment, development environ- developers When should be addressedIt in should an at official note. least explain what the reasons of the inter- ment environments. They led tothat several are lessons presented learned here. Intervention needs an initial introduction labReview prototype was installed. Instead, the proto- to enhance the CollabReviewadaptation concept, of and the supported prototype to social contexts of develop- 7.8.4 Results Most of the information obtained throughtation the resulted experimen- in lessons learned.also However, some raised results new issues, outaddressed of in which the several further have research. been 7.8.4.1 Lessons learned 500 of the developers. The dataerated shows by that CollabReview the do reports not gen- development seem very to much. have influenced But aschange said was above, not inducing the a primary goal of this experimentation. in Figure 7.15. Theof black source line code represents committed by the the quantity tracked developers (in and then observations and discussionsthe of developers its were effects collected. on Thelessons pursued learned results for are designing andCollabReview better prototype integrating in the a development environment. to be evaluated quantitatively. 7.8.3 Data 7.8.2 Method mal iterations, in which the CollabReview prototype vention are, and what is going to happen. velopers did not get an initial introduction when the Col- The results obtained through the experimentation served The technical debt karma scores collected are presented The design experimentation consisted of several infor- was modified, put into the development environment, While karma scores were tracked, they were not meant 188

VII — VALIDATION AND EVALUATION ihn ehia et n utne otk aet not to care take to need just and debt, technical no with etdt aeasrc ult rbesmr tangi- more problems quality abstract make to vented huad fErsi ehia etddntincrease not did debt technical alertness. estimated in several Euros of of amassing thousands The lot. were a who developers contributing punishing were they that notation quite himself. for it goal reached new he a set as himself to for but had goal debt he a quickly, of set initially 7000EUR he reach list, the to tried in he last while be that to said not were developer scores One karma jokes. debt to Technical subject have it. and keep score to best struggle the to with progress start of already indication developers an as contain not does because debt un- appear technical developers’ to the failed to effects add Motivating not derstanding. did They intended. as out touch they code shape. source good all in keeping by score their spoil start would Developers ratings. quality than concrete ( debt Technical numbers negative reason. publishing this avoids for introduced were debt technical personal 2010). developers’ al., the tracking et scores (Brown Karma managers and developers for ble debt Technical × × × × × × × × × × 0.8 1.6 2.4 3.2 4.0 4.8 5.6 6.4 7.2 8.0 10 10 10 10 10 10 10 10 10 10

ehia etsoe lortie h eaiecon- negative the retained also scores debt Technical work not did scores karma debt technical However,

3 3 3 3 3 3 3 3 3 3 0

ehia et(nEuro) (in debt Technical

12 18

h ocp ftcncldb a in- was debt technical of concept The

24 30

iue71:Eouinhsoyo ehia eti TornBond in debt technical of history Evolution 7.15: Figure 36

42 48 Experimentation Design TornBond — VII-8

< x

0 60

n smore is and )

78 84

ol uoaial eacmltn o ftechnical of they lot because a luck accumulating bad be in automatically be would would lot a contribute who elcigo uniyi o osdrdfair considered not is quantity of Neglecting o opt am crs tol rvdsoevesof overviews provides does only Sonar It that scores. is karma difference compute not the com- prototype, product mature a a to is pared Sonar not While would suited. 3.3.5.3) as- better Section be quality (see some Sonar like if tool question sessment the asked developer One issues Raised 7.8.4.2 has scores carefully. karma considered the be in to included therefore be and to If is CollabReview. quantity is for how quality quantity Still, above here. goal developers higher by the unfair bit a as inter- preted be indeed might quantity attention. contribution developers’ Neglecting the of for one competing only projects is TornBond several unfairness. development, for potential in is efforts there of amount similar developers Speciﬁcally a all contribute not less. where environment, does TornBond the who in someone tech- more than true accumulating debt is of it nical risk clean, higher code a source have their they kept that they would if they so because do debt not technical of automatically developers lot that a true accumulate not is it While debt. those that argued who developer a with arose discussion

102

108

114

120

126

132

138

144

150

156 162

long A 168 day

189 174

VII — VALIDATION AND EVALUATION ments VALUATION E motivation instead of relying on external rewards Try CollabReview intributed a consortium project new Vyrygh)confidence environment to in (dis- the increase earlierknowpedia field (Requirement-3: experiment “Flexible in applicabil- Mo- ity”). Build a rewarding system based on CollabRe- neglecting contribution quantity is consideredfair un- if contribution is —tary, to and some degree — volun- copy detection should be usedsponsibility to for still moved attribute source re- codedeveloper. to the original vation”, Requirement-6: “Involve contributors view’s karma scores that better addresses internal (Requirement-2: “Appeal to individuals’ moti- without dependence”). The previous AgileLab experiment had shown that the • The lessons influenced the design of further CollabRe- • • • exchange value offered for karma scoresthe is CollabReview concept. essential to However, offering expensive tangible rewards is not alwayscrowding, desirable (e.g. see motivation Section 3.5.4.1) orenvironments (e.g. possible public in service all administrative work tions). restric- ReputationForge therefore meant tomotivation explore possibilities. other Instead of simple rewardingreputation and mechanisms described earlierReputationForge in was this a web-based work, application thata added complex social media game.to The address game a builds set on of tasks to internal the motivators. CollabReview It prototype. is an extension 7.9.1 Purpose field experiments. 7.9 ReputationForge Field Experi- by Agaraj (2012). He testedin the Moknowpedia CollabReview and concept Vyrygh, but with focus on the re- tion. view prototypes and how they were deployed in the later The purpose of the field experiments was threefold: The ReputationForge field experiments were conducted warding system. Refer to this text for a detailed descrip- ALIDATION AND VII — V has to be tuned carefully, negative or negatively-connotated karmashould be scores avoided, technical debt is a negativenot karma fulfill score its and promise does of tangibility, developers should receive detailed information on CollabReview upfront, the rewarding system and thekarma exchange scores value are for vital, the amount of information pushed to developers • • • • • Some resistance appeared in the development team Another developer proposed that CollabReview could In summary, the following lessons about CollabReview labReview prototype was configured toship use algorithm its with author- copysponsibility detection. mostly stays Through with the this, original re- not developer, and the one who isconcern merely copying is the therefore source considered code.sufficiently. as The taken into account 7.8.5 Summary Most of the work wasre-organizing refactoring, old and source code more in precisely it new between places files. by copying The developersof argued development, that readability in was this of stage noonly concern. the quality Instead, of newlybe developed source regarded. code Apart should from somedecide if technical source difficulties code to is written anew or moved, the Col- developers would have less motivationthe first to place. not Therefore do thistradicting it issue the in is CollabReview considered concept, as and con- istreated not here. further because of the kind of work that was being carried out. scores are not dragged down by poor quality of oldproject soft- backgrounds of different extensiveness collaborate, developers with a long historyor might disadvantaged. be However, if advantaged time cures old sins, then a code base. The questiona if personal difference karma is scores picked make upSection again 7.11). in a later experiment (see were learned: ware process artifacts. Especially when developers with 190 “forget” about scores over time so that developers’ karma

VII — VALIDATION AND EVALUATION hl euainog ol nyb vial othe to available be only would ReputationForge while CollabRe- the testing for environment primary the As h rjc ssgie yalrenme fpotential of number large a by signified is project The iwcnet h iio rjc yyhwschosen. was Vyrygh project of wiki the concept, view h xsigrwrigshmsfo CollabReview. from schemes rewarding existing the augmented ReputationForge Moknowpedia, In infrastructure. development GForge project’s the into integrated the by consortium funded a project in partners distributed from contributors Method 7.9.2 xeietwt onwei eeas participants also were Moknowpedia ReputationForge with the pre- experiment in (not participants study Some usability here). later sented a for Reputation- needed used were had Forge a who that participants was of here number challenge high particular A similar. be CollabReviewshould by of computed ranks average scores the contribution And original groups. be both should in groups present work equally two the student of of Members number same employees. the receive were: should criteria groups the Both Here guidelines. similar followed dia in similar be should groups. employees both of logins) the of and number contributions through accumulated defined The (as groups. levels both activity onto be parts should equal member in each split of employees in The equally groups. represented both compara- be groups should two workpackages All the make ble: to chosen were goals ing several the into in split Work was project members. consortium project difference into taken not were position consideration. social or gender culture, tenure, organizational demo- education, However, like possible. differences The as graphic similar experiment. as be the should of groups preparation in concern major talk group. not control the to of members instructed with were about who difference, group any experimental see the not use and would before group were control they The as tools parallel. experimental an in and group control a running support to enough itiuino h 6priiat rmMoknowpe- from participants 26 the of Distribution nine from participants 38 were there Vyrygh For a was groups two the into users of distribution The large users of number a have environments Both • rae h xhnevleo optdscores. computed of in- value scores exchange karma the of creases use the how understand Better EU h euainog was ReputationForge The . workpackages I- euainog il Experiments Field ReputationForge — VII-9 h follow- The . h aacletddrn h vlaincnitdo the of consisted evaluation the during collected data The ievra on o nrae h oa ubro sub- of number total the increased so, Doing versa. vice ru n htte hudnttl ommeso the of experiment. members the to about talk group not control should experimental they the in that were and they group why to as and them ReputationForge, instructed use and access the to how on experiment, information participants gave period, until evaluation experimenter the 2011-12-12 the of (from beginning the weeks At two 2011-12-21). about for ran shorter considerably and was period Moknowpedia is- the organizational sues, to Due 2011-12-21). until 2011-11-16 too. here, comparable obtained somehow were two groups of end, distribution the the in in But considered participants. be several to for had holidays which Christmas people, of to start happened the period with evaluation coincide the Also, give it. could on and feedback CollabReview with worked had who jects and group, control Moknowpedia the in was experimental be constraint Vyrygh should the group a in So were who participants Vyrygh. that with experiment the in o yyh ih u fte sdReputationForge used them of out Eight group experimental Vyrygh. the for in were participants Nineteen Vyrygh for Data 7.9.3.1 scores. karma participants’ and the completed and started, tasks; of failed number the logins; of number Data 7.9.3 to experiment. the regard of with results hints the gave also the but at evaluation aimed usability of mainly end questionnaire the The after experiment. the participants group experimental sent was all that to questionnaire a prepared experimenter the con- ( participant 30% the least which at to tributed articles of timelinesses, number their the of and of sum the number by the determined quality, as quantity, reviews for logins scores of number karma the and concerned This experiment. the of days. four next the gain for to day or each levels, points two quantity by two articles, rank two quality one’s rate increase to to like par- tasks the included by They completed and ticipants. selected be could that Forge h vlainpro nVrg a v ek (from weeks five was Vyrygh in period evaluation The otdt ol ecletddrn h conduction the during collected be would data Most Reputation- in tasks eight prepared experimenter The C30 am cr) Furthermore, score). karma 191

VII — VALIDATION AND EVALUATION projects seem to be a very difficult environment VALUATION EU E ReputationForge had to compete withCollabReview. the original the evaluation time was tooable short due and to a Christmas bit time, unfavor- the Moknowpediaestablished among was itsfew already users, room for improvement, so and quite that there well- was Two participants of Vyrygh stated that Reputation- The true benefit of ReputationForge over simpler re- The field experiment with Vyrygh shows that Col- Possible reasons why the positive effect did not clearly • • • important for the CollabReview concept. But as offering Forge is probably more suitableHowever, for contribution more in active wikis. Moknowpediaonly initially grew also slowly after the introductionCollabReview of is CollabReview. made exactly for this kind of situations. tioned a lack of timetime as means the nothing else major than reason.therefore a to But priority be lack conflict. concluded of It that has just the not rewarding effective system enough was with them. be shown with high confidence.time It for is doing probable the experiment that was the unlucky. labReview and ReputationForge together clearlythe have power to change userever, behavior for the good.because How- by far not allsearch users may could be be necessary. reached. More re- 7.9.5 Summary 7.9.4 Results In Vyrygh, ReputationForge clearly made a difference. cept for the number ofsignificant logins based they on are two-sample are t-tests. alldia, statistically In ReputationForge Moknowpe- resulted only infor a the small experimental advantage group over the original CollabRe- showed statistical significance for theted number reviews. of submit- show in the Moknowpedia experiment are that view used in the control group. Two-sample t-tests only The exchange value offered for high karma scores is The subjects who did not work on a single task men- warding mechanisms used in CollabReview could not All results are better for the experimental group. Ex- , , 5 of . 17 07 . 0 . 2 0 25% . ALIDATION AND 10 in the con- as compared as compared articles in the in the control 22 of the available . 67 29 63 1 52 . . . . 2 0 0 0 5% VII — V as compared to . 8 as compared to 62 . 0 18 . times instead of only 0 84 . 4 as compared to as compared to 05 . 55 . 4 0 articles instead of only 15 . 1 , his quality karma score by , and logged in 01 056 . . 1 0 Figure 7.16b depicts the contributions by participants Figure 7.16a depicts the contributions by participants his quality karma scoreand by rated control group. However, the onlydifference statistically is significant that for reviewed articles. This is congruent to a lack of data in the experimental group. over the end of the evaluation period. in control and experimental groups.oper The in average devel- the experimentalkarma group score increased by his quantity group, his C30 karma score by of time. The average participantthe completed available tasks. In total,two the articles” group six completed “Rate times, “Makeleast two 30%” contributions twice, and of “Rate at everydaydays one consecutively” article once. for There two werecomplete a other few tasks attempts but to they were either failed or ran 7.9.3.2 Data for Moknowpedia regularly. They completed nine tasks.pants The never other logged partici- in, in the same way mentioning a lack to times in the controlcompared group. because Review the data control couldpossibility group not to did submit be not reviews. have the in control and experimental groups.oper The in average devel- the experimentalkarma group increased score his by quantity trol group, his C30to karma score by complete any tasks. The first and probably easiest task two participants. Other tasks were each completed once. tasks. by logging into the systemanother once on five a logged regular basis, inactive and participants from completed time thirteen tasks tostarted in time. another total, four and tasks, The while the most rest did not start and The experimental group for Moknowpedia consisted of The average participant completed with the finding that mostlywas the completed. “Rate Login two data articles” could task not be compared due 192 11 participants; out of them, six used ReputationForge “Rate two articles” was completed six“Make times, two followed contributions by of at least 30%” completed by

VII — VALIDATION AND EVALUATION uhasse a oivlepoetadconsortium and project all. involve to to management. attractive has are system a that and Such systems involved rewarding adopters create late to getting needed is for What techniques participants. are all involve to general, difﬁcult In is A it helped. time. have single might a period ReputationForge evaluation to longer on log even not participants Many did environment. difﬁcult a are projects motivational several a theories. on adds building that game social prototype complex CollabReview the an to is ReputationForge extension be ReputationForge. to by were value explored exchange perceived the means increasing other of possible, always not is rewards expensive nadto oti,teeprmnsso that show experiments the this, to addition In • • • • • • was ReputationForge 1 2 5 6

u ohfil xeiet upr h au of value the support CollabReview. experiments field both But period. test the in conditions unfavorable be the may to which due significant, statistically hardly they are simple but CollabReview the in over mechanisms improvements rewarding be to seem There clear. that not is system rewarding as benefit added Vyrygh. of wiki the its on However, impact considerable a had it and Moknowpedia, in contribution improve further could It between-group with designs. experiments field two in tested Mean oto group Control Quant a yyhcomparison Vyrygh (a) C30 iue71:Cmaio fcnrbtosi yyhadMoknowpedia and Vyrygh in contributions of Comparison 7.16: Figure Qual xeietlgroup Experimental I-0—McodtatgtrFedExperiment Field MicroEditCastigator — VII-10 n/a Login EU hnvrteeaetrecagsb h aeauthor same the by changes three are there Whenever The h lblhsoygt rwe ihmn irrelevant many with crowded gets history global The .0McodtatgtrField MicroEditCastigator 7.10 iscnb noae ihlgmsae ofrhrex- further to messages log with com- annotated Additionally, be and can base. logical mits code single, the a to have possible change to complete as logical necessary changes as one few many be as as but include should should commit It every change. should i.e. commits that atomic, is be Subversion of Section idea (see The repository 2.3.4): Subversion Karakos’s to mits used. be should function preview wiki’s doing the of edits, Instead the micro to messages. unclean problem sensible bigger write an a to pose contributor write for edits to micro message that difficult so log more commit, revision is commit a it appropriate course, have an Of not do Furthermore, message. typically log history edits local entries. preview the unnecessary such while with artifact, cluttered single is a of changes logs. history revision try-outs pollute as they created because be previews not that or should is revisions rationale new The wiki, artifact. a the in for rating poor a issue are changes MicroEditCastigator the between then passed minutes commit, called five subsequent than two name less same each and the row, with a artifact process in software a the to the of (2012b): opposite Schneider the and Singer is by in arbiter applied gamification committed This are succession. that short changes small many punishes iia rbe a omryocre ihcom- with occurred formerly had problem similar A 0.2 0.4 1.4 1.6 MicroEditCastigator

ir edits micro Mean Experiment oto group Control Quant b onwei comparison Moknowpedia (b) hnamcoei sdtce,the detected, is edit micro a When . C30 Qual stigrd twl then will It triggered. is Review xeietlgroup Experimental sa rie that arbiter an is n/a 193

VII — VALIDATION AND EVALUATION 21% decrease in total 9% was installed. It sent its was condoned. Note that VALUATION E Both of the stories above support that it is the karma One user started to rate articles with the best grade scores that make users refrain from undesired behavior. decrease in micro edits at only a first negative review, which promptlycomplaint created by the a user. first Theit arguments were put that forward frequent against savingdata, would and be that better it than would losing in do injustice the to the article arduousexpressed itself. work astonishment that A thearticle way few should how days he affect later, edits thethe an another quality course user of of its thecould content. be next heard months, from Over time similarMicroEditCastigator to time disagreement but the presencethe of question the if saving lotsbad of or small not changes is actually not is hold answered the here. view (However, I that personally the log is valuable documentation.) arbitrary rule, and what theis. influence on their behavior possible to compensate for aarticle poor had score received whenever a that micro edit penalty. Another user that. But heseem never submitted to any care review. forplausible He his since did he karma not joined scoreand, the anyways, as work a which group student, rather is wasMoknowpedia lately not reputation that game. much integrated into the 7.10.3 Data Figure 7.17 reproduces the historyknowpedia. of The micro table edits (Figure in 7.17a) Mo- printsnumbers, the and measured how theybefore relate the to intervention are each put other.corresponding above the month Months symmetrically after the intervention started. The Before the intervention, the dashedof line new software (total process number artifactsthe per same month) is as at the about continuousintervention, line however, (micro it edits); remains during above the it.of The micro number edits per monthcomparison has to decreased the in number total of new and artifacts. in 7.10.4 Results and discussion MicroEditCastigator values are plotted in chronological order in Figure 7.17b. The experiment just wants to test how users react to an The first micro edit occurred a few days after the And this is also what the data shows: there is a who was responsible for many micro edits, did not do ALIDATION AND s, developers was installed. It IDE could not know in VII — V was added as an ar- MicroEditCastigator MicroEditCastigator MicroEditCastigator For evaluation, the number of micro edits in time There was no announcement or other information When the same problem emerged in the well- between two edits. Ifmicro there edits, were they were multiple counted subsequent in as a one. row That were is, still fiveas counted edits as the one micro edit.advance However, how many moresubmit edits or would update come, reviews for it eachthe would edit third in on. the row from just started penalizing micro edits withthat bad they reviews were taking part in an experiment. frames of one month wasat measured. least A three micro edits edit in was a row with less than 300 seconds submitted non-anonymously by thetherefore arbiter, not rejectable. and The were onlyreview way was to therefore get that rid it of lostof a timeliness further bad changes. in the course biter to the Moknowpedia installation of CollabReview. it, it would automatically submitrating a and review an with a according poor comment. The reviews were an undesirable behavior with aand penalty to see karma how scores, developers reacted.CollabReview could The change question a washuman behavior if intervention could that not. repeated 7.10.2 Method Here, too, contributors were remindedcare several times for to evolution documentation but to no avail. 7.10.1 Purpose Hruschka and Starke, 2011; BuseBut and Weimer, due 2010a). to difficulties handlingdid their not follow this best-practice (see Section 2.3.4.5). established Moknowpedia with its integrated CollabRe- plain the changes. Aconsidered well-cultivated change as important history key is componentand for maintenance, awareness although it hasof the additional summarizing burden the change in a concise message (e.g. view, an opportunity for another experiment was born. The The purpose of this field experiment was to disapprove whenever they occurred. The participants did not know when the Whenever there was a series of micro edits detected by 194

VII — VALIDATION AND EVALUATION hneltr edcddt aetecag imme- change the make to decided the make He to or later. penalty, the change change get the to make risk either and to decision immediately a change. make to to had something now still He was recog- there and that saved, more once one, make nized that to made had also he that change, recognized another it, saved change, a made new of to number total the is in artifacts edits micro of percentage a a at is edits there micro out, start left the are after intervention and the before of month ninth the If artifacts. new sroc xlie o eeie natce He article: an edited he how explained once user A 2 iue71:Nme fmcoeisdtce ahmnhbfr/fe tr fitreto (July intervention of start before/after month each detected edits micro of Number 7.17: Figure

Number of micro edits/Total10 new12 artifacts14 34 . 68% 0 2 4 6 8 -9 uigi,wihi eraeof decrease a is which it, during 0 50 100 150 200 250 300 350 850 vrg percent Average edits micro Percent month per artifacts New edits Micro (experiment) Month percent Average edits micro Percent month per artifacts New edits Micro (control) Month 4 . -8 44% 2% -7 eoeteitreto scompared as intervention the before nraei e riat.Temean The artifacts. new in increase -6 -5 I-0—McodtatgtrFedExperiment Field MicroEditCastigator — VII-10 -4 .809 .329 .411 .825 6.04 2.50 2.48 1.19 2.94 2.94 3.92 3.03 5.31 0.93 4.00 2.08 4.44 2.01 4.93 5.51 4.72 3.25 4 0 2 8 9 8 2 4 204 245 325 180 298 284 127 106 246 13% 9+ 7+ 5+ 3+ +1 +2 +3 +4 +5 +6 +7 +8 +9 8181510128 2 8 563 280 323 84 102 170 165 108 48 9- 7- 5- 3- -1 -2 -3 -4 -5 -6 -7 -8 -9 b rpia itr fmcoedits micro of history Graphical (b) -3 34 7 8 1 8 3 13 5 13 8 5 6 1 14 1 7 5 8 ≈ eraein decrease 40% a itr fmcoedits micro of History (a) -2 . -1 mean= mean= iwcncosydcd obekteefre ue i.e. rule, enforced the break to decide consciously view elyhv enapt fi a isdoeviolation. one missed had it would if it pity that a been ironically have declaring really some- by be or might broken, there mentioning thing control”, call- of like “out comments it with event ing this Sev- to before. reacted deferred users had eral it it reactivated, that reviews later the was all it submitted When the after evaluation. reasons the technical of to end due days some for disabled alternatives the up other. weigh each and against before it about think they CollabRe- of avail). users no that (to shows penalty example the the avoid However, to hoping still diately, 0 noeicdn,the incident, one In 4 1 2 . . 44 68 median= , median= , 2 3 4 2 . MicroEditCastigator . 23 5 4 5 6 7 19 th 2011) 8 was 195 9

VII — VALIDATION AND EVALUATION . 40% ≈ (Day 1) and went until th 26 (Day 47, only counting work days). Two th VALUATION 29 E Undesired behavior was reduced by still adapted to the new rule promoted by CollabRe- view. The purpose of this experiment was to show that Col- The teams consisted of nine and eight students, re- • • bined skill score was computed inskill which weighted the thrice Java as coding muchdents as the were other then skills. distributed onto Thethe the stu- rank two of teams their based combined on skill, i.e. the highest ranking type did not have the expected effect on development. iment was found. labReview can work with source code just as well asprovide in data supporting the fulfillmentrequirements. of all remaining 7.11.2 Method independent software projects following the agilemethodology Scrum were to be executedpost-graduate students during in this two time teams. by spectively. At the beginning, students wereonto distributed both teams based on afill questionnaire out that before they had the to lab.assess The their questionnaire Java asked coding skills, them other to edge coding skills, of knowl- Scrum, experience with software processes, and CollabReview led to a lessimproved cluttered quality. revision history and 7.11 ScrumLab Field Experiment labReview prototype into two software projectsing follow- the Scrum development methodology. 7.11.1 Purpose Several problems had been identifiedexperiment. in Most importantly, the the AgileLab CollabReview proto- June The ScrumLab started April The ScrumLab field experiment introduced the Col- wiki environments. Additionally, the experiment should with collaboration tools like Subversion on a scale from With the ScrumLab, a new environment for a field exper- 1 (very low) to 10 (very high). For every student a com- arbiter ALIDATION AND VII — V MicroEditCastigator MicroEditCastigator did not agree with the and showed some recalcitrant and counteringthey behavior, years of the studied time period. There was some 2 • • Trading off the two behaviors (frequent vs. wise com- The experiment provides an indication that CollabRe- The total numbers from the control and experimental Making the arbiter’s reviews rejectable might give / 1 discussion. The point is that although users do not contain a “logical”decrease change. the Micro quality edits of therefore theother revision history hand, log. micro edits On save the of the data user if from he potential did loss not save his work for too long. mitting) against each other is not the purpose of this an artifact in a shortmay time frame. not Such be behavior desirable: mayhistory or micro of edits the article clutter and themaking the revision it revision difficult repository, for thereby others toapplied grasp and what what changes is were happening.also Of do course, not micro receive edits a change message as well, as they extended with the fluctuation in personnel, primarily inand student personnel, it could bestyles. that However, the new main users actors remained have the different same. editing ternal validity it cannot be called a “proof”. 7.10.5 Summary cognitive processes in userstransfer indicate to that other cases results as may well. phase are probably notpotential easily changes in comparable work due practice that to can the occur in the good anyways. 7.10.4.1 Threats to validity not be transferable to other cases. Yet the described users an opportunity to expressferent their way resistance than in countering a with dif- users good tend ratings. to But rate most the articles they contributed to rather view works, but due to problems in its internal and ex- The CollabReview prototype in Moknowpedia was This field experiment only covers a single case. It may which penalizes the creating of many new revisions of 1 196

VII — VALIDATION AND EVALUATION ilgttebns fadvlpr’proa cr s0 is score personal developers’ a If bonus. the get will wti-ru experiment). (within-group ( h nevnineddfrTa nMay on A Team for ended intervention The Team of performance the to compared be could A Team eyfweprec ihSrm( Scrum with experience few very nevnin uigti ie h efrac of performance the time, this CollabReview During A) the (Team receiving team intervention. while one project that the means started That between- designs. and within-group group with experiment an design to ( tools collaboration with experience less ( process development software general in programming in experienced them- rather considered as students selves the scales, five these than On less percent. by the differed and score average equal, skill the were combined end, teams average the both B, In in team on. ranks on so skill and combined two A, next team the on A, two then team into put was student oeo h ujcshdasoeo rls ttestart the at less or 0 of interventions. score the worse. a of grade had a subjects half the be of will None grade lab his then below, then or not, score quality If highest a grade. the receive with a will developers half team two by the the grade only in lab all their then on 9, bonus quality least a at has of ranks the score in shown everybody If students: 6.11). Figure for (see details scores hints the as and improving violations board, detected leader and a responsibilities on on ranks as a current scores developers’ contained all quality It of consisting team. report his quality within first it circulating for sible team’s respective the to May on B Team for started and 21), the when May time 10, the (Day at ongoing only really A was quality Team development first to the sent day, been first was had the report on intervention A the Team to While announced started. development until finally weeks had two about few that took first it was Instead, the problem in days. the A immediately of start half days. not the 22 could at development about be after to fixed i.e. was project, B Team for start and performance prior its to compared be performance the can project, B the Team of of end began the and At A B. Team Team the for for half ended intervention about the After project, experiment). (between-group B 6 . h olwn eadn ytmwsanucdt the to announced was system rewarding following The email instructional an with started intervention Each A Team for intervention the of end the for time The allowed ScrumLab the of organization and size The 5 n ihJv ( Java with and ) 5 . 5 ,adwr nwegal fthe of knowledgeable were and ), cu Master Scrum 5 1 . 9 . 5 .Te a little a had They ). 29 ). I-1—Srma il Experiment Field ScrumLab — VII-11 h a respon- was who , th Dy23). (Day 4 . 6 25 ,bthad but ), th 10 (Day th ). n hti htwsepiil eomne oTeam to recommended explicitly was what is that And tteedo ahta’ nevnin questionnaire a intervention, team’s each of end the At xte ntels e aso h lab. the of days few to last hope the or in immediately, them them fix fix time, recklessly over to problems not fix and and style improve carefully to B: score. better a approach carefully should to teams time both enough that have intervention meant respective That the relevant. of was Only end period grade. the the at for score important final not the were team the to sent o oprsn h e tm were: items new the resubmitted comparison, were for which 7.7.2.2), original Section the how (see to questions of addition understanding In better perceived. the was a to CollabReview given gain was to AgileLab members the team from one the to similar experi- the of completion after Interviews 7.11.2.1 • reports intermediate that informed were teams The • • • • hints: following the received also teams The • n:Wyhsyu oebcm uhntmuch much/not become code your has Why And: little/much you cause contest the did Why And: influ- little/much have contest the did Why And: eeoe e i fms iltosfudby found violations most of let rid easily should get which developer code, a source of style the other most or IDE settings format default with to Eclipse plug-in a also Eclipse. is within There from Checkstyle code in. source it their checking check before to Checkstyle themselves the by install could toolkit they then liked, they If oeunderstandable? more contest? the to due become under- code your more has much standable How — improvement quality effort? additional additional you? caused the contest was the that big effort How — effort additional code? developing of way your on ence code? developed you how way contest the the on did have influence much How — influence team. a as or individually win else’s could someone They fixing code. by other score, each karma help own could they his had developer every While quickly. very Checkstyle swl,aecpbeo uoaial fixing automatically of capable are well, as s ment 197

VII — VALIDATION AND EVALUATION 65 . 0 . Until ≈ 75 . 0 ≈ , while Team B had almost , while Team B had SLOC SLOC VALUATION E violations per 0 At the start of the intervention, Team A had During one of the first review meetings after the first Figure 7.21 displays the history of quality karma . Both teams increased their quality from almost one of reaching the nine points necessary for the bonus. He because he had workedpunished. so But much his team and members would could calm now him get down, bonus. the end of the intervention,to Team A increased its quality 0 teams almost reached the best possible result. 7.11.3.2 Observations On the last two days beforeincreased the on lab their ended, quality Team karma Athe scores much problems by found fixing by all Checkstyle. of However, a problem customers, not everything worked as theyit. had Some expected functionality which had originally been there, that they should carefullytheir and goal when steadily their work intervention started. towards scores had been published tohe had Team A, not slept Clark well said the that last night. He saw no chance Especially on the last fewcrease days to there the maximum was possible aof score sudden the for in- team. all members In thekarma very score beginning, below Wally had 0 a buttion fixed quality this started. before Still, the most interven- initially. quality Individual scores quality were scores rather started low as to the fall intervention as was soon over.spoiled Clark, his in good particular, score, rapidly whilethat others quickly. did Clark not was lose a points period. major contributor Although during Bruce this contributed even more,remained his rather score constant. scores in Team B in muchthat the while same the average way. score It ofthe can Team be first A seen intervention, rose the to scores 10slowly during of worsened subjects until in the intervention Team in B Team B started. towards the necessary minimum scores.subject While in not Team every B was able to attain the perfect score of violations per violation per line, to almost zero, and therefore both Team B used the full time available to them to work was very frustrated and felt offended by the coding rules was that they broke theirwhen they software wanted in to show this their process. finished software So to their was lost. For this reason, Team B was explicitly advised 10, they all reached the necessary 9 points for their team ALIDATION AND VII — V . The figure shows that scores continuously for you to only see an overall score? concern — How much did it concerncould you see that your others score? difference — Would it have made a difference for of individual scores? you to only see an overall score of the team instead And: Why did it concern you much/not much? And: Why did it make a difference/no difference HSLOC Figure 7.20 shows the development of quality karma Figure 7.19 visualizes the same data for Team B. The • • scores in Team A. The blackscore line in is the the other average team. quality in the So, other while team thein slowly average the stabilized quality first at half a of lowerstarted the average to project, improve the their members scores of during Team the A intervention. peak plateaus found in thedue second to half of renaming the efforts projectforming to are to make the the Checkstyle coding source styles.process, code In directories con- the with renaming source codebefore were first the copied old sourcetime, code duplicate was code existed, deleted. which issteep So responsible increments. for for the some increased. An interesting observation isClark that and on Day Bruce 19 suddenlyClark changed lost positions many because points while Bruce gained many more. helped him to fix his code (see also Section 7.11.3.2). in Team A. The blackscore line in is the the team. average quantity Moreas karma and subjects more became colored responsible lines for appear tifacts software by process committing ar- first sourcethe code repository. and The changes quantity to karmain scores are measured 7.11.3 Data the weekly Scrum review meetings betweenproduct customers, owners and the rest of the team. 7.11.3.1 Collected karma scores Figure 7.18 plots the evolution of quantity karma scores over the duration ofquestionnaire the given experiment, to responses subjects totervention at period, the the and other end observations of made during their in- The reason is that the team of Clark and especially Bruce The collected data consists of the subjects’ karma scores 198

VII — VALIDATION AND EVALUATION 60 14 20 22 25 28 12 18 24 30 36 42 48 54 17 11 6 3 6 8 0 0 uniy(nHSLOC) (in Quantity HSLOC) (in Quantity 2 2 Arthur Steven Matt Lex Harvey 4 4 6 6 1 1 iue71:Qatt crso emAcmae oaeaeo emB Team of average to compared A Team of scores Quantity 7.18: Figure iue71:Qatt crso emBcmae oaeaeo emA Team of average to compared B Team of scores Quantity 7.19: Figure st st crsTa A Team scores A Team scores 8 8 10 10 12 12 14 14 I-1—Srma il Experiment Field ScrumLab — VII-11 Victor Max Pamela A Team Average Clark n emA Team End n emA Team End 16 16 18 18 20 20 22 22 tr emB Team Start B Team Start 24 24 26 26 Dick Wally vrg emB Team Average Oswald 28 28 30 30 32 32 34 34 36 36 Bruce Peter Selina Johann 38 38 40 40 n emB Team End B Team End 42 42 44 44 day day 199 46 46

VII — VALIDATION AND EVALUATION 46 46 day day 44 44 42 42 End Team B End Team B 40 40 38 38 Johann Selina Peter Bruce 36 36 34 34 32 32 30 30 28 28 Oswald Average Team A Wally Dick VALUATION 26 26 E 24 24 Start Team B Start Team B 22 22 20 20 18 18 ALIDATION AND 16 16 End Team A End Team A Pamela Max Victor Clark Average Team B 14 14 VII — V 12 12 10 10 8 8 scores Team A scores Team A st st Figure 7.21: Quality scores of Team B compared to average of Team A Figure 7.20: Quality scores of Team A compared to average of Team B 1 1 6 6 4 4 Harvey Lex Steven Arthur Matt 2 2 Quality score Quality score 0 0 1 1 7 7 3 2 8 6 5 4 9 3 2 8 6 5 4 9 10 10 200

VII — VALIDATION AND EVALUATION ihteie fClaRve.Teoiia developer original The CollabReview. of idea the with al . umrzsteoiin bu h CollabRe- the about opinions the summarizes 7.6 Table iwitreto nteSrma.Ta ae the rated B Team ScrumLab. the in intervention view prcaetevleo la orecode. to source learned clean had of value he the There appreciate again. decrease to quality since started code that had source noticed there, Yet longer had no up He him. was followed CollabReview to Clark statements. pain 46, original a Day his on was on meeting it lab Still final have the not manually. in did all he it so do configure Eclipse, to to in how styling him code to automatic saved explained eventually Victor was had Also, it who that members him. continued team He his for code. time only source much the too him up taking clean been was to had Checkstyle It that argued mad. he him driving A) Team retrospective (end sprint 21 the Day In on code deadline. source the his before very fix day score him the his helped improve team his not Instead, did Dur- much. he deadline. weeks, the next until the left were ing weeks few a still since hti a rsn eas e“atdt aeahigh a have to “wanted he score.” because present was and mentioned it developer code, that Another source helpful”. their “really of it quality found the said for also developers care But they education. that consid- their they of that part and it final score, ered developers’ the by the influenced that was is grade scores karma of presence Presence ex- AgileLab the to periment. compared positive more opinions much the are Generally, A. Team than better intervention feedback Questionnaire 7.11.3.3 Section 3.5.3.3). (see developers between documentation efficiently writing negotiated of is work divide additional they The so developer, labor. first the of the implementation as finish fast not as could functionality others Or idle. others are developer while one tasks, where implementation situations with overburdened in way is But viable cases. a most be in may problems it the fix best line probably in could was labor of division This had them. but for quality it code do source others who for developers care were themselves there not for teams did it both did in just So else [her].” Someone me styling. code about care not ntefia etn,Pml xrse htse“did she that expressed Pamela meeting, final the In h ototnnmdrao o high a for reason named most-often The I-1—Srma il Experiment Field ScrumLab — VII-11 cdn tl tteedo rjc”ad“ntels two last the “in and project” of end the at style “coding ihls otiuin.Vc es,tels h contri- the less the versa, Vice contributions. less with glLb n eeoesfudi acrt” w de- Two “accurate”. it found developers and AgileLab, certain that states guidelines coding own its in which eoesaddta Smtmsi a ar Sometimes fair. was it “Sometimes that added velopers eeoe,wote hne oehn otewre so worse, the second to a something from changed file then a who corrected developer, once she Another that more.” added work one to the have somehow you motivation advance, and contributors less less [the] bonus, see the and get contribute you not all more may after that I saw work I score. “as my better demotivating: a it get found even to One probability the higher the bution, developer a than points karma less collec- thus had and always errors, code tive source the most that the explained with subject developer A about was scores. problem of first computation the The scores. karma of fairness Fairness neutrally”. it to attention pay B Team to “tried from they developer that A noted deadline”. the before days about caring started “focused They and instead”. first” programming at the anything on do not “did B Team from hnIgt1.” tesfudte“crsral helpful really “scores the task, found my Others with better 10.0”. much got feel I I then time I this [...] 4.2, time got time”. last only the example, of “for “most remembers fair” developer was One that totally but not, seems it straightforward”. the pretty grading as] for [regarded “criteria were the code that, from Apart this unfair. criticized as developer One project possible. to was styles peculiarities coding of experi- adaptations both no in groups, standards mental the coding of standard comparability the For from styles. different is provid- This of instead accessors. public ing be should classes of properties Javadoc.” the in spaces were trailing points like often issues, So minor all.” “to at lost eye not human are that the others to about obvious whiny too was named it to poorly while as methods, easy such important, catch beings: are “didn’t that human Checkstyle violations But some by reuse.” use to to read, code to easy extend, clean is that “opinion, code his is in that explained Another problem developer quality: second judge the to standard So, ability the Checkstyle’s quality.” concerned ...] code about sure the so judging not of was he [... but score points. less got both they then rmnnl ebr fTa ,btas e ones few also but A, Team of members Prominently vrl,fins a uhipoe oprdt the to compared improved much was fairness Overall, framework, a using were team one in developers The the with satisfied “very was he said developer One h eeoeshdtopolm ihthe with problems two had developers The 201

VII — VALIDATION AND EVALUATION concern - -

0 0 0 0 0 0 + + ------

2.00 2.38 2.18 difference -

0 0 + + + + + + +

+ + + + + + + + + + + + 4.11 4.14 4.13

provement ult im- quality - -

0 0 0 + + + + + + + --

+ + + + + + + + 3.11 4.25 3.65

effort additional - - - -

0 0 0 + + + + + + +

+ + + + 3.56 3.29 3.44 inﬂuence ------

0 0 + + + + + + + +

Two developers felt that they had + + 3.00 3.50 3.24 acceptable - - 0 0 0 0 + + + + + + + + + + + + + + + + + 3.33 4.50 3.88 2.56

VALUATION E ﬁtting - - - 0 0 0 + + + + + + -- + + + + + + + + 3.00 4.13 3.53 2.44 The majority, however, felt that they had a good under-

lines of code”. Some developerssome noted time that but it at took last them ing, they “it understood. helped For to understand- closely look at several reports” which Understanding feeling of what they shouldthere take care is of a next coding time,there that standard provided is for the Java, Checkstyleinstall and an tool. that additional plug-in, and It thenrecommendations. just was follow also its One style possible developerdifﬁculties had to understanding because “he thought somehow theof score lines] of codes and errors in code”. standing because it had been detailed to them in an email. team member’s scores were formed”, the quality report would be a compromise between no. line [sic! number While a few “still didn’t quite get how the individual “calculated on the basis of the number of violations per “Not any idea about it.” Some others at least had a “helped to understand it”, and detailed how scores were interesting - -

0 0 0 0 0 + + + + + -- --

+ + + + + + 2.67 4.00 3.29 2.56 likable 0 0 0 0 0 0 0 0 + + + + + -- --

+ + + + 2.67 4.00 3.29 2.67

ALIDATION AND

standable under- - 0 + + + + + + + + + + + -- -- + + + + 3.11 4.13 3.59 2.11

VII — V important - -

0 + + + + + + + + + +

+ + + + + + + + 4.00 3.88 3.94 2.56 fair - 0 0 0 0 0 0 + + -- -- + + + + + + + + + +

3.11 4.14 3.56 2.33

Table 7.6: Likert-scaled opinions about the intervention in ScrumLab present 0 0 0 0 0 0 + + + + + + + + + + + + + + + 3.67 4.13 3.88 2.89 One subject noted little importance be- Max Norman Oswald Pamela Selina Steven Harvey Lex Bruce Clark Dick Matt Peter Johann Average B Average AgileLab Average A Victor Arthur Wally 202 important because “On the onethe hand, grade I of don’t the care lab about to at all, finish but a on place thefound other, lower that I than “it didn’t helped the want me topOther to developers 2.” improve felt my it Another code was also quality”. importantflect so on they their could work, re- andcode keep in the mind. documentation of their Importance cause he was focusing ongram. the But the functionality majority of of the subjectshighly found pro- important. the intervention Very important wascould for them improve that their they own and their team’s final grade and it feels like we achieved something during this lab.”. for Eclipse, so thathelped they them had to few get a check good errors, score. which They also liked that they could use a Checkstyle plug-in when taking care of the scores. For one developer it was

VII — VALIDATION AND EVALUATION soe h itksadhw[o mrv h style”, the improve [to] how and mistakes the “showed ol o aegie aypit fhsta a not had team his if points many gained have not would hyas on titrsigta h niiulscores individual the that interesting it found also They aesnefo neuainlpito view. of point would educational it an thinks from he sense that make said trou- neutrally some one team Another his caused bles. thereby he and he him, because helped it like not did developer One working are team.” who not the people in should the “it between while relations team, the effect the in could competition it a that or create computed, not were did scores they how when understand were CollabReview liking from ers Liking styles”. correct “do actually to and omneadhwmc ecnrbt sirlvn.But irrelevant. is contribute per- we our much of how aspect and one I formance only grades pressure. contest under this were that already realize they especially people, that some fact to the distress given of think lot “I a team: caused For one they onto well. pressure lab much the put fitting they not example, were scores karma the and Fitting its of them notified made it disadvantages. had because and code quality, their improve code them They source judge objected: Others could it not. could said interesting. it very opinion, it his find in would But he right, work his evaluate style. own one his if correcting ask when to question who a know had qual- to code was source advantage their Another to ity. attention more pay them made group. whole the effect could they score when their especially that grade; recognized final the in team scores their better helped how interesting score”, it certain found [a] developers in- achieve because and to “confidence hard them working gave for It spiration go bonus.” to team attempted the have could for score “low that a especially with and someone effect, motivating its of because ing Interesting quality. of care take the to encouraged members it group that in other and increase code, an source caused of it readability that the it about liked their They increase scores. successfully are could errors they how that about and fairness, measured, quality its the for and code, styles source their coding of about alert them kept it that n eeoe ttdta fh huh hti could it that thought he if that stated developer One liked Developers positive. were comments most But h otfeun rbeskeigdevelop- keeping problems frequent most The oedvlpr eotdta CollabReview that reported developers Some eeoesfudClaRve interest- CollabReview found Developers I-1—Srma il Experiment Field ScrumLab — VII-11 okn elyhr,te hr hudb something be should there then hard, really working ihtersoe,adoeo hmseiclypointed specifically them of one and scores, their with petaa rmpr ou nfntoaiytowards functionality on focus pure devel- from re-focused away somehow opment It code styles. source code about followed cared it and who that members liked team also the showed Developers to others easily. for more code understand source made documented it well So have code. them whole source the well-formed to making motivation for under- a team gave to scores others karma for The easier is stand. it then code, source quality him. for someone special is there if But work. group as counted be also the got contributions score.” of lowest majority the “The make [that] enough: guys honored not were quan- contributions that was titative as topic ridiculous recurring something A for whitespaces.” penalized trailing get coding to nights lab sleepless the spend for you if discouraging is it oei rtisac. nte n ute explained: further one Another instance.” clean first do a “to in trying code always were they that said labReview Influence the got also style. their developers corrected other she that after score fair maximum was it that out and satisfied truth”, reason were Developers the no measurement”. “reflects saw of and They “quality fair” its its did. “since it it reject that to team and the well; assisted only scoring it in if matter stated not developers did Some acceptance that score. own to one’s able in being progress and see In- effect motivating team. its the praised in they competition stead, causing for contributors, potential mass its against and discrimination with its problem for no saw except and it ex- statements also their in They this pressed CollabReview. with problem ceptance Acceptability react. learning to the had of they “part which as to a it experience” took to just according they code and write style, to coding how it fit- learn because it grade, to that lab them noted their helped increase Others could it it. because achieve ted to others him because helped essence” “in had score his exactly was quite score not his that mentioned Clark accurate, relatively focus. functionality-and-quality combined a te eeoesrcgie hti vroehsgood has everyone if that recognized developers Other should work group as lab the that said developer One hl n eeoe oe htClaRve was CollabReview that noted developer one While eeoeswofl iteiflecdb Col- by influenced little felt who Developers lotnn ftesbet a nac- an had subjects the of none Almost 203

VII — VALIDATION AND EVALUATION Most agree- VALUATION E For a few developers it only mattered to see their own Others, however, found that the quality still improved I said, I don’taroused care some for unexplainable the competitive lab spirit in grade, me.“ but these scores somehow created a little bit of a challenge. quality report was seen as a little abstract but was still Individual scores makement a was reached difference amongindividual developers in scores the made point that score a for difference the from team.ceived an as overall trying Initially, to karma separatethen scores collaboration the were on tasks per- teams but together. found Developers out used that theguy they scores whose to could score ”pick effected also the outindividual, team“. work the to There know ”it’s who better toIt blame helped :-) developers like svn toproblems. does see it.“ In who the had case and ofbeen an who much overall more had score, difficult no it to would locate have problems. karma score. But scores wereimprove also on used their by source developers code to code by of looking someone at with the a source to higher work score. hard It to inspired getthat them a ”he good wouldn’t score. have One cared developer atthing stated all that for kept the me contest. motivated wasexcluded The that from I the was bonus going if to I be didn’t do something. As Checkstyle. Theaddress accusations major were understanding violations”, that and it that their “does not things “which are unimportant fromof the view developer point [... butone were developer still] added: decreasing “I the had score.” one And of the highestlost scores points to issues that I considered unimportant.” In the guidelines of adecided development to framework use. that This conflict they perception has of certainly quality influenced improvements the in that team. because of Checkstyle and karma scores. It madecreating de- quality source code. Theythe were comments now with ”writing the methods.“code Consistent more style easy made to read,documented while and comments added made to it its better understandability. Only the velopers refocus from only developing functionality to Team A, some of the default coding rules were against Another one agreed that seeing the other ones’ scores working well. warnings “didn’t add more clearness.” It would check without doing a thing to improve my style and I only “code was more or less clear” anyways so Checkstyle’s took the ALIDATION AND IDE VII — V Most of the controversy if Developers who integrated their But developers who first developed and then later While in Team A the influence was that the last days Those who felt influenced named as reasons that they the code.” Quality improvement 204 fore committing it in svn.” However,iness. it was One not all developer finick- admitted that fixing Checkstyle source code quality had improved revolved around and helping “others in improvingcause their they code were quality” already be- done withalso their referred own to tasks. their They before breaking the of demonstration. the Especially, softwareidentified Checkstyle shortly as was the major culprit because of its finickiness, most load off of them.they The had rest to was dedicate a time normal to. task that styled their source code, reportedCollabReview high and efforts Checkstyle. through Developers reportedhave to “spent a whole day fixing Checkstyle warnings”, oper described that there wasthat no they feeling had of a a “really contest nice but collaboration in the team”. Additional effort style checking into their programmingit practice simultaneously by reported doing little additionalnoted effort. that They using styling plug-ins for their functionality, and to then take care of code style later. in the evaluation period had toing be styles, dedicated to developers fixing declared cod- that itdevelopment did of not functionality influence very muchthem a but lot it to helped write good quality source code. One devel- changed their behavior to now “develop code directly organized manner.” One developer also admittedactively to influenced have the others: “Ihave wanted a everybody high to score.” Otherthey developers, still however, said stuck that to their old behavior of first developing patterns and object-oriented paradigms. Icare need of to that, take but itbe will involved take in time the and process.” Checkstyle will not code style”, needed theor bonus found to it improve really their helpful for grade, their future job. They had my code. Icode, am and relatively the confident errors that ICheckstyle: tend I to field write and make method cannot clean naming, be usage caught of by design warnings was a “little bit tricky if you don’t understand which made some review their “code several times be- with style.” And it kept them “from developing in a unwanted to “write right code”, just had “to care about “I haven’t really changed anything in the way I develop

VII — VALIDATION AND EVALUATION htteqatt am crssupport. scores karma quantity the what would it that stated developer One well. it doing was the publishing course, Of important.“ is team a within h vlto itr fteqaiykrasoe clearly scores karma quality the of history evolution The the regarding results discusses and infers section This .141Sottr fet n atn influence lasting and effects Short-term 7.11.4.1 to published were scores karma if matter different a be they that styles code so correcting and was bonus“ who team someone everyone the that of for see could go scores to the were see we if to important ”was scores other each knowing my think let I level, to my natural know feel to should ”I teammate others or and feedback“, group my about [a] know are ”we have hide“, don’t to ”We of nothing explicitly lines developers was the all along it Almost something But issue: answered privacy a score. as others karma regarded make not their to for and bad competition, feel a the create were could given about it reasons only that concerned the scores, were their of they publishing that answered scores jects published about Concerns hw htteClaRve nevninhstein- the has intervention CollabReview the that shows also is point last this fact, imple- In functionality. just less had and mented organized, started less development bit getting a problems was their was A it Team Instead, that goals). impression bonus team teams the both reached Col- that had by information computed the scores for the (except to labReview on access rely have not not did weaker did they mark and assessment, one their about For be B. Team to than A Team graded lab the of data. the from CollabReview of reception and effects Results 7.11.4 them. an to for sent waiting being on report dependent updated being them of compute could instead we time“, so any [sic!] score compute to script agree.“ [sic!] not I’m scores... competition the a about have team but clear, the to not in needs is project that a code think admit I not very scores. it’s the code not the but of important, clearness the me CollabReview ”for the intervention: against reservation general a had Peter General 7.11.3.4 teams. both eea eako ohtasi htteinstructors the that is teams both on remark general A the have to nice be would ”it that mentioned Oswald I-1—Srma il Experiment Field ScrumLab — VII-11 hl w sub- two While a oad h ol emAwn o h maximum the for went A Team goal. the towards way issues most fixed and oriented deadline rather worked A aigteClaRve-nevninaa essource lets away CollabReview-intervention the Taking h rsneo am crswsmc ihri this in higher much was scores karma of presence The eeoes ttmns oesi htte a learned had the they with that congruent said also Some is statements: This developers’ team. learn- the to in some due effects be possibly ing to are seem which there effects, So lasting limited 15%. karma about quality by intervention, fall the only of scores al- end of the that multiple at a but amount to the level increases quantity original code their source though scores to karma down that not fall is abruptly It learning. effects, from of lasting resulting limited rate some e.g. the are secondly, there that But reveals decay decay. slowly quality code scores. in increase the for responsible proto- CollabReview was the which was type, actually it that support ther reach just to of threshold out necessary set the rather B Team while possible, score its worked steadily B Team deadline, the before shortly Team While teams. development both on effect tended motn yms eeoes edn oa average an to almost of leading score developers, most by important happening, was though. what reconstruct better general, could developers In rules. Checkstyle’s considering and without quantity, scores contribution of computing the problems to some relating were there a Still which of artifact. process contributors report, software major quality like detailed information included more also computed, and are larger scores a how and of explanation due better probably the is to This AgileLab. the since considerably earlier actions. started overhasty avoid B therefore Team could goal, and the to respect karma with quality the scores for late all rather caring While started present. developers scores making took for masters responsibility scrum full their and presence teams additional Instead, no was mechanism. In there only. emails occasionally the but to daily addition sent not with were emails report although the AgileLab, the in than experiment CollabReview of Reception 7.11.4.2 changed have not to style. claimed their Others style. their on keeping eye or an comments, writing immediately like things h otitreto eut o emAa rtfur- first at A Team for results post-intervention The nti xeiet h nevninwsfudhighly found was intervention the experiment, this In improved CollabReview of fairness perceived the Also 4 hc sa nraeo oeta one than more of increase an is which , 9 points. 205

VII — VALIDATION AND EVALUATION VALUATION E In addition to this, developers did not report an over- All developers had no privacy concerns regarding the But it may also be just hastiness. The team had few Primarily in Team A, the opinions whether or not Far reaching agreement existed among developers that emphasis on quality goals.a Team A bit was inept probably dealing just but with steadily the worked scores. its way Team over B the carefully threshold for the publishing of scores within their team. 7.11.4.3 Rat race Some form of a rat race may have occurred inthe Team A. team even brokestrator. its previously Of functional course, demon- thisin was front of a the very customers. badfected Also, situation their the right team karma members scores. per- have Only been scores necessary for of achieving ninethey the would fixed team each bonus and but every singletheir Checkstyle code warning base. in These findings could indicate that there time left for fixingthrough all several iterations problems. of firsttified fixing They previously problems could iden- and not waiting for go being the sent next to quality them. report Theytions. had So to they do just it fixed inbe everything one they the could. or reason two why This itera- one may developers developer direct recommended access to to give the computation of scores. or when doing stylingcase in Checkstyle’s warnings parallel were to ignoredproblems coding. for occurred too later long, Only on. in could Hasty break last the minute project actions and lead to developers spending quality had improved were controversial.this, In members contrast of to Team B consistentlyor reported much improved improved source codeuttered quality. concerns The about developers the suitabilitystill of found Checkstyle, but that it ledpromoting to descriptive comments more and readable consistent source style. code by individual scores made a differenceoverall as score compared for to the an entireoverall team. scores The differences were from thatcompetitive they spirit awoke to an achieve unexplainable bettercould scores be used and for quality, pickingthe out team developers who score, damaged and couldothers. also be used for learning from was reckless perfectionism occurring. whole days fixing style problems. When fixing all style problems right before the deadline, ALIDATION AND VII — V Many developers reported defensible additional effort The majority of developers did not consciously feel Acceptance has much improved compared to the Ag- CollabReview fitted much better into the Scrum en- Few improvements were made with respect to interest- There was no improvement in the liking of CollabRe- Understanding is still one of the weaker points but 206 eral developers emphasized that thehave intervention a did lasting not effect onnonappearance them. seems The very main much reason to fordid be the not that agree developers with Checkstyle’s rules. through CollabReview, especially when using style tools CollabReview’s motivating effects were praised andceived per- as being fair. neither a strong positive norCollabReview. negative influence While through objectively the effectis on evidenced quality in the karma scores of all developers, sev- source code, so itDevelopers also is felt mostly that they seen werefrom as learning it, a something which suited fitting the support. teaching perspective. ileLab. There were no severeonly acceptance problems. issues The raised wereand the discrimination against potential mass for contributors. competition Instead, tivational aspects for the individual.adds CollabReview also a source of informationbetter for understand developers the so team, they and can what is happening. helped the team to attain the grade bonus through better increase their score. ingness. Being already on aileLab, rather it high did level not in increase theDevelopers not Ag- find as it much interesting as for otherresulting its aspects. from motivational effects the social group-context, and various mo- tified problems were thatinside it of caused the some team, that competition and it unpredictable still thing was due a to somehowit, problems unknown and understanding that developers couldcolleagues feel indebted when to they their fixed team their problems for them to it became much better,understood it and sufficiently subjects well. foundscores Exact that is computation still they of aknew bit what to unclear take to care them of to but get all a eventually better score. mark compared to theprimarily AgileLab. rooted in High the grade importance bonus,it is but important some because also of found reflection and learning processes. vironment than into a pair programming scenario. It very much. No one in the team did not like it. The iden- view for Team A since the AgileLab, but Team B liked it

VII — VALIDATION AND EVALUATION ihu aigt oayhn manually. anything do to having without intervention the saw they programming; were low they was while effort additional the report: quality the with ( emAcue tefsm rbesb rigt fix to trying by problems some itself caused A gain Team quality lower a at effort higher a estimated A Team ohtasas on hr a igrices in increase bigger a was there found also teams both ( CollabReview addi- by some caused was effort there tional that estimated together teams Both improvement quality versus Effort 7.11.4.4 fixing by or code. tools, source right others’ the each with ei- help scores, providing necessary by the ther reaching in other started developers each contrary, helping the to Quite occurred. have rat occur. a not that did therefore the race is have conclusion did The member score. team maximum every not and bonus; team rmwr,wihhdsm tl eomnain that recommendations style some had development which a un- framework, using major were address They to failing violations. was derstanding instead clear- it to that adding and not ness, Check- were the received that they said warnings They style Checkstyle: of doubts benefits greater the had in they that indicates furthermore ment before in. times it several checking it reviewed and code, manually source warnings, fixed Checkstyle A Team fixing in days Developers whole started late. spent rather and problems tools, their less fix used had to they they that approach: direction different the feedback a in The points also A deadline. Team the from before right problems all that considered be to has it But CollabReview. view.through of point economic an from beneficial been have problems and most to, fix time immediately dedicate to tools to styling had used they they that task normal a as away, right problems style potential fixed they because dealing to approach planned and relaxed a had indicates they developers that the from Feedback eminent. more the costs. than that benefits at more hints brought This intervention CollabReview improvement. quality in plus the ( quality 3 . edakfo emArgrigteqaiyimprove- quality the regarding A Team from Feedback to seem not does rather intervention the A, Team For effort additional between difference the B Team In to seem not did non-performers of pommeling Also 29 oprdt ult mrvmn ( improvement quality to compared ) 3 . 65 .Tepu nadtoa foti esthan less is effort additional in plus The ). I-1—Srma il Experiment Field ScrumLab — VII-11 3 . 44 4 . .However, ). 25 seven is ) Apa oidvdas motivation”). individuals’ to “Appeal a esognzd n htsm ao contributors major some that and organized, less was A hc rvn orecd htfisCektl tests Checkstyle fails that code source prevent which Rqieet4 N aadt h team”). the to hazard “No (Requirement-4: quality”). software internal “Improve (Requirement-1: understandable”). “Easily Requirement-9: (e.g. w edeprmnswr u nsfwr development software in run were experiments field Two emA sopsdt titcmi ue,eg scripts e.g. rules, commit strict to opposed As A. Team surpassed even or reached were set goals quality The eto otl eta enwatal ol appreciate would actually now he that me tell to vention eada hl em r fte aldfrti goal, this for failed they if or, team, team whole a a win as either reward could teams Both into environments. brought both was prototype CollabReview The projects. Summary 7.11.5 for results favorable CollabReview. interview more given later have A therefore efforts their questionnaire. might of the rewards out the filling reap to before time A Team the have problems, quality not to internal did their due of that fixing probable late is the It quality. in improvements the inter- their of end the Interestingly, after weeks me cowboys. approached coding Clark like more were rather Team that impression the in- had the ScrumLab Also, the of styles. structors coding standard the from differed nevee ujcsmd orsetv eak,too remarks, respective The no found. made not subjects was interviewed evidence A, Team in occurred have (Requirement-2: tasks fixing distribute style to and or implementation problems, of fixing postpone developers to grants flexibility CollabReview in, checked being from in intervention of end the beyond seen be could effects, dependence”). without contributors “Involve (Requirement-6: quest team’s quality their for in engaged somehow was Everybody per one than less decreased little violations a rule of from number The teams. both by correctly. employed is it if internal project, the software on a showed have of It quality can CollabReview success. that a effect was the experiment Check- by field measured The as code style. source of quality the crease dimensions interviewed all on neutral The than better CollabReview scored reward. well. the rather perceived win was could intervention team per individuals two hl hr a oeidcto htartrc might race rat a that indication some was there While learning to due possibly influence, lasting Limited in- considerably could CollabReview teams, both In SLOC oams zero almost to 207

VII — VALIDATION AND EVALUATION ). , in 60% ≈ svn blame VALUATION E Experiments to validate the more human and social The first field experiment with real users was the “Mo- Under the assumption that willingness to invest in in- “Consistency of Karma Scores” (Section 7.3) aimed group design, and succeeded inquality increasing as contribution well as quantityOther (by important up results to were as insights muchcesses into as through the interviews social and pro- observations, and the non- ternal quality is a ratheronly constant should trait fellow of developers developers, have not afor rather a good developer’s feeling contributions but itin should other also documentation reflect media likelogs, Subversion commit or the documentation wiki.gated This in issue was “Validity of investi- Computed Karma Scores” (Section the developers’ reputation among their peers, and their aspects of CollabReview were initiated bythe a expert summary reviews that of were conducted several times in opinions offer the opportunity tolabReview virtually cheaply validate and Col- in environments whereaccess normally would not havetended been to lead possible. to heated CollabReview received discussions, positive. but was Issues generally identified byeventuated, experts which actually could argue for thethe transferability experts’ of assessments.equally CollabReview suitable for may all not environments. be knowpedia Field Experiment” (Section 7.6)prototype , was where integrated the into amanagement work wiki. group’s knowledge The experiment was set up as within- off, in the worst case 30%the best worse than case 30% better.tween It but is with probably a some value tendency in to be- the better case becausepredecessors the than Subversion, and canmoving, roughly copying handle and reformatting. at reducing the uncertainty leftstudy. behind If in developers have the rather previous consistentit should coding be styles, possible to predict the internal quality of soft- scores. The laboratory analysis comparedcomputed karma scores with several algorithmsprojects. The applied results to were consistent several correlation with for a CollabReview’s algorithm; substantial similar results could be obtained withtoo, some but of choice the of other theprototype algorithm should algorithms, also for consider the tampering CollabReview resistance. 7.4) . The results found that karma scores well reflected validation showed that the algorithm determines better willingness to contribute to other documentation media. ware process artifacts from correctly computed karma “Analytical Concept Evaluation” (Section 7.5) . Expert ALIDATION AND VII — V Responsibility established through authorship for text The studies that are presented first are more concerned Some developers found that CollabReview further In Team B the gain in quality through the intervention 208 tion 7.2) investigated if theresults. algorithm It generated had sound to befind tested suitable with predecessors regards to of itshandle artifacts, moving ability (and to and copying) if ofIn text it between a could artifacts. laboratory analysis withand the FreeCol, source it code returned of a Karakos of different author lines for of about text. 30% That means that its results are not totally in software process artifacts isscores a in key CollabReview. ingredient In to order karma authorship to has obtain to karma be scores, determined.ponent A is technical therefore key the com- algorithmtation. for authorship First compu- validations of thesented authorship in algorithm pre- “Evaluation of Authorship Algorithm” (Sec- type. Later studies focus on the integration of CollabRe- business environments. While it isscribed not topic an of explicitly de- the experiments,always technical a viability matter was in the early studies. 7.12 Overview of Studies dressed in ten individual studies.the Table 7.7 key facts summarizes of the conducted studies and aligns them purpose of CollabReview. It isto supposed quality to which give may a otherwise voice be forgotten. increased the pressure on them,through which the was Scrum-based already development. high Nowadditionally they had take to care of sourcemay code be quality. true, While this exactly this personal responsibility is the this may be due toproblems the rather fact late that they leading hastilycould to have fixed prevented slips their them of from enjoying the thefore pen. benefits answering be- the This questionnaire (Requirement-8:overhead “Few for users”). Furthermore, onement of the frameworks develop- they decidedcoding to style use recommendations had as discussed conflicting in Section effort associated with CollabReview. Team Aslightly reported more effort than gain from the intervention but view concept and prototype into actual development and 7.11.4.4. The validation concerned 9 requirements that were ad- with the validation of requirements. with the technical validation of the CollabReview proto- was estimated considerably higher than the additional

VII — VALIDATION AND EVALUATION eurmns =mrv nenlsfwr ult,2Apa oidvdas oiain =lxbeapplicability, 3=Flexible motivation, individuals’ to 2=Appeal quality, software internal 1=Improve Requirements: AgileLab Analytical of Validity Al- Authorship (7.9) (7.8) (7.4) (7.3) TornBond am Scores Karma of Consistency (7.2) gorithm of Evaluation section) (and Study et(7.11) ment Experi- Field ScrumLab (7.10) ment Experi- Field Castigator MicroEdit- Field Experiments Forge Reputation- Ex- perimentation Design (7.7) ment Experi- Field (7.6) ment Experi- Field Moknowpedia (7.5) uation Eval- Concept Scores Karma Computed =ohzr oteta,5Rbsns n eitneaantatcs =aiiyo opttos 9=Easily computations, of 7=Validity attacks, against resistance and 5=Robustness team, the to hazard 4=No nesadbe =e vredfrues =nov otiuoswtotdependence without contributors 6=Involve users, for overhead 8=Few understandable, ok ihsource with works concept that Validate (1, programmers with validation Algorithm 1 ,6) 2, (1, social) all (implicit, (7) requirements) (and etsca rewarding social Test field in prototype Test with prototype Test oe(,2 ,6 ,9) 8, 6, 4, 2, (1, code 2, 6) 3, (1, behavior able undesir- suppressing by validity concept Show 6) in 3, 2, (1, project and game test failed after design and prototype Evolve 6) 8, 4, 2, 6) 8, 4, (1, 2, wiki a of users real improvements and design mitigation Risk are (7) valid hence and social karma reputation reflect that scores Show scores karma through quality of prediction by 7) 5, (3, ness correct- algorithm of validation Preliminary Purpose al .:Oeve fcnutdstudies conducted of Overview 7.7: Table EU I-2—Oeve fStudies of Overview — VII-12 ru rewards group & ihngop observa- group, within data group, within evaluation Analytical predict Analysis: snlg wiki) log, (svn ewe n within experiment; and between Field gathering data tion, experiment; Field data interviews gathering, groups, tween be- experiment; Field learned lessons field; collect to observation in prototype with Experimentation interviews gathering, experiment; Field interviews ing, gather- data servation, ob- experiment; Field experts by concepts of proxies (2) interview reputation (1) to computed karma Compare from karma quality artifact comparison blame svn goodness, predecessor analysis: code Source Method neves individual interviews; gathering, data groups, workshops AgileLab AgileLab, Vyrygh, AgileLab TornBond, TornBond, TornBond virnmnts. Expert pedia, Moknow- Log4j FreeCol, Karakos, FreeCol Karakos, en- Study ScrumLab pedia Moknow- pedia Moknow- pedia Moknow- ayn suitability varying rep- social to similar valid; algorithm of choice valid, ulise,transferability, issues, actual of identification reception, early Controversial document to ness willing- reflect and utation are scores karma Computed substantial ity capabil- predictive matters, algorithm consistent, karma robustness search, copy/move predecessor good authorship, different 30% notes & Results iil des effects adverse ligible neg- deniers, no concept, of reception improved much reached, fully goals Quality behavior recalcitrant show re- by duced behavior Undesired limited is of game ReputationForge benefit additional but improvements, Significant unfair quantity ing neglect- information, tune karma, negative avoid tal, vi- value karma users, Brief play inter- socio-technical into insights effect), (no Failed anecdotes effects, bad no improvements, tion contribu- results, Promising 40% oeusers some , 209

VII — VALIDATION AND EVALUATION over the course of 40% ’s ratings. prove internal software quality” and Concept Evaluation VALUATION E Table 7.8 repeats the requirements’ titles and their fit The suitability of the CollabReview concept for the In addition to this, validation of the requirement has The “ScrumLab Field Experiment” (Section 7.11) cles increased by 40%creased to by 60%, about their 20%, average and sizedas about the in- half content a was point perceived ter better the on field a experiment 5-point (Section Likert 7.6). scale af- ReputationForge for the CollabReview concept and prototype.for It each discusses requirement if oron to results what of extent the it studies is presented met in based this chapter. criteria. The last column containsthe an requirement estimate is whether met as discussed below. 7.13.1 Validation of Requirement-1 “Im- Fulfillment of Requirement-1 is determinedrion by “There the is crite- an increasedocumentation in created the by quantity the and development quality team”. of purpose of improving theprojects internal has quality been presented in to several software expertsengineering. of software The majority deemedpurpose it (see Section suitable 7.5). for this been addressed in several fieldknowpedia experiments. In field the experiment, Mo- the number of new arti- could be reduced by asseveral much months. as However, some users showedrecalcitrant enduring behavior objecting against andMicroEditCastigator countering labReview in a programming environment.combined It within- featured and a between-groups experimentalsign. de- The rewards had ascores, high and exchange combined value group for karma andamplify individual exchange rewards value. to The resultsNot were only very were positive. the quality goals fully reached buteffects also were almost non-existent. Therebut were groups no engaged deniers, in division ofmembers labor from to unloved alleviate activities. some 7.13 Validation of Requirements This section reiterates the requirements that were defined was set up as a second attempt to trial the improved Col- was the reception of the prototype much better. Adverse ALIDATION AND VII — V The “MicroEditCastigator Field Experiment” (Sec- As a result of the AgileLab experiment, more empha- The “TornBond Design Experimentation” (Section In the “AgileLab Field Experiment” (Section 7.7) the 210 tion 7.10) served thebreaking purpose a to trial bad CollabReview habit in regardingrevisions the checking in in Moknowpedia. of article Tooof frequent revisions adversely committing affects theof history documentation an article andcan the therefore wiki increase as internal agroup quality. whole. design, Using the Suppressing within- study it showed undesirable behavior pedia and the wiki oftistically Vyrygh. significant A considerable positive and effect sta- was measured for game elements originally used inever, Moknowpedia, not how- all improvements were significant. sis was put onscores. the The exchange ReputationForge prototype value implemented offeredcomplex a social for game karma that wasexpensive to tangible substitute rewards. potentially TheField Experiments” “Reputation- (Section 7.9) Forge trialed the prototype receive a detailed briefing upfront, that the exchange karma scores like technical debtinformational should load be should avoided, that be tunedand to that the neglecting users quantity in needs, perceived the as karma unfair. scores can be the previous failed experiment.study There methodology was but no ratherof formal iterative the improvements prototype’s design followedhow it by affected observations developer of behavior. Thetaught experimentation several lessons, for instance that users should because of profiteering, understanding howcomputed scores is are important for workingIt towards seems high that scores. CollabReview is notprogramming, compatible which with obfuscates pair responsibility. environment with source code. However, thefailed experiment because there wasdevelopers. no Interviews measurable conducted effect with on theat the the developers end of thekarma evaluation period scores correlated and with provided their deepof insights CollabReview; into e.g. the low effects ranksception tend of to presence displace but the increase per- importance, CollabRe- appearance of fatal negative effects. Furtherto experiments repeat the success could follow. previous success should be replicated in a programming value for karma scores is vital, that negatively connoted 7.8) served the purpose to optimize the prototype after view is welcomed by quality-oriented developers but not Vyrygh. There was an increase compared to simpler with a between-groups experimental design in Moknow-

VII — VALIDATION AND EVALUATION a rte seScin7.11). Section (see written was ie,silqaiyi uhmr ope concept complex more much a is been quality has still measures both given, number for Justification the edits. and micro Checkstyle of i.e. used experiments, were that some quality in of assessments automatic the to suc- concept cessful. CollabReview the of employment make circumstances that the of understanding the Both have improved studies not experimentation. TornBond did the it similarly, in change; influence a much cause not did experi- it AgileLab ment, the In environments. development all documentation inline more that means also violations of little per a one from than decreased less violations of number The development software teams. both were by surpassed goals even quality or reached set the the experiment in field Finally, ScrumLab quality. repository increased by sequently behavior undesired of amount The the 7.9). im- Section MicroEditCastigator further (see and Moknowpedia Vyrygh, of in wiki provements the in articles cre- of and ation quality quantity, in increases significant caused tas a ob oe htntalraesmgtagree might readers all not that noted be to has also It in work not did prototype CollabReview the However, 9 8 7 6 5 4 3 2 1 # aiiyo computations of Validity motiva- individuals’ to Appeal aiyunderstandable Easily users for overhead Few dependence without contributors Involve resistance attacks against and Robustness team the to hazard No applicability Flexible tion ity qual- software internal Improve titles Requirement SLOC oams eo h reduction The zero. almost to I-3—Vldto fRqieet n ocp Evaluation Concept and Requirements of Validation — VII-13 al .:Smayo eurmns aifcincriteria satisfaction requirements’ of Summary 7.8: Table edeprmn reduced experiment field ≈ h rttp osntrqiebodue upr from support user broad require not does prototype The robust be should system reputation its and prototype The to hazards of manifestations weak only or no are There determination the to regard with flexible is prototype The docu- of quality and quantity the in increase an is There eiinrpstr,sz ftepoet itiuinof distribution mechanism rewarding project, the the and development, their of and size artifacts repository, of kind revision artifacts the of method, assessment quality the of and assessment the responsibility, of sr nesadtepooyeadhwt mrv the improve artifacts to of how quality and prototype the understand Users users to caused is overhead direct low Only valid are resulting scores the karma and information, effect quality its and shun Responsibility cannot contributors individual and start, the attacks against resistant and team the choice a have still they i.e. comply, to forced not although are behavior they desired the of more show Developers team development the by created mentation criterion Requirement 40% n con- and , Dvlpr hwmr ftedsrdbhvo although behavior desired the of more show “Developers eiinlgmsae utemr,cnrbtr r free are contributors Furthermore, appropriate message. an log without revision artifacts process submit- software reject of or all ting committing, fix before to style users force code example, source for not, does It informative character. only has does What prototype scores. CollabReview karma the like information provides further and with interaction, them for ways additional repository, users revision gives the from information prototype evolution The reads processes. development the with a terfere have still they i.e. comply, to choice”. forced not are they condition: following the under met is Requirement-2 “Ap- Requirement-2 of Validation 7.13.2 to back refer quality 3.2. of Section of discussions purpose For the Perfectly research. not this is too. quality matter, measuring subjective and a modeling cases some in and ydsg,teClaRve ocp osntin- not does concept CollabReview the design, By h eurmn scniee sflymet. fully as considered is requirement The elt niiul’motivation” individuals’ to peal Fulfilled + + ++ ++ + ++ ++ ++ ++ 211

VII — VALIDATION AND EVALUATION VALUATION E quality, quantity or others.nisms Several have been rewarding implemented mecha- intotypes the used CollabReview pro- in the field experiments. They range from of project. However, larger projects have higher hard- therefore only required to worksize with projects. small- to This mediumhas been validated in Section 7.2. 7.13.3.5 Distribution of development CollabReview prototypes do not require co-locationdistribution. or In fact, most communicationshould like occur reviews anonymously through the prototype any- location may be beneficialrewarding for systems. some Prototypes reputation-based have beentrialed successfully with co-located and distributedple, teams. ScrumLab For (bi-weekly exam- meetings) andReputationForge Vyrygh experiment in were the distributed projects in Moknowpedia and AgileLabtions were 7.6 co-located and (Sec- 7.7). 7.13.3.6 Rewarding mechanism currently implemented in theallow prototype, for its easy extension interfaces withtool other software kits. metrics or 7.13.3.3 Supported kinds of artifacts CollabReview is designed forsoftware collaboratively process created artifacts thattion. have a In text its representa- lar current programming form, language it but ising it not algorithms relies bound at on to the string granularity akey-measures edit- of particu- like lines timeliness to and determine gives responsibility. the CollabReview This prototype the abilitymainstream to programming support languages like Java or C#, or revision repositories of Subversion, GForgeMediaWiki. wiki, Other and kinds ofcan software be process supported artifacts byalgorithms, switching and to providing importers appropriate for editing revision the repository. respective 7.13.3.4 Size of project The CollabReview concept builds on karma scores for The CollabReview concept should work with any size (Sections 7.11 and 7.9). As opposed to this, contributors ware and implementation requirements. The prototype is ways, so co-location is not necessary. However, co- wiki markups like that from MediaWiki. It can access ALIDATION AND VII — V that results from the ible applicability” social indirect responsibility Still contributors in several field experiments (see Sec- 212 form of reviews with ratings. Multiple ratings from re- process artifact. By default, reviewsby are their only timeliness weighted but otherpossible. integration algorithms Reviews are can be submitted by users, through like Checkstyle. While few automatic assessments are needs of software projects. Assessments are made in the changes to a softwareported process Java artifact packages as recently, described or in Sectionresponsibility im- 7.3. metrics Further not directly based on contribution ple, social context of the development environment. 7.13.3.2 Assessment of artifact quality sibility metrics, including its ownOther authorship responsibility algorithm. metrics are contributors who made method, the kind of artifacts andsize their of revision repository, the project, distributionrewarding of mechanism”. development, and In the all ofthe the CollabReview named concept dimensions and prototypeible. are highly That flex- should makeprogramming it environments. adaptable to many different 7.13.3.1 Flexibility of responsibility 7.13.3 Validation of Requirement-3 “Flex- flexible with regard to the determination ofthe responsibility, assessment of quality of artifacts and the assessment tions 7.6, 7.9, 7.10, and 7.11)and showed more less of of the the desired undesiredprototype behaviors. The has CollabReview addressed andmotivation, resulting altered in the an contributors’ increase in internal quality. to amass any amountto. of Contributors are bad not karma forcedsupported points to through comply they the with CollabReview wish the prototype. rules views are integrated into the quality rating of a software view prototype can be adapted in various ways to the The quality assessment method used in the CollabRe- The CollabReview prototype implements several respon- The criterion for Requirement-3 is: “The prototype is The requirement is therefore fully met. web services, or automatically through software tools were discussed in Section 5.2.3. It describes, for exam-

VII — VALIDATION AND EVALUATION :“hr r oo nywa aiettoso hazards of manifestations weak only or no are “There 4: Scin76 rSrma Scin71)atal stated actually 7.11) (Section ScrumLab or 7.6) (Section h olwn rtro a endfie o Requirement- for defined been has criterion following The iwerjce otk ati td eas ewas he because study a in part take to rejected viewee danger- inherent the to pointed reviews Expert view. .338Summary 7.13.3.8 less- were projects. agile with planned, tested that were projects All prototypes necessary. CollabReview is research difficult More is obtain. partners to pair both re- of form, information pro- current sponsibility pair its in to because applicable environments not gramming be to seems CollabReview Limitations 7.13.3.7 ReputationForge complex game. social the to boards leader simple ati h akn al,o eedsorgdb the inter- by One discouraged 7.6). (Section were contributors or major of table, being progress ranking enjoyed the ratings, in bad last articles others’ jokingly each users gave Moknowpedia, in In early dynamics process. social research on the based concept a of ousness CollabRe- with problems have might users cumstances intervention. the liked they Moknowpedia that e.g. of users interviewed The pro- CollabReview the the totype. for of employment hazards the of through manifestations team exper- of field free all were indeed, iments And reviews. con- bad the of fear sequences to having from reviewers protects concept team”. the to “No Requirement-4 of Validation 7.13.4 re- met. the fully exception, is this quirement for Except responsibility though. acquiring pro- information in pair problems for to suitable due seem gramming not does It reward- mechanism. the of ing or size development, of artifacts, distribution process project, software the of kind responsibility, the like quality, aspects many to regard very with are flexible prototype and concept CollabReview the Both oee,sm tde eeldta ne eti cir- certain under that revealed studies some However, CollabReview the reviews, anonymous its Through aadt h team” the to hazard I-3—Vldto fRqieet n ocp Evaluation Concept and Requirements of Validation — VII-13 Scin76.The 7.6). (Section eldta hyswters htuescudattack could users that risk the saw they that vealed y Tepooyeadisrptto ytmsol be should system reputation given its and is prototype Requirement-5 “The by: by demanded as Robustness “Ro- Requirement-5 of Validation 7.13.5 requirement The there. is risk the fulfilled. that is noted be But to presence. has its it through project software the endanger while scores. software top their reach broke to they trying some when from race, rat suffered of have In form might respectively). team 7.11 one and ScrumLab, 7.8, honored 7.7, not (Sections was quantity enough because unfair treated a felt contributed lot, who developers that revealed ScrumLab and TornBond, AgileLab, of 7.4). experiments (Section Programming computed being scores about concerned takdb one-eiw ihvr odratings good very with counter-reviews by attacked reviews generating automatically user by a system Moknowpedia the In attacked experiments. the and in seldom severe not were prototype CollabReview attacks However, the against 7.5). (Section system reputation the works. it how be out can figuring quality by of attacked assessment automatic the pos- Also, reviews is sible. one’s behavior malevolent having detect or to repeatedly, articles rejected own of instance biases for behaviors, rating reviewing of analysis further Theoretically, the objectiveness. add and can rejected, metrics be software can reviews pro- Unwarranted software of artifacts. ratings cess a quality only the has on user influence Every limited attacks. to susceptible more fore forge to contributors information. for responsibility difficult it de- makes is This reformatting created tected. Also are temporarily. artifacts code, only process removed of software or copying if detects detects It it resistant. and and robust designed be been to has algorithm authorship The authorship. two ratings. takes quality It and responsibility principle. input: in of attack kinds to easy not CollabReview is The concept attacks”. against resistant and robust vral nnn fteeprmnsddCollabReview did experiments the of none in all, Over h nltclcneteauto iheprsre- experts with evaluation concept analytical The there- and resistant, and robust less is input quality The through defined is responsibility prototypes, the In utesadrssac against resistance attacks” and bustness MicroEditCastigator was 213

VII — VALIDATION AND EVALUATION different from the 30% command. This difference blame overhead for users” lidity of computations” VALUATION E Third, the contributors’ karma scores were compared Fourth, developers in ScrumLab felt that scores were While karma scores can certainly be improved in their First, authorship information obtained through Col- Second, consistency of the computed responsibility the following criterion: “Onlycaused low to direct users”. overhead It is has to be noted that the CollabRe- to their social reputation fortheir contributing disposition to to the write project, commit log messages, and their found here, too. The weakerkarma correlation scores between and quality the social reputationinates in from TornBond a orig- low acceptanceenhancement of tool Checkstyle (Section as 7.4). a quality understandable and fair (Section 7.11).cates that This they also are indi- valid because otherwise interviewees the CollabReview prototype had problemsright making responsibility the attributions (Section 7.7). precision, they have torequirement is be fully considered met. as valid. The 7.13.8 Validation of Requirement-8 “Few 7.13.7 Validation of Requirement-7 “Va- Requirement-7 is satisfied if “Responsibilityinformation, and and quality the resulting karma scores are valid”. labReview’s authorship algorithm was analyzed.found It was that its results areclassical about Subversion an improved determination of softwarepredecessors process (Section artifact 7.2). information was validated by usingdictors karma of scores software as process pre- artifactsubstantial quality. significant Moderate correlations to were foundassessed between and predicted quality ratings.results Furthermore, are the comparable in precisionmethods to like other using prediction the reputationpredict of quality (Section Java 7.3). packages to view prototype causes additional effort by making the The decision if Requirement-8 is fulfilled is based on wiki karma scores. Significant positive correlations were would not agree to them.with Developers Checkstyle only had as problems aAgileLab quality was not measure. so high Agreement because in in pair programming was attributed to its abilities of detecting copies, and ALIDATION AND VII — V dence” volve contributors without depen- CollabReview concept and prototypes are vulnerable Still the CollabReview concept is vulnerable to broad In some environments, the CollabReview prototype It is not necessary that all contributors actively use the It has to be expected that with an increasing exchange 214 eral one: without an exchangeCollabReview does value not for work. karma scores, to weak exchange values in thethey rewarding fully system. meet But the requirementall of contributors. being involving for nisms. A critical mass hasis to no be social supporting group it: toscores, If give then there an the exchange CollabReview value concept to does karma not work. is less of a bootstrapping problem but more of a gen- reviews of software processtherefore artifacts. motivated reviewers by Moknowpedia offering bonus points to get user involvement started. rejection. This is specifically a problem when the re- totype to compute karma scoresautomatically anyways. part So of users the are game andeven the if rewarding not system, them but only others actively participate. is dependent on more assistancestance from the in users, Moknowpedia. for in- They are needed to provide and individual contributors cannot shun its effect”. CollabReview prototype from the start.they There need is to nothing install, andinteract even if with they CollabReview, they do still not interactrevision consciously repository. with This the enables the CollabReview prototype does not require broad user support from the start, tacks will increase. In thean experiments, issue this though. has CollabReview not therefore been compliesrequirement. to the 7.13.6 Validation of Requirement-6 “In- the Checkstyle quality assessment by providing empty Javadoc comments for some entities (Section 7.11). value for karma scores, the number and severity of at- This has been the problem inTornBond (Section AgileLab 7.8). (Section 7.7) Yet insufficient and exchange value The fulfillment criterion of Requirement-6 is: “The pro- (Section 7.6). This helps as a bootstrapping mechanism (Section 7.10). And one team in ScrumLab attacked warding system builds only on social reputation mecha-

VII — VALIDATION AND EVALUATION a ecie smsl ossigo fot osatisfy to efforts of consisting mostly as described was reduces further assessments quality automatic Applying h rtro eemnn ufileto Requirement-9 of fulfillment determining criterion The minimize to web the through accessible is it Therefore h edfrfebc rmusers. from feedback for need the the into wiki ins the development into contributors’ widgets the as environment into integrated be forms Review can side. it. contributor the use on not efforts installation will they time, CollabReview their the if users that costs is rationale prototype The users. its for empirically. is and causes analytically CollabReview analyzed that what CollabReview overhead the only The of is use prototype. Overhead the with overhead. directly not associated is is This would do. normally not he that things do developer individual tesddnt h rbesmgthv enta they that been have might problems The not. computed, did were others scores they karma that how out understood pointed probably ones Some understanding. least the bit. a them they with computed, experiment were to order scores needed In how scores. in their confidence improve gain to to they order what in and do computed to were needed scores understand- karma basic how a of had ing usually users experiments, all in improve artifacts”. to of how quality and the prototype the understand “Users is: “Eas- Requirement-9 of Validation 7.13.9 require- the to ment. complies fully therefore It contributors. 7.6). (Section automatic cost the the no worth considered provide effort was too, additional they, to there But had mechanism. as Moknowpedia assessment themselves in efforts reviews in users than the The quality in gain greater spent. a found was contributors there that, that from quality, Apart in overheads. spent effort on is not which rules, Checkstyle the effort all this But 7.11). (Section effort created additional prototype much CollabReview Scrum- the the that in reported Contributors Lab reports. users’ the through atcpnso h glLbeprmn rbbyhad probably experiment AgileLab the of Participants perfect not was prototype the of understanding While the to overhead little very only causes CollabReview empirically found overhead low only was There overhead low only cause to designed is CollabReview l understandable” ily IDE oacs ote sqikadeasy. and quick is them to access so , I-4—Eauto fCollabReview of Evaluation — VII-14 GUI ra plug- as or , eecmue.Hwvr h xc a o karma how way exact the However, computed. were ihteClaRve ocp n rttp oconfirm to prototype and concept CollabReview the with de- the during distilled were requirements nine of set A as ee akt al 7.8). Table to back refer (also h ebr ftedvlpettasi ScrumLab in teams development the of members The h eerhtei sta aicto hog com- through gamification that is thesis research The deter- to requirements all through iterated section This .31 umr frqieet valida- requirements of Summary 7.13.10 satisfied. is requirement their The better. influence the to to how scores well the karma rather experiments, knew the usually in users good very not was labReview 7.11). (Section to scores what their knew improve they to But order done. in was do they it so how them, know to not explained did not was computed are scores scores how for feeling good rather a had they that said 7.6). (Section articles their influence rating could by scores they others’ that each knew it, about ex- asked not although plicitly users, Moknowpedia them, As to 7.7). opposed (Section made unreliable which computations programming score their karma pair improve doing to were tried and really score, not feedback, few too got ettas sn ope oilgm srewarding as game social complex a Using teams. develop- distributed ment in Similar recorded also docu- perception. were users’ of improvements the quality in the increased Also could mentation documentation increased. substantially general be with software of artifacts number the process team, co-located a In promising: 7.13). Section (see requirements of fulfillment the conducted were studies Several requirements. into of elaborated set then was a It product. of software quality whole and the documentation internal themselves, better artifacts in process resulting software for care to more developers ac- take motivate an can and system, scores, rewarding reputation cording compute that tools puter CollabReview of Evaluation 7.14 met fully or met been have requirements all that The is studies. result conducted satisfied the as from considered results the be summarizing can requirement the if mine requirements. these fulfill to crafted concept were CollabReview prototype the and Next, research. this of sign hl nesadn fteinrwrig fCol- of workings inner the of understanding While h eut bandi h edeprmnsare experiments field the in obtained results The tion 215

VII — VALIDATION AND EVALUATION algorithm com- UTH A algorithm has been shown UTH A algorithm’s nearest neighbor search and clone VALUATION E Under the assumption that willingness to write docu- CollabReview had the power to involve contributors Users usually enjoyed the game created by the Col- The CollabReview prototype defines responsibility UTH mentation is a character trait,scores computed were quality compared karma to social reputationtions and of manifesta- documenting behavior as proxies. Contributors’ made with the help ofkarma karma scores are scores. not That arbitrarymeaningful. means but that measure However, something quality predictionskarma made scores are with not perfect.pares The well to other algorithmstion for authorship regarding computa- its precision.other But algorithms as are the ratheralgorithm differences small, is to used the by choice abe of CollabReview prototype influenced what should by other criteria like robustness. Here the detection make it a goodCollabReview choice. prototype against Yet attacks resistance was of notchallenged the in really the field experiments. Malevolentcan behavior be easily addressed throughtion. organizational There regula- was only onethe incident system where through a automated user submission attacked of reviews. in the creation of documentation. Even if it does notutors in- admitted that they madeonly their contributed first because contributions of or CollabReview. Experiments like ScrumLab showed that even ifnot a join few the developers game, do qualityand standards that can still some be peer-pressure achieved arises. labReview prototype. The rewarding systempresented was as never a threat but as an opportunity. While there ardous effects, a threatnot to evidenced the in development the teamlabReview field presents was itself experiments. as involving, Instead, causingcontributions first-time by Col- persons who had notumentation contributed before. to doc- Itbenefits is in cost-effective, documentation causing than costs more effort through for additional contributors. through authorship. Its to make much different attributionscommon in approaches. authorship Additionally, than it addsness basic robust- against attacks based on authorship information. mation they are computed from)are are significant consistent. moderate There to substantialtween correlations quality be- predictions for software process artifacts A volve each and every single contributor, several contrib- The quality karma scores (including the authorship infor- were a few incidents which might be precursors of haz- ALIDATION AND VII — V Negative karma scores and karma scores with a neg- Weak and strong contributors to quality perceive Col- The CollabReview prototype has proved to be highly However, the critical point is the rewarding system 216 software. They make good confederates for it. ative connotation can lead to frustration and rejection. problems of poor quality toso managers, to it the does not team. seem Instead, positive scores that show indi- ored; and they easily eliminatepresence every thought of about a the CollabReview prototype.to Awareness has be created.CollabReview. Strong However, they contributors, do instead, not welcome welcome iteers it but as as prof- a tool that helps them corral team members screws have not been taken full advantage of. CollabRe- labReview differently. Weak contributorsunfair feel because treated the mass of their contributions is not hon- group rewards can amplifycontrol peer-pressure in the and team mutual that strives to achieve a goal together. flexible and configurable. It hasent been kinds applied of to documentation, differ- integratinghumans feedback and from static analysis, usingity different responsibil- or software process artifact importance weights, and field experiments. Factors like includingity understandabil- of how karmaof scores the are rewarding system computed, are or crucial.management the remains In a the design management end, task quality commitment and by requires management. Here motivationlike theories Vroom’s VIE model become more important. Also, ment of a CollabReview prototypechange depends value on that the ex- iskarma scores. offered to Only if contributors karma for scores their have a subjective does not guarantee an effect as evidenced in one of the teams, and both teams achieved their goals. CollabRe- contribute to the software project’s documentation,therefore and to its internal quality. that provides the exchange valueeconomic for karma theory scores. predicts, As the success of the employ- system, further improvements seemnot probable statistically significant). (results Undesirable behaviormassively was cut in an experiment. High quality standards viduals and teams their achievements should be used. various rewarding systems. Still many of its adjusting view has further potential. value, then will contributors work towards better scores. view provides very valuable support to make developers The exchange value can be tangible like money but it who would otherwise harm the internal quality of the were set for two independent software development While technical debt may be well-suited to explain the

VII — VALIDATION AND EVALUATION epcnrbtr rmtyn oipoeterscores. their improve to trying can from scores contributors the they keep understanding what not However, for do. feeling to a need get their quickly optimize they to scores, try karma to start contributors Once scores. scores. developers karma as their long for care as took efforts, considered immediately the were quality than in valuable even gains And more the run. term, long short the the in off in pay to overhead; expected no be and can desired they documenta- as and are styling investments code These source tion. to that effort due most was The caused it them. to overhead low only meant validity. their supporting further quality, software artifact of process predictors as soundness proved scores karma nhwatosafc crsaeneeded. are scores affect actions feedback how informative on and briefings initial of form in Help nesadn fe oe rmpaigwt karma with playing from comes often Understanding CollabReview statements, users’ the to According I-4—Eauto fCollabReview of Evaluation — VII-14 eehzr ota prt a outadmanipulation- and robust has spirit, team to hazard vere thinks majority vast A controversial. are concept view be h eerhtei steeoeaccepted. therefore is thesis research understand- is The and able. overheads, low to due cost-effective game, is the into individuals draws se- calculations, a resistant pose not does flexible, is in- motivation, the dividuals’ to appeals quality, internal improves labReview concept. the of opponents arose and discussion proponents heated between experi- often field groups, the expert in In confirmed ments. out later pointed were issues experts the the of by Several concept. some the devel- showed in source also interest open management for industry suitable But more worthwhile opment. be is might it It that said try. and a work, can concept the that CollabRe- the about opinions experts’ note, general a On aiaino h eurmnshssonta Col- that shown has requirements the of Validation 217

VII — VALIDATION AND EVALUATION VALUATION E ALIDATION AND VII — V 218

VII — VALIDATION AND EVALUATION A ilb nvtbe eeoeslanadrlantere- the relearn and learn Developers inevitable. be will or knowledge, necessary the have already either who fragile a builds this of All made. are decision and with, work. h ersetv ossso noeve fconducted of overview an of consists retrospective The . Retrospective 8.1 Conclusion 8 Chapter urdkoldei aywy,icuigsupervision including ways, many in knowledge quired Learning head. their in software the about need they that is heads developers’ the expensive. into However, knowledge know. to that need they getting what learn to enabled are developers requires software the Building its in right. science own a is software every construct; knowledge experimented are alternatives the explored, which is in space process problem evolutionary an through comes being that into product knowledge-intensive a is Software Summary 8.1.1 lessons personal of set a and research. art, the conducting the from learned of makes state it the that contributions to main of summary a research, eeoescno lashv vr i knowledge bit every have always cannot Developers FTER rsnsa ulo nopsiiiisfrfuture for possibilities into outlook an chapter concluding presents this research, this in executed ersetv ntewr hthsbeen has that work the on retrospective a ra Kernighan Brian — vroekosta eugn stiea ada rtn program a writing as hard as twice is debugging that knows Everyone ntefis lc.S fyur scee syucnb hnyuwrite you when be can it? you debug as ever clever you as will you’re how if it, So place. first the in ihuual n isn ouetto.Ti internal This documentation. missing and unusable with means no by is Documentation artifacts. process ware aubeotoe h orecd,i tsaeo being of stake at is code, source the outcome, valuable yohrkoldebedvlpr,eprmnain or experimentation, developers, knowledgeable other by ocet au at n oflxbyaatt changing research to in adapt as requirements flexibly emerging to with and environments fast, allow value They create development. to software today’s in portant finally. lost be to and quickly, age to unmaintainable, becoming of therefore and incomprehensible, most project’s software The debt. technical is un-quality polluted gets that but in functionality everybody press that to property uses common just a like are is well, effects code guarded source not The If development: others. software in reaped and ubiquitous is developers reward fellow the but his time by precious it. his write him to costs has It who himself developer individual the for prototype. a of out product a make to engineers in allowslater customers documentation cre- to only knowledge transferred and be development, whole can contracted the that of project a things in few ated the of one is complexity, alleviates communication tracks, reduces on problems, development distance keeps It project. health a the of in investment long-term environment. a learning is good Documentation a project for software ingredient re- a important the making an get is to it but use knowledge, do quired or can developers that soft- everything other and code source includes but to limited not documentation is where documentation; from it acquiring gl eeomn ehdlge aebcm im- become have methodologies development Agile value few has documentation that argued was it Next, 219

VIII — CONCLUSION on the effect of compute the author- of developers for soft- that allows developers that empirical studies strengthen communication and sustainability ting, moving and copying of source code,thus and has a higher tampering resistance, not requiring strict processbeing light-weight regulations instead, but functioning in agile projects withownership, weak code where few hierarchical power is available, and while granting developers thedecision autonomy of of when to go into technical debt, delivers better results than typicalalgorithms, line-based has an increased robustness against reformat- – – – – – – – communication channel a telling critique of potentiallearned problems when and implementing lessons a reputationopment in environment, a devel- a and software metrics to storements, opinions and to assess- integrate them,as and quality to information feed to themprocess the artifacts back authors without of unnecessarily software riskingsonal per- conflicts, a new view on incentives and rewardsnot to to apply directly them improve performancebut of to individuals mance, providing several rewards on voluntary contributions, and the CollabReview prototype thatreuse is in available further research. for computing the responsibility and possibly other relationships, an algorithm based onnearest string neighbor searches manipulations to and ship of developers within a team, and thereby improving team perfor- ware process artifacts based on their contributions The presented concept is certainly not a silver bullet • • • • • • • for the software crisis. Yeteffective it nudge can towards give making developers long-term a investments cost- in the internal quality of software they produce. ONCLUSION VIII — C of the software to improving the novel and evaluated approach rational individual economic thinking ofers, develop- a internal quality of software byers motivating develop- to contribute to various kinds of documentation extending current scientific view crisis by attributing iteconomic in effects of part free to riding andphenomenon the the of well-known sociological the tragedy of commons, i.e. to the with a game, thus • • The karma scores serve in the gamification of inter- This situation is addressed by the CollabReview con- 220 the emerging field of gamificationmanagement. to Its major software contributions quality are therefore quality can be achieved. 8.1.2 Contributions itself and its rewardsresearch, are several not field the experiments had centralgames to focus rely to of on offer such this an exchangedevelopers. value The success for of karma CollabReview pointspends critically to de- on the perceived exchangechange value. value is But chosen if right, thein then developer ex- considerable behavior changes towards more caring for internal by contributing to them. Thescores result for are developers personal that karma expresscontributed how to well internal they quality. have nal quality, for instance, inawards, leader or boards, tangible recognition rewards. While the gamification cept and prototype. CollabReview wantsopers to an offer incentive devel- to contribute moreby to establishing documentation a sensecollaboratively of written individual responsibility software in processsource artifacts code. like Responsibility for artifacts is established its complicated relationship towards classical processes ment. Agile development is oftenneeding misunderstood documentation, for as not a harbor for cowboy coders, of motivating developers to write documentationquired. are re- software projects. However, agilityof comes weakened at the code cost ownershiptionships. and Moreover, responsibility agile rela- for development its is ambivalent position renowned towards documentation, and vances in mining software repositories, motivation and This research has applied basic economic theory to ad- which are central in traditional software quality manage- where it is difficult to control what is done. Other means

VIII — CONCLUSION ilhv adtm ocag oehn.Tereason The something. change to time hard a have will confusing. was counts view and size including weighting was it scores, high achieve to order in that told only were ehp ehia eti o omnctn ihman- with communicating for is debt “ridiculous”. technical it Perhaps called even a once from company engineer collaborating software unexperienced problems An some it. had derstanding developers al., that et found (Brown I visible 2010), problems make to concrete graspable something and as invented to was software it a While of developers. condition the about information porting deem I that lessons few a note: out to point important to want through I experienced years, Besides and learned the myself. have affected I also that it things but the all art effect the an of had state have the research on this conducting has only Not learned Lessons 8.1.3 o epycmitdt ult,te loCollabReview also then quality, to committed deeply quality not that think a I is before management than attain More still quality. and projects internal in control management for need complexity that mind in understanding. kept complicates be should creating it When scores, perfect. karma not was understanding computations. still intermediate But the on data further reports added quality that detailed featured which in ScrumLab, best perhaps the was a Understanding behavior. induce in to change enough well was understanding However, article the especially questions; open were still and there article so the read they barely how Contributors and rewards. computed, effected were scores dedicated how article explaining long to a The had contrast, scores. in the Moknowpedia, with to experiment questions or ask details not investigate did among they murmur was scores, there about developers While ap- random. and bit diffuse a too peared were source Scores the contributed. in Javadoc they was code there that sure make to enough They AgileLab. the in worst Understand- was developers way. among desired ing the in behavior developers their making change for factor important very a is standing be should the scores in avoided. negative score and karma experiments, a field as used later not therefore was technical techni- Anyhow, debt so connotation. cal negative and strong indebted, a Euro has be the debt to of wants times In nobody crisis people. technical with not agers, h olbeiwcnetwsivne ordc the reduce to invented was concept CollabReview The under- of level certain a that showed experiments The trans- for vehicle bad a be to seems debt Technical management ak fmngmn is management If task. II2—Ftr work Future — VIII-2 ihu aaeetmnyt a o hm hr will there them, for pay to money management without glLbadSrma nymaue ult without quality measured only ScrumLab and AgileLab iwr h eesetclaottecnet n less and concept, the about skeptical were who viewers tdfeig fufins.Tetcncldb scores debt technical The cre- unfairness. scores of karma proto- feelings in the ated — is which measure — performance quantity typical out surprised leaving was that I find So to 1994). Deming, (e.g. 1997; badly Humphrey, performed they people why some asking leave and and demotivated problematic, performance be measuring to happen because may was never It mance. alone. work not will it its but in quality; management for supports quest successfully that tool concept a a and is CollabReview them. man- to committing without develop agement not cultural will because quality pressure regarding peer norms be developers, rarely the will to there matter and that rewards intangible no be rewards tangible no be will there exchange in scores: offer karma to high value for exchange misses then it that is hrce fasotv aelk optto hnus- CollabReview. when ing competition fun game-like and sportive light-weight a the of keep character to essential probably were. is implementations It actual its seem then may threatening concept more The experiments. field of re- participants concept a mostly have were people it However, the of it. percent with ten problem opposer. about strong guess one would least at I au- not was discussion there no professor where almost a dience was of There because idea. the started theses rejecting have master the not of almost one could on work ques- even any and out fill tionnaires, to rejected heated colleague create one to discussions, tended emotional concept an the workshops, on On controversial level. highly is scores karma than work more doing for others. penalized they were Instead, they style that fine. coding felt been fixing have in would they time and their problems of have bit could little contributors a Major invested scores. quality quantity in in reflected than better be to seemed Additionally, reputation social contributors. was major this by projects unfair three as all considered In quantity. of indication any in Scores code. source (good) produced not of violations; amount of the amount the only counted TornBond in olbeiwwsntitne omaueperfor- measure to intended not was CollabReview oto l,Ilandta h oi fcomputing of topic the that learned I all, of Most 221

VIII — CONCLUSION The rewarding system and the exchange value it pro- The prototype and the concept have many more ad- CollabReview was not applied to issue trackers al- Also, the AgileLab field experiment was the only trial Reputation plays an important role in open source CollabReview extracts information aboutkarma developers’ scores from the revision history of software pro- such an environment, making theent. process more transpar- In the fieldschemes experiments, as it a used meanskarma different of scores. delivering rewarding But exchange they value werether for only research means can to focus an on building end.rewarding better Fur- system. or the With “perfect” respect topressed this, a wish users for even punishment ex- a like thank-you paying money box, into or makingof karma earning points a per certain week number rewards compulsory. like More an valuable additional day off were proposed, too. to deliver more exchange value. justing screws to offer that arementation worthwhile and further can experi- lead to optimizations and adjustments. in different environments are importanteffectiveness. With to the confirm results its oftiveness more and studies, efficiency its could effec- becisely. determined More more studies pre- can alsofor provide aspects stronger like support the validity of karma scores. 8.2.2 Further research opportunities though bug databases are importantknowledge. in It managing should project be possibleto to issue adapt trackers the without prototype much effort. So it could be used has not been tried. Stillartifacts more might types also of be software addressed. process in an environment with pair programming.the While failure part was of attributed to theinformation diffusion due of to authorship pair programming,also the have other failure reasons. may Especiallyof if determining appropriate responsibility ways in programmingavailable, pairs are then CollabReview couldenvironments. also fit into such software development. Many projects reward theircontributors top with more development rights,commit e.g. rights, earning and give themdeveloped. more A control reputation system of could what fit is naturally into vides for karma scores are essential to CollabReview. Another possibility is further research into social games Additionally, more and larger studies with CollabReview with tracker-based requirements management. But this ONCLUSION VIII — C open-ended quality scores is that responsibility can be indirect closed-range Experiments with robust karma scores could deal with What has been termed The CollabReview prototype relies on a variety of 222 not involve quantity.quantity The scores and problem with open-ended score components tend torange outweigh components. closed- So the focuskarma of score a would shift combined from robust quantity quality is to “scaled” quantity. down But to if tackle a the closed-range, developers’ can feelings it of still unfairness? scores (e.g. therated number that of no one software else processmore has artifacts karma rated) scores that could could be berobust integrated tried, karma into and scores. the so-called feelings of unfairness regarding karma scores that do quality assessments of wikianother articles. information that Test could coverage be is ternal incorporated into quality in- assessments.humans While were reviews used written in by thesource wikis, code and in were the foreseen originaltested concept, for with they source have not code. been Finally, there are other karma semantics, so that superficiallyretains modified its original source author. code analysis tools and software metricsadd available that to could the quality assessments used for source code. ing up is a valuableship contribution, too. algorithm While was the good author- presented enough here, there for is the still room experiments sion for and improving its its resistance preci- against attacks.technique Its could clone be detection enhanced withods more that advanced rely meth- on fuzzy matching, or exploit source code manipulations, for personalized assistanceearn on the next how points, to orthat for could the compete possibility against to each form other. teams metrics to compute its scores.sibility For metric example, relies the on respon- authorshipout but to therefore acknowledge misses deleting as a contribution. But clean- time constraints, and opportunities for further research. 8.2.1 Open questions Several questions were left open ining this research. the prototype, Regard- users wished for hardening it against 8.2 Future work Future work can be splitthat into have two remained parts: unanswered open in questions this research due to Analogously, natural language metrics could enhance worthwhile further research. There are many more static

VIII — CONCLUSION ae eas oeaeo estv oewt reviews with code sensitive of coverage because ware, ex- ﬁeld AgileLab the from data the of analysis the As aebu,20)aentvr rsn.Frhrresearch Further present. very not are 2001) Tanenbaum, scores. obtained the in potential research more is There iwn iscudhl hmlanmr bu their about more learn them help could bias viewing lrim ute eerhi esnlt psychology personality like in traits research to Further related only also altruism. not but trait experience this from perhaps is results trait; there a that such trait shown like implicitly character something was a it is So document developers. of to willingness that true reviewers. other to compared as expectations quality developers re- their the developers to showing example, given For be themselves. also can And information developers. by such with provided relation or into about put information be other can scores shown, has periment before. form that in available not was that artifacts cess nyoeqaiycaatrsi.ClaRve ih in might CollabReview characteristic. is quality security one Moreover, only issue. this into investigate could (cf. effects deterring desirable Hence guaranteed. not is soft- relevant security for suitable be not Col- may instance, labReview For reviews: costly formal more especially replace concepts, whether to unclear potential is the Therefore has CollabReview threat. a never as and perceived character be fun its of Col- part of keep should instance labReview An 2004). Spolsky, 1994; lit- Deming, 1997; management, (e.g. Humphrey, for it doing interest against of arguments area erature While an motivation. be to would threat that a become easily can ment measure- Performance performance. developer measure direction. this into investigate could ti o h neddproeo olbeiwto CollabReview of purpose intended the not is It holds assumption the that required experiments Some II2—Ftr work Future — VIII-2 eomn skoldewr.AdClaRve was CollabReview And work. knowledge as velopment This management. knowledge in place its have can view go scores karma developers’ the Whenever velopers. outr otiuin swl.I ol,frexample, for could, It well. as contributions voluntary nweg aaeetwk.Wieti a ihnthe within was this a While to contributions wiki. increasing management for knowledge tool a as tested already de- software of characterization the with started research workload. rec- implementation a increase as to this system use ommendation can managers threshold, certain a above de- at tasks manage- implementation while new throws scores, constantly karma ment quality their increase them to urges constantly CollabReview workload: perfect information. and log tracker commit repository bug revision evaluating through source example software for to the failures code, software inside relate components to respective found the are ways if characteristics well quality as external to applicable be general icpie n edt oepoutv meetings. productive more to lead and discipline, support could CollabReview then into representations, converted text automatically be could assignments, and task keeping, minute late, like arriving work people supportive delays, interruptions, with conduc- meetings the work if of instance, tion For pro- supported. such be also could potentially cesses then representation, text a into al., et Reiners (see database 2012). a in patterns software collecting of efforts collaborative to contributions amplify on rely that databases knowledge other to applicable bly proba- is CollabReview context, development software usd h otaedvlpetdmi,CollabRe- domain, development software the Outside near developers keep to used be could CollabReview fi a osbet rn te elwrdprocesses real-world other bring to possible was it If 223

VIII — CONCLUSION ONCLUSION VIII — C 224

VIII — CONCLUSION e Vorsitzenden dem (Acknowledgments) Danksagung riamdeUnterst die Freiraum Zeiten. schlechten in wie guten in Worte, aufmunternden f Dank wenig ein Praxis und Forschung auch sondern sammeln, aradrnhtseshnjttir prnhinterlassen. Spuren ihre jetzt schon sie hat anderen paar eakn uamnmit Zusammen bedanken. rtazshißn o enrVredgn ak ich danke Verteidigung meiner Von abzuschließen. erst e ehe e ger Weg rechten den ver Begeisterungsf tu ndnKlerglndrHochschulbibliotheken der Kellerregalen den in Staub In enDn itaßre e ruhfrIsiu f Institut Fraunhofer dem außerdem gilt Dank Mein esce unerl Versuchen Anf Dencheva Beitr he iee oshn ihieBeitr wichtige Forschung eigenen ihrer iksinn i Ratschl die Diskussionen, enbsnee akgl enmmrangetrauten mir meinem gilt Dank besonderer Mein enWnc s ae,ds en retnctnur nicht Arbeit meine dass daher, ist Wunsch Mein nbsneei e sp den in insbesondere i eodre Dankesch gesondertes Ein Oppermann eshnneasdsgwre,wssehueist. heute sie was geworden, das niemals Menschen B enneeaie Kollegen ehemaligen meinen neadeIfraintcnkFTudseinen und FIT Informationstechnik Angewandte ee e magece ret i c ndiese in ich die Arbeit, umfangreichen der Neben Doktorvater oshn etcthb,w habe, gesteckt Forschung hlce es erftdies betrifft Weise ahnlicher nenwr.A i eisudvelih ohein noch vielleicht und gewiss mir An wird. andern ¨ ¨ a,uda en hmlgnStudenten ehemaligen meine an und hat, esnDs-ocig,Aliugnudgroße und Anleitungen Diss-Coachings, dessen uro-Mitbewohner ¨ Eisenhauer nemie oshn i piimsbegleitet Optimismus mit Forschung meiner ange ¨ nese tlem Stelle erster An Unterst g n optn ie ilalvnschlauen von Vielzahl einer Kompetenz und age ¨ rdeM die ur ¨ und tug if n osrkieKritik konstruktive und Hilfe utzung, ¨ beln etrausgebaut. weiter Abteilung oshn eese haben. geleistet Forschung a rmtwrvle anhe und Maßnahmen wertvollen mit er hat gihetmieDoktorarbeit meine oglichkeit hgetmieAbi ace a auf Mal manches Arbeit meine ahigkeit ¨ ¨ atisJarke Matthias er Agaraj Henri assi isnbiSuinund Studien bei Wissen sein dass , slc a.Zsme mit Zusammen war. asslich ¨ ofagThomas Wolfgang tugf utzung cthbn enrgl f gilt Ferner haben. uckt ¨ ¨ Ren ct c ihbimeinem bei mich ich ochte ¨ trnPae enrArbeit meiner Phasen ateren ¨ Reiners e ´ okrWulf Volker g,dsFebc n die und Feedback das age, ¨ nm on rPooirnei seiner in Promovierende ur ¨ ¨ i i he retund Arbeit ihrer mit die , r i ohon die ohne doch sie are ¨ f nra Zimmermann Andreas rdeErmunterung, die ur tfnApelt Stefan ¨ ct c ihe an richten ich ochte ¨ f rdezahllosen die ur ¨ und geb g umeiner zu age ¨ os Lichter Horst htimder ihm uhrt ¨ e die der , Silviya ur ¨ Reinhard uberhaupt ¨ Markus ur ¨ , . Bernards oshn rterm erst Forschung Rahmenprogramme, EU der Projekten aus insbesondere Umfeld das um sowohl Feldversuche, und Befragungen nshliuga en rud n aii,ds ich dass Familie, und Freunde meine an Entschuldigung iabien—mie hmlgnKlee i mir die — Kollegen ehemaligen meinen — Mitarbeitern en eud enVerst dein Geduld, deine eimtdmaoye turalri Deutschland in Steuerzahler anonymen dem Gewidmet Kirschenmann enrFmleudinsbesondere und Familie Meiner (Awesomeness!), Carsten eetihf Wesentlich escsaice ae;mrMtra ifre;mir lieferten; Material mir waren; Versuchskaninchen i nee isncaten aleceSuinwie Studien zahlreiche Wissenschaftlern, anderen mit akf Dank ndnvragnnMntns ei ethte Ich hatte. Zeit wenig so Monaten vergangenen den in nra Suchan Andreas he R Ihren e -T e Universit der B-IT, dem l uhmieFrcugsls esrz verstehen. zu besser selbst Forschung meine auch als Pramudianto uehl e Gr des außerhalb esrun z ice sorgten: Tische” “zu Zerstreuung ak uh n speziell und euch, danke ifec u et tne;udoe f und/oder standen; Seite zur hilfreich Bonerewitz Elmasllari iasdneihgn besonders ganz ich danke hinaus Fisseler aj Niemann Katja eakngbah n i eeg at ases dass habt, gezeigt mir und gebracht Gedanken Mertens i fee ou oe;mrRtee,freudige Ratgeber, mir boten; Forum offenes ein n e U e i ennSeeglenmeine Steuergeldern seinen mit der EU, der und ucsctmie nwre petn Meine opferten. Entwurfes meines Durchsicht e iktne,TinhenudAkteuren und Teilnehmern Diskutanten, Den rOfnetudafeclseeKooperation. aufgeschlossene und Offenheit ur ¨ chl epihe.S verpﬂichtet. uckhalt retewszur etwas Arbeit ¨ i c f ich bin , iksinknrhne n eifrige und Diskussionskontrahenten tfnHarms Stefan uhan Auch . , atnWolpers Martin rmieFrcugwrndrAustausch der waren Forschung meine ur ¨ , asirmc eeetihafandere auf gelegentlich mich ihr dass , , , asCrsinSchmitz Hans-Christian akVinkovits Mark moAl-Akkad Amro i iieSudnirsLbn der Lebens ihres Stunden einige die , rIr ib,Ir Unterst Ihre Liebe, Ihre ur ¨ d G Eda , nnZmesenNctee gibt. Nachtleben ein Zimmers unen ¨ gih a.Ihhfe h i dieser mit ihm hoffe, Ich hat. oglicht ¨ acJentsch Marc ennDank. meinen a Nonnen Jan , nnsuddieZi!Dar Zeit! deine und andnis ¨ ung aa enRojas Leon Sarah ¨ c ee uk zu geben uck tBn n e SC2011, ESEC der und Bonn at ¨ ¨ hmsP Thomas or , ¨ ae Scheffel Maren , ankju ¨ , , alCatewicz Karl oahnSimon Jonathan lxne Schneider Alexander , o e n Bonn Uni der von asZahl Lars Reinhard ac Jahn Marco rtaMeys Britta arnWolter Katrin utz ¨ , uihFlory Judith onnen. renwenig ein ur ¨ ¨ und und , tugund utzung ¨ , Helga , Daniel Gabriel Erion , Daniela Andrea , , und Uwe Ferry uber ¨ f und 225 ur ¨ ,

IX — DANKSAGUNG ANKSAGUNG IX — D 226

IX — DANKSAGUNG FP5 FLOSS EU EC DvC CVS CvA CSCW CPU CMMI CMM CASE AvP AST ASCII API Acronyms of List Lists and Registers IvG IvC IT ISO IP IDE HTTP HTML HSLOC HotW GvL GUI GB FTP FP6 KLOC JSP J2EE nertdProject Integrated nomto Technology Information uoenUnion European Euro- the of executive (the Commission European iaByte Giga ehooia Development Technological Development Technological enUnion) pean change nemtetv.Continuous vs. Intermittent aaSre Pages Server Java Group vs. Individual plcto rgamn Interface Programming Application h it rmwr rgam o eerhand Research for Programme Framework Sixth the and Research for Programme Framework Fifth nentoa raiainfrStandardization for Organization International nnmu s Personal vs. Anonymous nertdDvlpetEnvironment Development Integrated rpia srInterface User Graphical Protocol Transfer File ual s Consumable vs. Durable Achievement vs. Competitive btatSna Tree Syntax Abstract urnedv.Luck vs. Guaranteed ocretVrin System Versions Concurrent eta rcsigUnit Processing Central aa2Pafr,Etrrs Edition Enterprise Platform, 2 Java aaiiyMtrt Model Maturity Capability eoo h ekaward week the of Hero optraddSfwr Engineering Software Computer-aided yetx rnfrProtocol Transfer Hypertext mrcnSadr oefrIfrainInter- Information for Code Standard American huadlnso code of lines thousand aaiiyMtrt oe Integrated Model Maturity Capability optrspotdCoeaieWork Cooperative Computer-supported ye etMru Language Markup Text Hyper reLbeOe oreSoftware Source Open Free/Libre et o-mt ie fsuc code source of lines non-empty hecto b A a a ] XP XML WWW ∆ WSDL Word WebDAV VvR VIE USB UML TvO TQM SvT SQuaRE SPICE SPI SME SLOC SCM SCAM RAM PvR PvD PvC P-CMM OSGi NFS LOC LARI LAN ε h eetdrve function review rejected the ucinfradn ou on o te achieve- other for point bonus adding for function otaepoetartifact project software e falsfwr rjc artifacts project software all of set tigei itnefunction distance edit string a mt artifact empty xrm Programming Extreme ments Evaluation and Determination Manipulation and Analysis Code Source otaePoesImprovement Process Software hwofv.Take-away vs. Show-off Valence-Expectancy-Instrumentality uihetv.Reward vs. Punishment Discoverable vs. Proclaimed Closed vs. Public ewr ieSystem File Network ita s Real vs. Virtual nvra eilBus Serial Universal Open-ended vs. Targeted oa raNetwork Area Local ml n eimEnterprise Medium and Small ie fcode of lines otaeCngrto Management Configuration Software xesbeMru Language Markup Extensible nfidMdln Language Modeling Unified admAcs Memory Access Random oa ult Management Quality Total pnSrie aea initiative Gateway Services Open irsf Word Microsoft ag nnmu eerhInstitute Research Anonymous Large oreLnso Code of Lines Source EEItrainlWrigCneec on Conference Working International IEEE e evcsDsrpinLanguage Description Services Web Capability and Improvement Process Software ol ieWeb Wide World epeCpblt auiyModel Maturity Capability People ytm n otaeQaiyRequirements Quality software and Systems e itiue uhrn n Versioning and Authoring Distributed Web a ε a 227

X—LISTOF ACRONYMS CRONYMS A to u 1] ISTOF .. [0 a X—L into a 0 a for artifact u u an edit script that transforms a sive) artifacts from the available review ratings from its contributors’ karma scores assigned to an artifact or one of its ancestors (recur- developer specialization index set of all agents a function that computes an artifact’s quality score

an importance function that assigns weights agent a function that gives the karma score of an agent a function that gives an artifact’s optimal predecessor a function that computes an artifact’s quality score responsibility of agent 0 function that gives the rating score that an agent has a timeliness function for reviews 0 a ds 228 t u U w q r q δ p s i k

X—LISTOF ACRONYMS ito Figures of List 102 ...... 103 ...... 82 ...... 100 101 ...... 87 ...... 76 . . prototype . . ﬁrst . . . the . . in . . . . communication . . user . . . . and . . Internal ...... concept 84 original . 5.5 . code . . the source of . . about . . . Poster opinion . . strong . . . a . . having . . . 5.4 for 72 61 . . notorious . . . are . Developers . 66 ...... 5.3 . . . . 74 ...... research . . conducted ...... of . . spiral ...... the 5.2 . . in ...... milestones . . Iteration ...... 61 . . . 5.1 . . . VisualStudio . . . . in . . . functions . . . . . out . . . try . 59 . . . developers . . . . make . . . . to . . . Badges . zones . . . . . chaotic . . 3.14 . and . 59 . . . . self-organizing . . conventional, ...... of . . diagram ...... Stacey ...... 3.13 ...... tools . CSCW ...... of . Matrix ...... changes 3.12 . . . . undoing . 56 revision ...... a . of ...... Illustration ...... 3.11 ...... of . . Output . . . revisions . . two . . . 3.10 . 48 between . . . differences . . . . showing . 59 . . MediaWiki ...... 52 summary . . . project . . . . . a 3.9 . . . with ...... dashboard . . . Sonar ...... levels . . . and . . . . . areas 3.8 . . . process . . . key . . . . . P-CMM ...... spot . . 3.7 sweet . . . . . cost . . failure . . . . . Low 45 . . quality . . . . . of . cost ...... 3.6 and . 46 level ...... quality . Software ...... management . . . . . project . . 3.5 of . . . . . Triangle . . Golden . . . . . The . . 34 . . . . cycle . . life . . . . software 3.4 . . . the . . . . in . . . Quality . . . 9126-1 . . . ISO . . . of . 39 . . sub-characteristics . 3.3 . . and . . . characteristics . . . . Quality . . . . 24 ...... Doxygen 29 35 . . . by . . 3.2 . generated . . . . reference . . . . Class ...... 3.1 ...... responsible. . . . those . . . . . are . . . So ...... disgusting. . . . is . . . . . Litter ...... 2.12 . . . goods 18 . . . . economic . . . of . . . . classes . . . Four . . . . innovativeness . . . . of 2.11 . . . basis . . . . the . . on . 14 . . categorization . productivity . . Adopter and . . . . quality . 2.10 of . . . . . Triangle 27 . Iron revision . . . . . The of . message . . . . . log . Subversion . . . . and 2.9 . consortium . . project . . to . Email Karakos . . . . the . of . . . . packages . 2.8 work . . . . and . phases . . . . Development ...... Karakos 2.7 of . . . . structure Organizational ...... 2.6 . . . . of . . . . Excerpts . . . code . . . source 2.5 understanding . . . of Problems . . debt . . technical of . 2.4 . kinds four . . The . . . artifact 2.3 process . software requirement Volere data and information knowledge, 2.2 of pyramid The 2.1 Dendro˘c 100 ...... text of pieces scattered from documentation interconnected constructing for v blame svn ekadPoke and Geek 74 ...... code source Java with ﬁle some for 30 ...... cartoons - uuework Future — X-2 229

X—LISTOF FIGURES (right) ...... 115 svn blame IGURES F ISTOF X—L 5.9 Three iterations5.10 of a Selecting tree origin search by . size of . footprint .6.1 . . . . Simpliﬁed class6.2 . . diagram of . . CollabReview server CollabReview . .6.3 portal . . log . in . . page . Browsing artifacts . .6.4 . . in CollabReview . . . . portal . . Viewing . . artifact .6.5 details . . . . in . CollabReview . . . . server . Moknowpedia web . . main . portal .6.6 . page . . augmented . . . with . . . several . . Reviewing . widgets widgets . .6.7 ...... Moknowpedia . . . . domain6.8 . . . ranks . . . . . and . . . special . . . . . features . Moknowpedia . . . avatars .6.9 . . . augmented . . . . with . . . . . levels . . . resulting . . . . Main . . from . page .6.10 . karma . . . . of . . scores . ReputationForge . . XML . . within . . . . output . a . . of . . . GForge . . . . karma6.11 . project . . scores ...... web . Quality . . . service . . report ...... for . . . . . a6.12 ...... Java ...... project . Loading ...... a ...... review ...... from6.13 ...... the129 ...... web . . Review . . . . service . . plug-in . . . .120 for . .119 . . . . . VisualStudio . .6.14 ...... Artifact . . .134 . . . . reviewed . . . notiﬁcation ...... mail . .6.15 ...... Hero . . . .135 . . of .136 . . . the ...... Week6.16 . . . announcement . . .133 . . . . Daily ...... email . . . . with ...... quality .6.17 . . . ranks . .133 . . . . Technical . . . debt . . . . . ranks . . . . . mail ...... 7.1 ...... 136 ...... Screenshot . . . . . from . .7.2 . . . the . . . . . FreeCol . . . .137 game ...... Choosing . . . . . the7.3 . . . . . next . . . . . task . . . . . from . . . . . the . . AgileLab . . . available . . kanban7.4 . . tasks . . . board . . in . . . . . ReputationForge ...... ScrumLab ...... Kanban7.5 ...... board ...... 135 . . .138 . . Predicting . . . . . software .7.6 . . . . . process . . . . . artifact ...... quality . . . . . using . Number . . . . . developer . . of .7.7 karma . . . FreeCol . . . scores . . . . artifacts . . . . per ...... quality . . Quality . . . class . . . karma7.8 . . .140 . . scores ...... by . . .139 . three ...... selected . . Scatter . . authorship . . plots .7.9 . . . . algorithms . . of . . in . . predicted . . . . . FreeCol . . and . . . . assessed . . Scatter . . . .140 . quality . plots .7.10 . . for . . of . . . . . FreeCol . computed . Scatter . . .141 karma . . . plots . . . scores . of . . . and . . . computed .7.11 . . .155 social karma . . . reputation . . . scores . . . Scatter . in and plots . . . Moknowpedia . . . social . . . of . . reputation quality .7.12 .141 . . . in . . . karma . . TornBond scores . . Results ...... versus . of . . other . . Moknowpedia . . . .162 . proxies . pre-7.13 . . . . and ...... post-evaluation . Amount . . interview . . . of .141 . . . . source . . . . . code7.14 . . . . . contributed . . . . . by . . . . Average . . the quality . . . . . developers of ...... (quantity source . . .7.15 . . karma code .164 . .143 . scores) written . . . . . Evolution by . . . history . each . . . .152 of . developer . . . technical .7.16 (quality169 . . . debt . karma . . . in . scores) Comparison . . . TornBond . of . . . . . contributions . . . . . in . .7.17 . . . Vyrygh . . . . . and . . . Number . . Moknowpedia169 . of ...... micro . . . edits7.18 ...... detected . . . each . . . Quantity . . . . month . scores . . . . . of . . .165 . Team7.19 . . . A . .164 . . . . compared . Quantity . . to . . . scores155 . . average . of .182 of . . . Team7.20 . . Team B . . B . . . compared182 156 Quality to . . . scores . . . average of of . . . Team7.21 . . Team . A A . . . compared . . Quality . to . scores170 . . average175 . of . of . Team . Team . B . B . . compared . . to . . . average . . of . . . . Team . A ...... 193 ...... 189 ...... 199 ...... 199 . . . .195 . . . .200 . . 200 5.6 Software process5.7 artifact evolution . Parent relations5.8 . forming a . straight tree Markup . as . obtained . by . the . algorithm . (left) . compared . to ...... 106 . 106 230

X—LISTOF FIGURES ito Tables of List 209 . . . . 211 ...... 202 ...... 158 ...... 177 ...... 148 ...... criteria . . satisfaction . . . requirements’ . . of . . . 183 Summary . 97 . . . . . 124 ...... 7.8 . studies . . conducted . . . 118 . . of . Overview ...... ScrumLab . . in . . . . intervention AgileLab . 7.7 the . in . . . . about ranks . opinions . score . . . . Likert-scaled with . . correlation . . . . and 166 . . intervention . . . . 7.6 . the . . about . . . . . opinions . . Likert-scaled . questionnaire . . . post-test . . . Moknowpedia . . 80 . in . . . 7.5 items . weights . to and . . . agreement . algorithms . . User authorship . . . . different . . for . . . . correlations . . 7.4 Spearman . . . . as . . quality 119 . . 57 . . Prediction ...... heuristics . . . . . neighbor 7.3 . . nearest . . . . . of . . Comparison ...... strengths . . . . . correlation . 7.2 science . . . . . social . of . . . . . Nomenclature ...... 7.1 . . patterns . . . reward . . the . . . of . characteristics . Karakos . . of from . Overview . code . . 41 source . Java . . . of . distinctivenesses 5.4 . . . normalized . . for . . . Examples ...... features . . 5.3 of . . occurrences . . of . . Distribution . . . 31 ...... 5.2 . . requirements . . of . . Summary ...... 5.1 . . . . response . and . . target . for . . table . Contingency ...... 3.2 . . home-grounds methodology . . Development . . projects . 3.1 EU in . processes of . importance perceived and Maturity background by problems documentation 2.2 seeing experts of Percentage 2.1 - uuework Future — X-2 231

X—LISTOF TABLES th 8 Annals , pages , pages Modern , 16(1):185– , ESEC/FSE, Information and Annual IEEE Sym- th 45 Proceedings of Joint , pages 973–980, New York, . Addison-Wesley. , 25:22–29. , 50(9–10):860–878. Joint Meeting of the European ACM SIGPLAN conference com- th th , 3(1):75–89. 6 24 International Journal of Applied Soft- Proceedings of European Conference on Software Archi- Proceedings of AGILE Conference rd 3 , WICSA/ECSA, pages 81–90. IEEE. IEEE Software ment. In edge creation. ware Technology Kumar, R. (2004). Approximatingficiently. edit In distance ef- posium on Foundations of Computer Science 550–559. IEEE. of the Institute of Statistical230. Mathematics based clone detection with flexible tokenization. In on Foundations of Software Engineering New York, NY, USA. ACM. Sharp, H. (2008). Motivationing: in A software systematic engineer- literature review. bugs. tural practices and challenges indevelopment using approaches. agile software In ture & tecture panion on Object oriented programming systemsguages and lan- applications NY, USA. ACM. Information Retrieval J., and Pugh, W. (2008). Using static analysis to find Software Engineering Conference and the Symposium Software Technology Proceedings of Proceedings of 44–47. Adopting code reviews for agile software develop- A market-based approach to software evolution. In Working IEEE/IFIP Conference on Software Architec- Bernhart, M., Mauczka, A., and Grechenig, T. (2010). Bailin, S. C. (1997). Software development as knowl- Bar-Yossef, Z., Jayram, T. S., Krauthgamer, R., and Barankin, E. W. (1964). Probability and the east. Basit, H. A. and Jarzabek, S. (2007). Efficient token Beecham, S., Baddoo, N., Hall, T., Robinson, H., and Ayewah, N., Hovemeyer, D., Morgenthaler, J. D., Penix, Babar, M. A. (2009). An exploratory study of architec- Bacon, D. F., Chen, Y., Parkes, D., and Rao, M. (2009). Baeza-Yates, R. and Ribeiro-Neto, B. (1999). , IBLIOGRAPHY IEEE X—B , 51(1):117– Proceedings IEEE Interna- th , 6(3):166–175. 9 ¨ old, H. (2007). In- International Conference on ¨ oteborg University. Proceedings of International , 5(10):2–13. , ICSE, pages 361–370. ACM. , WikiSym. Proceedings of , SCAM, pages 55–64, Los Alami- Proceedings of the 2010 ACM Con- . IEEE Computer Society Washington, Communications of the ACM International Workshop on Mining Software First Monday , 27:16–22. th Journal of American Science 4 mentation expertise from bug reports.of In the DC, USA. free-riders: Reading as a form ofference participation on on Computer Supported CooperativeCSCW, pages Work 127–130, New York, NY, USA. ACM. should fix this bug? In algorithms for approximate nearest neighbordimensions. in high study. test coverage. In tional Working Conference on Source Codeand Analysis Manipulation tos, CA, USA. IEEE Computer Society. dividual vs. team basedon reward how systems organizations — argue for a theirlor study essay choice. in Bache- management control,Economics School and of Law, Business G engineering literature: An analytical and comparative tem based on reputation. Master’s thesis, RWTH gnutella. Pye, I., and Raman, V. (2008). Assigning trust to Software Engineering Symposium on Wikis Software Repositories wikipedia. In wikipedia content. In Aachen / Fraunhofer FIT. Agility and architecture: Can they coexist? 122. 232 Anvik, J. and Murphy, G. C. (2007). Determining imple- Anvik, J., Hiew, L., and Murphy, G. C. (2006). Who Andoni, A. and Indyk, P. (2008). Near-optimal hashing Antin, J. and Cheshire, C. (2010). Readers are not Alves, T. L. and Visser, J. (2009). Static estimation of Al-Qutaish, R. E. (2010). Quality models in software Agaraj, H. (2012). Building an effective rewarding sys- Ahlgren, A., Andersson, I., and Sk Adar, E. and Huberman, B. A. (2000). Free riding on Adler, B. T., Chatterjee, K., de Alfaro, L., Faella, M., Bibliography Abrahamsson, P., Babar, M. A., and Kruchten, P. (2010).

X—BIBLIOGRAPHY ol .B 18) ontv vlainter:A ex- An theory: evaluation Cognitive (1981). B. K. Boal, R. Walker, and S., Greenberg, A., Voida, D., Bertram, G. S. Siegel, and D., V. Henderson, H., E. Bersoff, ot,J (2005). J. Bortz, soft- Prioritizing (2006). L. Moonen, and C. Boogerd, (1976). M. Lipow, and R., J. Brown, W., B. Boehm, with methods, agile for ready Get (2002). B. Boehm, develop- software of model spiral A (1986). B. Boehm, (2005). F. Bodendorf, Coordination (2007). V. Wulf, and B., Nett, A., Boden, (2009). V. Wulf, and L., Bannon, G., Avram, A., Boden, aeiseto eut sn ttcpoln.In proﬁling. static using results inspection ware ofrneo lblSfwr Engineering Software Global on Conference the bugs: and collaboration, Communication, (2010). tuto- A management: conﬁguration Software (1979). Engineering oreCd nlssadManipulation and Analysis Code Source Engineering Software In matter? culture does — teams velopment senschaftler 149–160. pages of ceedings In of ceedings quality. software of evaluation Quantitative care. Notes ing enhancement. and ment edition. zweite Springer, Society. Computer IEEE 235–244. pages In small enterprises. of development software distributed in practices Society. Computer of ings de- software distributed in management Knowledge 310. Performance Human and Behavior outcomes. tional and processes of test perimental ACM. USA. NY, York, New 291–300, Work Cooperative Supported collocated puter small, in In tracking teams. issue of nature social rial. EEComputer IEEE Computer 4 th 11(4):14–24. , rceig fAMCneec nCom- on Conference ACM of Proceedings EEItrainlCneec nGlobal on Conference International IEEE 2 pigrMdznVra,sxhedition. sixth Verlag, Medizin Springer . ae 9–0.ACM. 592–605. pages , nd 6 th rceig of Proceedings 35:64–69. , nentoa ofrneo Software on Conference International ttsi f Statistik EEItrainlWrso on Workshop International IEEE ae-udWissensmanagement und Daten- 12(1):6–14. , CS,pgs1–7 IEEE 18–27. pages ICGSE, , ISF otaeEngineer- Software SIGSOFT rHmn n Sozialwis- und Human- ur ¨ 2 nd EEInternational IEEE SW pages CSCW, , 28(3):289– , Organiza- SCAM, , - uuework Future — X-2 ICGSE, , Proceed- Pro- Pro- . rchu,eio (2005). editor Brockhaus, large- a of anatomy The (1998). L. Page, and S. Brin, W C., L. Briand, (2010). M. Melvin, and W. Boyes, own- Reconstructing (1999). R. Holt, and I. Bowman, ucmn,F 21) erigfo alr,pr 2: part alles. failure, from ist Learning sein (2010). F. Buschmann, Schnell (2010). H.-J. Bullinger, Groupware: (1988). R. R. Johansen, and V. C. Bullen, Br (2000). H. A. Dutoit, and B. Bruegge, M., Kim, R., Kazman, Y., Guo, Y., Cai, N., Brown, (2003). P. F. Brooks, okhpo uueo otaeEgneigRe- Engineering Software of Future on Workshop etr egg Learning. Cengage Western e omngn uiestas ehia re- Technical teams? business managing to key A ACM. 20) xlrn h eainhp ewe design between relationships the Exploring (2000). S lop In of engine. search web hypertextual scale 273. sys- object-oriented tems. in quality software and measures Society. Computer IEEE Comprehension Program on In sys- software tems. understand help to architectures ership ware diseases. other and performitis, Featuritis, MIT. Management, Sloan of Research, School Systems Information for Center port, edition. second Oldenbourg, Praxisleitfaden mit Softwareentwicklung der in dard and complex Conquering systems changing engineering: software search In in debt systems. technical software-reliant Managing K., Sullivan, (2010). N. C., Zazworka, Seaman, and R., R., Nord, Sangwan, A., I., MacCormack, Ozkaya, E., Lim, P., Kruchten, mitp. h,A-. dtr(1995). editor A.-P., ohl, ¨ detceZeitung uddeutsche ¨ 7 de n3 B 30 in adie: ¨ th 27:10–11. , nentoa ol-ieWbConference Web World-Wide International oE,pgs4–2 e ok Y USA. NY, York, New 47–52, pages FoSER, , ora fSsesadSoftware and Systems of Journal rceig of Proceedings s,J,Dl,J . n otr .V. D. Porter, and W., J. Daly, J., ust, ¨ anden ¨ rnieHall. Prentice . o yhsdsMann-Monats des Mythos Vom 3:2. , rchu,2 edition. 21 Brockhaus, . 7 th rchu i Enzyk- Die — Brockhaus a -oel e Stan- Der V-Modell: Das rceig fFSE/SDP of Proceedings nentoa Workshop International WC ae 28–37. pages IWPC, , Economics Object-oriented 51(3):245– , Proceedings EESoft- IEEE South- . 233 . . .

X—BIBLIOGRAPHY , The Data- . The . IBM Overview . Carnegie Information, . Centennial Books. , 3(1):1–44. . PhD thesis, University , 31:1268–1287. , 55(8). . Mentor, New York. Addendum to the proceedings Quality is Free — The Art of PMD applied The Social Science Encyclopedia , OOPSLA Addendum, pages 29–30, Pragmatic Project Automation Towards Computer-Supported Collab- , July 6. Proceedings of the Conference of the Centre , 14(4):28–31. tion und dokumentation in der wikipedia. Practical language-independent detection of near-miss clones. In for Advanced Studies on Collaborative research Press. agement system. In on Object-oriented programming systems, languages, and applications New York, NY, USA. ACM. of the People Capability MaturityMellon Model University - Software Engineering Institute. study of the software design process for large systems. tice in knowledge management —management part framework. 1: Knowledge Pragmatic Programmers. chapter Externalities, page 340. Routledge. mation orative Software Engineering of Canterbury (New Zealand). Technical report, Carnegie Mellon University, Soft- Boston Globe Communications of the ACM ware Engineering Institute. Making Quality Certain Journal of Law and Economics Wissenschaft und Praxis Danowski, P. and Voss, J. (2004). Bibliothek, informa- Copeland, T. (2005). Cordy, J. R., Dean, T. R., and Synytskyy, N. (2004). Crosby, P. B. (1980). Cunningham, W. (1992). The wycash portfolio man- Curtis, B., Hefley, W. E., and Miller, S. (1995). Curtis, B., Krasner, H., and Iscoe, N. (1988). A field CWA Std. 14924-1 (2004). European guide to good prac- Clark, M. (2004). CMMI (2010). CMMI for development, version 1.3. Coase, R. H. (1960). The problem of social cost. Collard, D. (2009). Connell, C. (1999). Creating poetry in software code. Conway, M. (1968). How do committees invent. Cook, C. (2007). IBLIOGRAPHY IEEE , 14:247– X—B , 17(6):65–70. Proceedings of the Proceedings of In- IEEE Transactions Joint Meeting of the IEEE Transactions on , MSR. IEEE. th Journal of Systems and 7 , 20(6):476–493. IEEE Software , 36(4):546–558. , 18(2):22–29. , ASE, pages 33–42, New York, , ESEC/FSE, pages 101–110, New , MSR, New York, NY, USA. ACM. Proceedings of Proceedings of International Workshop on , 20(6):46–51. , 80(11):1930–1937. IEEE Software Software reviews: The state of the practice. improvement on effort. source code copy detection.ternational Working Conference In on Mining Software suite for object oriented design. on Software Engineering formats, and open source. between process maturity andtion: geographic an empirical distribu- analysis ofquality. their impact In on software ware Engineering global software development: A structured approach. 262. fying changed source code linestories. from version In reposi- for alleviating distance inment. global software develop- NY, USA. ACM. a metric for code readability. documenting program changes. In software engineering IEEE/ACM international conference on Automated Software Software Software Process Improvement and Practice Software Engineering Repositories European Software Engineering Conference and the Mining Software Repositories York, NY, USA. ACM. ACM SIGSOFT Symposium on Foundations of Soft- 234 Ciolkowski, M., Laitenberger, O., and Biffl, S. (2003). Clark, B. K. (2000). Quantifying the effects of process Chidamber, S. R. and Kemerer, C. F. (1994). A metrics Cerri, D. and Fuggetta, A. (2007). Open standards, open Chang, H.-F. and Mockus, A. (2008). Evaluation of Casey, V. and Richardson, I. (2009). Implementation of Cataldo, M. and Nambiar, S. (2009). On the relationship Carmel, E. and Agarwal, R. (2001). Tactical approaches Buse, R. P. L. and Weimer, W. R. (2010b). Learning Canfora, G., Cerulo, L., and Penta, M. D. (2007). Identi- Buse, R. P. L. and Weimer, W. R. (2010a). Automatically

X—BIBLIOGRAPHY ekr . a,E,Rc,J,Juet . n Rieth, and P., Jaubert, J., Rech, E., Ras, B., Decker, (2002). A. D. Vaus, de M. K. Oliveira, de and N., Anquetil, B., C. S. Souza, de Cre- (2008). S. Minto, and C., G. Murphy, B., Alwis, de ecea . rue .R,adZmemn,A. Zimmermann, and R., C. Prause, S., Dencheva, Dy- (2011). W. Prinz, and R., C. Prause, S., Dencheva, evaluation und Entwicklung (2010). S. Dencheva, (2000). E. W. Deming, (1994). E. W. Deming, mainte- software of study Delphi (1992). S. Dekleva, Implementa- (1979). M. Pozefsky, and E. L. Deimel, 12(3):31–44. nomllearning informal & oenet Education Government, ACM. 20) hc ouetto o otaemainte- software for documentation Which (2006). olbrtv raiiyPoes ikn creativity Linking Process: Creativity Collaborative creativ- fostering - moderation Collaborative (2009). otaeMaintenance Software hpo ehd ol o optrSupported Computer for Tools & Methods on In shop wiki. corporate a with ity Springer. 1–20. pages Work Cooperative Computer-Supported on L., Ciolfi, and V., Wulf, editors, W., Lutters, O., N. S., Bødker, Bouvin, In quality. improve contribution to and wiki participation corporate a in self-moderation namic Universit FIT, Fraunhofer Diplomarbeit, firmeninformationssystem. einem in selbstmoderation dynamischen zur komponente soziotechnischen einer Press. University In problems. nance Conference Regional east science In computer a department. in standards programming of tion 35. in engineering. participation requirements stakeholder Wiki-based (2007). M. edition. fifth Routledge, nance? Engi- Software of neering Aspects Human and Cooperative on diffi- task programming In of culty. metric cognitive a ating HS,pgs2–2 e ok Y USA. NY, York, New 29–32, pages CHASE, , rceig of Proceedings ora fteBaiinCmue Society Computer Brazilian the of Journal rceig fteItrainlWorkshop International the of Proceedings rceig fte1t nulSouth- Annual 17th the of Proceedings rceig fteCneec on Conference the of Proceedings h e cnmc:frIndustry, for Economics: New The . u fteCrisis the of Out ae 01.IEEE. 10–17. pages , I rs,scn edition. second Press, MIT . uvy nSca Research Social in Surveys 12 C Press. ACM . EESoftware IEEE th rceig fteWork- the of Proceedings uoenConference European Cambridge . tBonn. at ¨ 24(2):28– , ECSCW, , - uuework Future — X-2 , . i,D,Mnor . ono,R,adNue,T N. T. Nguyen, and R., Johnson, K., Manzoor, D., Dig, approach pragmatic A (1996). T. Klein, and K. Didrich, (2005). J. Diamond, (1995). DGQ L. Nacke, and R., Khaled, D., Dixon, S., Deterding, uoht .(09.Cmue oea eimfor medium a as code Computer (2009). G. Dubochet, legend. a of Father (2011). J. Drommert, product software for model A (1995). G. R. Dromey, Crowdsourc- (2009). M. Vojnovic, and D. future. DiPalantino, Fragmented (1999). D. DiNucci, Dyb G., Hanssen, T., Dingsøyr, 5 e ok Y S.ACM. USA. NY, York, New 15, 2,NwYr,N,UA ACM. USA. NY, York, New 128, alo Succeed or Fail C ofrneo lcrncCommerce Electronic on Conference ACM M 20) eatrn-wr ofiuainmanagement configuration Refactoring-aware (2007). gamefulness: to elements design game From (2011). ehiceUniversit Technische rcs Improvement Process nentoa ofrneo otaeEngineering Software on Conference International o betoine rgas In programs. object-oriented for 96-4, Report Technical documentation. software to edition. second VDE, im Gesellschaft technische f Gesellschaft Deutsche Environments Media Future ing Envision- Conference: MindTrek Academic national In “gamification”. Defining ftePyhlg fPormigItrs Group Interest Programming PPIG. of Psychology the of In improving? languages programming Are communication: human magazin 21(2):146–162. quality. In auctions. all-pay and ing 53:32,221,222. Heidelberg. Science Computer in editors, R., Messnarz, and P., Richardson, Runeson, In I., project. cross-organizational small scrum a with in software Developing (2006). J. Nygaard, Society. IEEE Computer USA. DC, Washington, 427–436, pages ICSE, gihetn L oglichkeiten, ¨ EETascin nSfwr Engineering Software on Transactions IEEE 9:72–78. , otae-Qualit - Software rceig of Proceedings Penguin. . osungen olpe o oite hoeto Choose Societies How Collapse: ¨ tBerlin. at ¨ oue45 of 4257 volume , ae –5 pigrBri / Berlin Springer 5–15. pages , rQualit ur ¨ et elgGmbH, Verlag Beuth . rceig of Proceedings rceig fthe of Proceedings tsceug Aufgaben, atssicherung: ¨ 21 ,T,Akr . and G., Anker, T., a, ˚ idrk ae 9– pages MindTrek, , rceig of Proceedings st tudNachrichten- und at ¨ nulWorkshop Annual etr Notes Lecture ae 119– pages , 15 Lufthansa Software th Print Inter- 29 10 235 th th , , , ,

X—BIBLIOGRAPHY . , American Proceedings CESifo Work- , Fall:25–43. . Carl Hanser Ver- ¨ orterbuch Psychologie W , pages 113–122. . Pearson Education. ¨ ur die bundesrepublik deutsch- ¨ atssicherung , 43:623–643. International Workshop on Princi- , DocEng, pages 26–33, New York, http://www.martinfowler. Refactoring: Improving the Design Software Quality Assurance: from . Addison-Wesley Longman. th , 245:1–25. 8 Proceedings of the Symposium on Docu- Sloan Management Review . ples of Software Evolution experiments: First statistical interpretation. theory: A survey of empiricaling evidence. Paper Series linked programming documentation. In of the International Workshop on HypermediaIWHD, Design pages 151–173. Springer. dtv, 24 edition. mean? lag. land. ing and maintaining artefacts of communication.ter’s thesis, Mas- Ottawa-Carleton Institute forScience. Computer of software documentation, tools andsurvey. technologies: a In ment Engineering NY, USA. ACM. of Existing Code Blog, online. com/bliki/TechnicalDebtQuadrant. html Sociological Review Proceedings of (2005). How developers drive software evolution. In Handbuch der Qualit ¨ Theory to Implementation ohlich, W. D., editor (2002). ırba, T., Kuhn, A., Seeberger, M., and Ducasse, S. ˆ G Fowler, M. (2006). Codeownership. Online. FP6 (2011). Fp6 at a glance. Franke, R. H. and Kaul, J. D. (1978). The hawthorne Frey, B. S. and Jegen, R. (2000). Motivation crowding Friendly, L. (1995). The design of distributed hyper- Fr Galin, D. (2004). Garvin, D. A. (1984). What does ”product quality” really Geiger, W. (1988). Begriffe. In Masing, W., editor, GG (1949). Grundgesetz f Forward, A. (2002). Software documentation — build- Forward, A. and Lethbridge, T. C. (2002). The relevance Fowler, M. (1999). Fowler, M. (2009). Technicaldebtquadrant (in bliki blog). . , IBLIOGRAPHY . Thomas X—B Companion IEEE Trans- , 12:219–245. Software Met- , 27:2–5. Building Web Repu- , 33(11):725–743. IBM Systems Journal . Cambridge University , page 109, Los Alamitos, ` a vu: The life of software ´ ej Qualitative Inquiry IEEE Software A Theory of Cognitive Dissonance The Academy of Management Re- Social Psychology: Attitudes, Cog- , OOPSLA, pages 318–319. ACM. The Cement of Society. A Study of . O’Reilly. . Cambridge University Press. Proceedings of the International Conference , PLoP. The Hillside Group. , 14(1):57–74. case-study research. grams Stanford University Press. grained source code change extraction. actions on Software Engineering tation Systems rics — A Rigorous andInternational Practical Approach Computer Press, second edition. errors in program development. Press. engineering ideas. ment and review. view nition and Social Behaviour CA, USA. IEEE Computer Society. matic grading meets test-first coding.of In the 18th Annual ACM SIGPLAN Conference on and Applications guage independent approach for detectingcode. duplicated In on Software Maintenance Social Order (2007). Change distilling: Tree differencing for fine- Object-oriented Programming, Systems, Languages, Fourth Conference on Patterns Languages of Pro- 15(3):182–211. 236 Flyvbjerg, B. (2006). Five misunderstandings about Foote, B. and Yoder, J. (1997). Big ball of mud. In Festinger, L. (1957). Fluri, B., Wursch, M., Pinzger, M., and Gall, H. C. Farmer, F. R. and Glass, B. (2010). Fenton, N. E. and Pfleeger, S. L. (1997). Fagan, M. (1976). Design and code inspections to reduce Farlex (2012). The free dictionary. Online. Elster, J. (1989). Erdogmus, H. (2010). D Eisenhardt, K. M. (1989). Agency theory: An assess- Eiser, J. R. (1986). Edwards, S. H. (2003). Teaching software testing: Auto- Ducasse, S., Rieger, M., and Demeyer, S. (1999). A lan-

X—BIBLIOGRAPHY ry . als . n aDnl,S 19) Software (1997). S. MacDonell, and P., Sallis, A., Gray, (2004). P. Graham, a of theory economic The (1954). S. H. Gordon, (2005). P. R. Gabriel, and R. Goldman, rßan .(09.Twrsa applicable an Towards (2009). M. soft- your Großmann, fix to How (2011). Z. Grossbart, to library From reuse: Software (1993). L. M. Griss, (1999). E. D. Perry, and D., J. Herbsleb, E., R. Grinter, work articulation Supporting (1996). E. R. Grinter, M. Williams, and A., Y. Unrau, M., R. Jr., Grinnell, discipline. Mental (2010). A. D. Grier, to partners industry Attracting (2009). M. Grechanik, 10.1007/BF00136714. oescLinguistics Forensic aestcncldb,pormfse,adspend and faster, program debt, technical ware’s 21) ru-ee ein.I rnel r,R M. R. Jr., Grinnell, In designs. Group-level (2011). Notes optrSpotdCoeaieWork Cooperative Supported Computer oiia Economy Political h egah fcodnto:Daigwt distance with Dealing coordination: of geography The Prac- Evidence-Based of Foundations — Evaluation otaeqaiymdlfridvda software individual for model quality software Online. code. useful writing time http://www.zackgrossbart.com/hackito/techdebt/. your of more factory. Press. ACM 306–315. Work Group Supporting on ference In work. r&d in systems. management configuration software using edition. ninth Press, tice editors, A., Y. Unrau, and 8. research. engineering software of Association International the of Conference nual In programs. computer to techniques analysis authorship Extending forensics: Inc. Programming Associates, of & Art O’Reilly the on Essays Age: Computer the fishery. The resource: common-property Strategy Kaufmann. Business Morgan as Source Open Elsewhere: pens hpe 5 ae 5–8.Ofr University Oxford 258–283. pages 15, chapter , 34(6):4. , B ytm Journal Systems IBM rceig fteItrainlCon- International the of Proceedings akr anes i da from Ideas Big Painters: & Hackers 62:124–142. , uhmN,USA. NC, Durham , oilWr eerhand Research Work Social rceig of Proceedings 32(4):548–566. , ISF ot.Eng. Softw. SIGSOFT RU,pages GROUP, , Computer noainHap- Innovation 5:447–465. , ora of Journal 3 - uuework Future — X-2 rd 43:6– , Bian- . . amn .(00.Wysuc oeaayi n ma- and analysis code source Why (2010). M. Harman, commons. the of tragedy The (1968). G. Hardin, (2010). G. Antoniol, and F., Khomh, N., Haderer, (2000). M. Sommer, and H.-P. Gumm, ekasn .adGlwte,P .(97.Thought (1987). M. P. Gollwitzer, and H. Heckhausen, are You Metrics: (1998). G. Katz, and J. Hauser, Refin- (2010). R. Robbes, and M., Lanza, L., Hattori, of history the Mining (2009). M. Lanza, and L. Hattori, Historage: (2011). T. Kikuno, and O., Mizuno, H., Hata, (2005). R. Schroeder, and M. Harry, experiments. Field (2004). A. J. List, and W. G. Harrison, 162:1243–1248. 11110 10.1007/BF00992338. 11:101–120. 16(5):517–528. 10.1007/s10664-010-9145-5. ora fEooi Literature Economic of Journal Qualit htyumeasure! you what oeAayi n Manipulation and Analysis Code ehiceUniversit Technische ratruhMngmn taeyRevolutionizing Strategy Management Breakthrough o lmts A S.IE optrSociety. Computer IEEE USA. CA, Alamitos, Los Source on Conference Working International IEEE In of important. be always will nipulation Maintenance Software on Conference tional In of quality systems. the software monitoring for framework A Squaner: Informatik die matik. In projects. u oiinlsae fmind. of states volitional sus ver- motivational in functioning cognitive and contents Engineering Software Empirical of changes. nal synchronous with ownership code ing Society. Computer IEEE USA. CA, tos, Soft- Repositories Mining ware on Workshop International of In ceedings ownership. code refine to changes synchronous Evolution Software on and Evolution Software on ples In of java. ceedings for system control version Fine-grained Corporations Top World’s the tmdlirn n -bewertung und atsmodellierung ¨ augbn .Wrso u Software- zur Workshop 2. Tagungsband 12 lebug edition. 4 Oldenbourg, . th nentoa okhpo Princi- on Workshop International S,pgs1110 o Alami- Los 141–150, pages MSR, , uoenMngmn Journal Management European tM at ¨ ae 610 C Press. ACM 96–100. pages , rceig fIE Interna- IEEE of Proceedings nhn ntttf Institut unchen, ¨ oiainadEmotion and Motivation urny rtedition. first Currency, . 7 XLII:1009–1055. , CM ae 7–19, pages SCAM, , th RI Workshop ERCIM i im The – Sigma Six Einf ae 1–33. pages , Proceedings M , hugin uhrung ¨ ICSM. , rInfor- ur ¨ Science unchen. ¨ Jour- Pro- Pro- 237 , , ,

X—BIBLIOGRAPHY . , , , Java ¨ , OOP- ur soft- ¨ unchen. Software, IEEE Der pragmatische Managing Technical People Project Management Journal The Ethics and Economics of , 7:108. , chapter 2. Edward Elgar. . Carl Hanser Verlag M , 23:88–95. Companion to the 19th Annual ACM , 4:86–87. Personality and Social Psychology Bulletin concern with being right andliked. the concern with being Cooperation theme 3 ”ict -cations information technologies“ and (version communi- v1). TechnicalEuropean report, Commission C. standard for software reviews and audits. developing a software project life cycle process. software engineering terminology. software test documentation. management journals. is easy. In ming Systems, Languages, andSLA. Applications ACM. quality investment is enough: A value-based approach. process: a maturity5(2):73–79. framework. IEEE Software SIGPLAN Conference on Object-oriented Program- Taylor, S. (1985). Conformity and group size: The Programmierer Private Property Bild der Wissenschaft 40(4):6–19. Addison-Wesley. warearchitekten — antimuster: der codeheld. Magazin 11(1):41–50. Humphrey, W. S. (1997). Hunt, A. and Thomas, D. (2003). ICT Work Programme (2012). Work programme 2013: IEEE Std. 1028 (2008). IEEE Std. 1028-2008: Ieee IEEE Std. 1074 (2006). IEEE Std. 1074 – standard for IEEE Std. 610.12 (1990). IEEE standard glossary of IEEE Std. 829 (1998). IEEE Std. 829 – standardIka, for L. A. (2009). Project success as a topic in project Insko, C. A., Smith, R. H., Alicke, M. D., Wade, J., and Hoppe, H.-H. (2004). Horeis, H. (2009). Kein nobelpreis - trotzdem spitze. Hovemeyer, D. and Pugh, W. (2004). Finding bugs Hruschka, P. and Starke, G. (2011). Knigge f Huang, L. and Boehm, B. (2006). How much software Humphrey, W. S. (1988). Characterizing the software , IBLIOGRAPHY , pages X—B IEEE Com- . ACM New International Pragmatische , ICSE, pages Proceedings of st 21 , 18(2):16–20. Compensation & Beneﬁts Proceedings of the National ¨ atsverlag Konstanz. Proceedings of , 16(5):63–70. IEEE Software Proceedings of the 2008 international . Universit , 27(63):63–68. , 34(9):120–127. , ICSE, pages 81–90. Online Communities and Social Computing In 362–371, Berlin Heidelberg New York. Springer. scientiﬁc research output. tions using social rewarding techniques — a focus on large commits tell us? -commits. a taxonomical In study of large workshop on Mining software repositories ssystemen development: The business of innovation. puter the International Conference on Softwareing Engineer- development. law revisited. In 85–95. ACM Press. R. E. (2001). An empiricaldevelopment: study Distance of and global speed. software In coordination, and distance: Conway’s law and beyond. the organization and integrating the code: Conway’s group and individual rewards: Rewarding individual contributions to the team. IEEE Software Technology. Review Conference on Software Engineering wiki systems. Master’s thesis, Vienna University of warding in wiki systems — motivating the community. York, NY, USA. Academy of Sciences of the United States of America Aspekte beim Entwurf und Betrieb von Information- 102(46):16569–16572. 238 Hoisl, B., Aigner, W., and Miksch, S. (2007). Social re- Hirsch, J. E. (2005). An index to quantify an individual’s Hoisl, B. (2007). Motivate online community contribu- Hindle, A., German, D. M., and Holt, R. (2008). What do Highsmith, J. and Cockburn, A. (2001). Agile software Herbsleb, J. D. and Moitra, D. (2001). Global software Herget, J. and Kuhlen, R., editors (1990). Herbsleb, J. D., Mockus, A., Finholt, T. A., and Grinter, Herbsleb, J. D. and Grinter, R. E. (1999a). Architectures, Herbsleb, J. D. and Grinter, R. E. (1999b). Splitting Heneman, R. L. and Hippel, C. V. (1995). Balancing

X—BIBLIOGRAPHY S 00(05.IO90:05 ult management Quality 9000:2005: ISO (2005). 9000 ISO engineer- Software 25000: ISO/IEC (2005). 25000 ISO Information 15504-1: ISO/IEC (2004). 15504-1 ISO Software 14764: Std ISO/IEC/IEEE (2006). 14764 ISO design Human-centered 13407: ISO (1999). 13407 ISO ono,P .adTajn,D 19) Assessing (1997). D. Tjahjono, and M. P. Johnson, sys- Choose dilemma: Lean (2007). T. H. Johnson, Jim A., N. Kraft, D., Hale, K., R. Smith, E., J. Hale, G., Jay, (2005). P. Jalote, F., Pramudianto, R., C. Prause, M., Jentsch, M., Jahn, engi- Software 9126-3: ISO/IEC (2001). 9126-3 ISO engi- Software 9126-1: ISO/IEC (2001). 9126-1 ISO 1–2,NwYr,N,UA C Press. ACM USA. NY, York, New 118–127, Wiley. John 16. lAkd . n enr,R 21) h energy The (2010). R. Reiners, and A., Al-Akkad, etPatcsfrSsanbeIntegration Sustainable for Practices Best Engineering eomn:Asseai review. systematic A velopment: ainlcneec nSfwr engineering Software on conference In national csrs. using experimental study controlled a meetings: review software editor, J., Stenzel, In both. controls, accounting not management or principles tem Engineering ware de- software distributed in improvements and lenges linear plications stable a of evidence relationship. and Empirical complexity code: Cyclomatic of lines (2009). C. Ward, and Society. Computer IEEE 1–8. pages FutureTech, Technology Information Future on Conference In tional home. smart aware metrics. Internal 3: Part quality: product – neering model. Quality 1: Part quality: product – neering vocabulary. and fundamentals – systems square. to guide eval- — and (square) requirements uation quality product software – ing Concepts 1: vocabulary. part and – assessment process – technology mainte- – processes nance. cycle life software – engineering systems. interactive for processes nz . itii . n Vizca and M., Piattini, M., enez, ´ 2:137–143. , pigr hr edition. third Springer, . ora fSfwr niern Ap- & Engineering Software of Journal nItgae praht Software to Approach Integrated An oue2009:1–14. Volume , rceig fte1t inter- 19th the of Proceedings rceig f5hInterna- 5th of Proceedings ´ n,A 20) Chal- (2009). A. ıno, enAccounting: Lean dacsi Soft- in Advances ae 3– pages , - uuework Future — X-2 pages , , imre .adCes .(07.Gopawareness Group (2007). U. Cress, and J. Kimmerle, development: distributed in Experiences (2003). L. Kiel, reputa- a of value The (2003). A. Schiff, and J. Kennes, (2012). M. Kanat-Alexander, documentation of state The (2001). M. Kajko-Mattsson, of survey A (2007). C. Boyd, and R., Ismail, A., Jøsang, oh .S 20a.Tepol premium. people The (2005a). S. A. Koch, (1992). E. D. Knuth, (2007). R. Kneuper, wikipedia: Beyond (2010). E. R. Kraut, and A. Kittur, Software (1996). L. S. Pﬂeeger, and B. Kitchenham, Wal- J., Kanerva, M., DeMoney, P., Naughton, P., King, okhpo lblSfwr Development Software Global on Workshop Maintenance aesuy In study. case A cec fSfwr Development Software of Science otae n ytmnwclnsrzse i Ca- mit Systementwicklungsprozessen und Software- ae 44–47. pages system. tion Society. Computer IEEE USA. CA, Software on Conference International IEEE of ings In maintenance. corrective within practice provi- service online sion. for systems reputation and trust rjcsWr Journal. Projects@Work com/content/articles/227504.cfm Online. Information. and Language of Study the edition. 3 (CMMI-DEV) dpunkt.verlag, Integration Model Maturity pability ACM. USA. NY, York, New Work Cooperative ported groups. In production online in conﬂict and Coordination 21. target. elusive The quality: conven- code Java (1997). al. tions. et S., Hommel, K., rath, Sciences. Learning the of Society Learning Collabo- rative Supported Computer on Conference In tional approach. interactional information-exchange an the dilemma: in self-presentation and rceig fteCneec nCmue Sup- Computer on Conference the of Proceedings eiinSpotSystems Support Decision nutilOrganization Industrial CM ae34(i!,LsAlamitos, Los (sic!), 354 page ICSM, , http://www.projectsatwork. SL ae 7–7.International 370–378. pages CSCL, , rceig fteInternational the of Proceedings ieaeProgramming Literate MI–Vresrn von Verbesserung – CMMI SW ae 215–224, pages CSCW, , EESoftware IEEE oeSmlct The — Simplicity Code 43(2):618–644. , O’Reilly. . 0301011:1–31. , 8 etrfor Center . th 13(1):12– , Proceed- Interna- GSD, , . 239 .

X—BIBLIOGRAPHY , , ¨ o ¨ axj Inter- Soviet th 5 The Wiki Way — Communications , 6(4):308–320. . Carl Hanser Verlag . Addison-Wesley. . PhD thesis, V ¨ at , 20(6):35–39. Proceedings of , 17(3):6–17. ˚ Agerfalk, P., and Fitzgerald, B. . Auerbach. The Transfer and Diffusion of In- Software Testing and Continuous , pages 119–137. Springer Boston. Software Dokumentation und Mes- Interfaces , 10(8):707–710. Validation of a Standard- and Metric- IEEE Software IFIP International Federation for Infor- , 24(11):763–769. Advances in Computer Science and Engi- The Computer Journal . ) , 4(1):23–36. 3 c ( ¨ unchen. ically distributed work with communicationthe tools in remote district sme’s. In national Conference on Global Software Engineering ICGSE, pages 155–164. IEEE Computer Society. recting deletions, insertions, and reversals. application software maintenance. of the ACM University. opment. In Donnellan, B., Larsen,DeGross, T., J., Levine, editors, L., and formation Technology for Organizational Resilience mation Processing pressions. tions neering sung der Dokumentationsqualit M software methodologies: A case ofcess innovation technology. in pro- How software engineers use documentation: Theof state the practice. J. (2010). Supporting collaboration in the geograph- volume 206 of Physics-Doklady Based Software Quality Model (2006). Ten strategies for successful distributed devel- Quality Improvement Quick Collaboration on the Web Levenshtein, V. I. (1966). Binary codes capable of cor- Lewis, W. E. (2005). Lientz, B. P. and Swanson, E. B. (1981). Problems in Lincke, R. (2009). Lings, B., Lundell, B., Liukkunen, K., Lindberg, K., Hyysalo, J., and Markkula, Landin, P. J. (1964). The mechanical evaluation of ex- Laramee, R. S. (2010). Bob’s concise coding conven- Lehner, F. (1994). Leonard-Barton, D. (1987). Implementing structured Lethbridge, T. C., Singer, J., and Forward, A. (2003). Leuf, B. and Cunningham, W. (2001). , , IBLIOGRAPHY Pro- Pro- . . , ESEM, X—B Oxford Re- Working Conference th 4 , 17:248–264. , pages 44–54. IEEE. Journal of Economic Theory Communications of the ACM , 43(11):22–28. , pages 11–20, New York, NY, USA. International Symposium on Empiri- Proceedings of the 2005 international http://www.projectsatwork. http://www.projectsatwork. st , ICSM, pages 304–313. IEEE. 1 Proceedings of . Wiley-VCH. Computer ¨ uhren mit Psychologie: die Managementpraxis fest dot: Effects of feedback oncommunity. new members In in an online work (GROUP) tenance in norway: Aceedings survey of investigation. the International In Conference on Software im Griff 38:69–81. imperfect information. 27(2):253–279. P., and Thomas, J. C. (2010). Scientiﬁc foundations: theory. software development. M. (2007). Evaluating the impactnance of process adaptive on mainte- open source softwareceedings quality. of In cal Software Engineering and Measurement pages 186 –195. IEEE Computer Society. com/content/articles/228609.cfm com/content/articles/229539.cfm view of Economic Policy the detection of programming patternsmetrics. using software In on Reverse Engineering line. Projects@Work Journal. development and distributed innovation. Online. Projects@Work Journal. ACM. A case for technology-mediated social-participation Maintenance ACM SIGGROUP conference on Supporting group F 240 Lampe, C. and Johnston, E. (2005). Follow the (slash) Krumbach-Mollenhauer, P. and Lehment, T. (2007). Kreps, D. M. and Wilson, R. (1982). Reputation and Krogstie, J. and Sølvberg, A. (1994). Software main- Kraut, R., Maher, M. L., Olson, J., Malone, T. W., Pirolli, Kraut, R. E. and Streeter, L. A. (1995). Coordination in Kozlov, D., Koskinen, J., Markkula, J., and Sakkinen, Kontogiannis, K. (1997). Evaluation experiments on Koch, A. S. (2006). The role of tools.Kogut, B. and Metiu, A. (2001). On- Open-source software Koch, A. S. (2005b). Processes for people.

X—BIBLIOGRAPHY orne .adWge,R .(95.A extension An (1975). A. R. Wagner, and R. Lowrance, prevent to tools source open Using (2009). S. Loveland, develop- software in wikis Using (2006). P. Louridas, Ruta, E., J. Tarver, D., Pisani, E., K. Davis, T., G. Lloyd, cdm .adMCed,S 20) rtqeof critique A (2000). S. McCreedy, and R. McAdam, (2011). P. Marwedel, (1983). L. C. McClure, and J. Martin, E. S. Bledsoe, and L., J. Bellamy, L., Fang, J., Manuel, Cost/bene- (1988). J. T. Teorey, and M. M. Mantei, (2008). P. Choudhary, and K. Malik, Cor- (2006). D. Yates, and C., Wagner, A., Majchrzak, con- Organisational (1998). P. Layzell, and A. Lynex, 15(2):155–168. rciinrsApproach Practitioner’s A C Press. ACM In code. write-only ofrneo nomto ehooy e Genera- New Technology: Information on Conference 21) vlaigeiec.I rnel r,R M. R. Jr., Grinnell, In evidence. Evaluating (2011). B vlain—Fudtoso vdneBsdPrac- Evidence-Based of Foundations — Evaluation Engineering s model. construction- ist social a using management: knowledge Sys- tems Cyber-Physical of Foundations System bedded Solutions its and Problem The nance: edition. ninth Press, tice editors, A., Y. Unrau, and the 31(4):428–439. in factors lifecycle. human software incorporating for analysis fit (WikiSym) Wikis on Symposium In International survey. the a of of Results users: wiki porate reuse. software for siderations ACM the problem. correction string-to-string the of Society. Computer tions ment. cretaceous the and revolution. terrestrial Dinosaurs and (2008). R., J. Jennings, M. E., W. Benton, D. Hone, M., Sakamoto, M., 275:2483–2490. , hpe ,pgs1519 xodUniversity Oxford 145–159. pages 9, chapter , pigr eodedition. second Springer, . ae 7–7,LsAaio,C,UA IEEE USA. CA, Alamitos, Los 671–677, pages , EESoftware IEEE 22(2):177–183. , e ehooy okadEmployment and Work Technology, New :0–2.10.1023/A:1018928608749. 5:105–124. , meddSse ein—Em- — Design System Embedded rceig of Proceedings rceig fteRylSociety Royal the of Proceedings 23(2):88–91. , omnctoso h ACM the of Communications aaMcGraw-Hill. Tata . oilWr eerhand Research Work Social naso Software of Annals otaeQaiy— Quality Software 6 otaeMainte- Software th rnieHall. Prentice . International Proceedings ora of Journal - uuework Future — X-2 , , . oks . ilig .T,adHrse,J .(2002). D. J. Herbsleb, and T., R. Fielding, A., Mockus, for system Computer-implemented (1999). P. Merel, Mellstr (2004). C. Ambroise, and K.-A., Do, J., G. McLachlan, (1977). F. G. Walters, and K., P. Richards, A., J. McCall, measure. complexity A (1976). T. McCabe, emi,I,Fse,J . n ik,M 20) Under- (2005). M. Hicks, and S., J. Foster, I., Neamtiu, string approximate to tour guided A (2001). G. Navarro, (1969). editors B., Randell, and P. Naur, Prototyping: (1982). M. A. Jenkins, and D. J. Naumann, (1996). M. A. Brandenburger, and J. B. Nalebuff, Expertise (2002). D. J. Herbsleb, and A. Mockus, okhpo iigSfwr Repositories Software Mining on Workshop nlzn iraryGn xrsinData Expression Gene Microarray Analyzing ok Y S.ACM. USA. NY, York, t-1hOtbr1968 October 7th-11th ie Sons. & Wiley Quarterly w aesuiso pnsuc otaedevelop- software source open of studies case Two niern eoto ofrnesosrdby sponsored conference a of Report Engineering development. systems for paradigm new The otaeEgneigadMethodology and Engineering Software et pceadmozilla. and Apache ment: policies. and resources controlling Association Economic right? European titmuss the was donation: blood in out Company. Electric General 1 Vol. 3, Report of Technical quality. software in Factors Engineering Software on actions e ok Y S.ACM. USA. NY, York, New In matching. syntax tree abstract using evolution code source standing matching. NATO. Germany Garmisch Committee Science NATO the Spieltheorie Unternehmenserfolg der zum mit - konkurrieren kooperativ tition: Engineering Software on In expertise. identifying to approach quantitative A browser: 346. m .adJhnesn .(08.Crowding (2008). M. Johannesson, and C. om, ¨ rceig of Proceedings 6(3):29–44. , C optn Surveys Computing ACM rceig fteInternational the of Proceedings 24 cetfi far Division, Affairs Scientific . CE ae 0–1,New 503–512, pages ICSE, , Campus-Verlag. . th nentoa Conference International C rnatoson Transactions ACM 2(4):308–320. , 33(1):31–88. , 6(4):845–863. , 11(3):309– , EETrans- IEEE ora of Journal Software MSR, , Coope- John . MIS 241

X—BIBLIOGRAPHY . , Man- ¨ orner, C., Journal of ¨ at Bonn. International . PhD thesis, . Project Man- st Proceedings of 1 Guide to the Soft- ¨ atssicherungsmethoden Experimental and Analytical , 30:187–199. Proceedings of Comtemporary clinical psychology , 44(7):973–979. A Guide to the Project Management , SEW, pages 99–108. IEEE. , ICST, pages 258–267. International IEEE/NASA Software Engineering st mers, A. B. (2007a). Interconnectingharnessing documentation the - different powers of currenttation documen- tools in software development. In Cardoso, J., tion of internal software quality. In agement Institute, fourth edition. Incentives and domain-specific software. agement Decision recall and f-factor to roc, informedness, markedness Informatics and Engineering, Flinders University of South Australia. dokumentations- und qualit am beispiel derRheinische Friedrich-Wilhelms-Universit fit-lernplattform. Diplomarbeit, Faculty of the Graduate School University ofCalifornia. Southern ware Engineering Body ofchapter Knowledge Software (SWEBOK) Maintenance (ChapterComputer Society, 6). 2004 edition edition. IEEE Schiffer, S. (2008). Tool supportcode for assessments. expert-centred In idation Pomberger, G., Schiffer, S., Saft, M., and Storck, S. John Wiley & Sons. 31 Systems and Software Body of Knowledge (PMBOK Guide) (2007). The emisq method — expert based evalua- Comparison between Pair Development and Software Conference on Software Testing, Verification, and Val- Development with Fagan’s Inspection & correlation. Technical report sie-07-001, School of ¨ ¨ Workshop osch, R., Gruber, H., Pomberger, G., Saft, M., and osch, R., Gruber, H., Hentschel, A., K Prause, C., Kuck, J., Apelt, S., Oppermann, R., and Cre- PMBOK (2008). Poulin, J. S. (1995). Populating software repositories: Powell, S. (2006). Spotlight on richard t. pascale. Powers, D. M. W. (2007). Evaluation: From precision, Prause, C. (2006). Design und evaluation von Phongpaibul, M. (2007). Pigoski, T. M. and April, A. (2004). Plante, T. G. (2011). Pl Pl nd IBLIOGRAPHY 2 Com- , pages Annual . X—B nd . 22 Proceedings of Proceedings of Proceedings of the In- , PROMISE, pages 1–10, New , 11(4):36–45. , 20(2):26–33. , MSR, New York, NY, USA. ACM. Lehrbuch der Personalpsychologie IEEE Software , PPIG. , 26(11):32–41. http://thedailywtf.com/Articles/ International Conference on Predictive Models in , SE, pages 58–65. ¨ ottingen. th on memory for programming tasks. In ment. dictionary of current english. line. Documentation-Done-Right.aspx Programmer-based fault prediction. In 6 online documentation.http://radar.oreilly.com/archives/ O’Reilly2007/09/recent_conversa.html radar, online. development environment by recordingerations editing of op- source code.ternational Working Conference In on Mining Software 349–371. Hogrefe Verlag, RohnswegG 25, D-37085 puter P. (2006). What software evolution and biologicallution evo- don’t have in common. In ity H., editor, environments. EFEPLE. International IEEE Workshop on Software Evolvabil- Software Engineering Software, IEEE Repositories (1994). People, organizations, and process improve- Group Meeting of the Psychology of Programming Interest York, NY, USA. ACM. 242 Perry, D. E., Staudenmayer, N. A., and Votta, L. G. Papadimoulis, A. (2010). Documentation done right. on- Parnin, C. (2010). A cognitive neuroscience perspective Ostrand, T. J., Weyuker, E. J., and Bell, R. M. (2010). Oxford Dictionary (1994). Oxford advanced learners Oram, A. (2007). Recent conversations about Omori, T. and Maruyama, K. (2008). A change-aware Nielsen, J. (1993). Iterative user-interface design. Nordberg, III, M. E. (2003). Managing code ownership. Nerdinger, F. W. (2001). Motivierung. In Schuler, Nehaniv, C. L. (2011). Evolvability of personal learning Nehaniv, C. L., Hewitt, J., Christiansen, B., and Wernick,

X—BIBLIOGRAPHY rue .R 21) otaepoetperspective project software A (2011). R. C. Prause, code fine-grained Maintaining (2009). R. C. Prause, rue .R,Nne,J,adVnois .(02.A (2012). M. Vinkovits, and J., Nonnen, R., C. Prause, (2011). M. Eisenhauer, and M., Jentsch, R., C. Prause, results First (2012). M. Eisenhauer, and R. C. Prause, as- Social (2008). M. Eisenhauer, and R. C. Prause, de- Architectural (2012). Z. Durdik, and R. C. Prause, for approach An (2008). S. Apelt, and R. C. Prause, oQ e ok Y S.ACM. USA. NY, York, New WoSQ, ua set fSfwr Engineering Software of Aspects Human ACM. D., B. R. C. Souza, G., Venolia, B., Tessem, D., A. ofrneo oreCd nlssadManipula- and Analysis Code Source on Conference eerh(IJHCR) Research Personal of Evolvability and Fitness the Exploring erigEnvironments Learning otaeEngineering Software edeprmn ngmfiaino oeqaiyin quality In code development. of agile gamification on experiment field work- warehouse for ers. system support mobile a — Mica ACM. USA. NY, York, New code. source and In articles developer wiki of from validity derived reputation the into investigation an from of Aspects Human and Cooperative editors, on E., Workshop S. the Sim, and Sharp, J., F., Singer, Maurer, H., O., Hazzan, M., M. John, Y., Storey, Dittrich, J., Silito, L.-T., Cheng, In software code. for source platform inspection continuous a of pects Society. Process System and ware development? agile In in Waste documentation: and sign In of code. source of inspection continuous learning personal In of evolvability environments. and fitness the on tion merging. and In copying moving, of regardless metadata INSTICC. Systems tion editors, 9 J., Filipe, and J., Cordeiro, th 6 rceig fteWrso nCoeaieand Cooperative on Workshop the of Proceedings rceig fItrainlCneec nSoft- on Conference International of Proceedings rceig of Proceedings CM EECmue Society. Computer IEEE SCAM. , nentoa ofrneo nepieInforma- Enterprise on Conference International th nentoa ora fHnhl Computing Handheld of Journal International nentoa okhpo otaeQuality Software on Workshop International oueIA of ISAS volume , 2(1):1–24. , rceig of Proceedings 9 th HS,NwYr,N,USA. NY, York, New CHASE, , rceig ftePsychology the of Proceedings EFEPLE. , EEItrainlWorking International IEEE CS.IE Computer IEEE ICSSP. , ICEIS rceig fthe of Proceedings 1 st rceig of Proceedings ae 63–68. pages , okhpon Workshop Proceedings CHASE, , - uuework Future — X-2 , rue .R,Shle,M,Zmemn,A,Reiners, A., Zimmermann, M., Scholten, R., C. Prause, Zimmer- and S., Dencheva, R., Reiners, R., C. Prause, (2010a). S. Dencheva, and R., Reiners, R., C. Prause, process software A (2011). R. Reiners, and R. C. Prause, aii,S 20) aiet lct cynicism. elicits Manifesto (2001). S. Rakitin, experi- Ownership, (2011). P. Devanbu, and F. Rahman, and Ownership (2010). P. Devanbu, and F. Rahman, (2001). S. R. Pressman, study Case (2012). A. Zimmermann, and R. C. Prause, Scholten, S., Apelt, T., Jong, de S., Ternier, R., C. Prause, ofrne2011 Conference Computer neetGopWr-nPors Meeting Work-in-Progress Group Interest eurmnspoesi ut-ainlpoetusing project multi-national a in iterative process the requirements Managing (2008). M. Eisenhauer, and R., In high- code. maintaining source for quality Incentives (2010b). A. mann, Society. Computer IEEE 23–32. pages Engineering ICGSE, Software Global on Conference tional In distributed highly projects. in research support tool of study Empirical In evolution. and and creation time widget of reduce costs to development ple for framework appear). to publication, for (accepted pages PPIG, Conference Annual Group Interest Programming of nSfwr Engineering Software on authorship. of study In fine-grained A defects: and ence report, Davis. Technical California, of University code. fix-inducing in experience edition. fifth Approach titioner’s JSME). at review third (Under FIT. Technical Fraunhofer project. report, consortium of a stones in grinding partners the multiple between process software a of (LODE’07) Exchange & Ob- Discovery Learning ject on Workshop International editor, First F., the Assche, of van In MACE. reposito- in object ries learning Unifying (2007b). and E. M., Duval, Specht, B., Vandeputte, M., Eisenhauer, M., Society. Computer IEEE 151–159. pages Engineering ICGSE, Software Global on Conference tional In tracker. issue an rceig fthe of Proceedings 34(12):4. , ae –.webscience. 1–5. pages , crwHl,NwYr,NY, York, New McGraw-Hill, . rceig fthe of Proceedings rceig fthe of Proceedings otaeEgneig Prac- A Engineering: Software 33 scooyo Programming of Psychology rd CE ACM. ICSE. , nentoa Conference International . Proceedings 3 5 PGWiP. PPIG , rd th h PLE The Interna- Interna- IEEE 243 , , ,

X—BIBLIOGRAPHY , , , Pro- ISESE , ICSE, SIGSOFT , 14(2):131– Usability En- , 25(1):54–67. IET Software IEEE International , MSR, New York, , pages 137–142, Al- th , volume 2 of 4 , 31:1–2. International Conference on . Morgan Kaufmann. th 9 International Symposium on Em- th 5 ¨ ost, M. (2009). Guidelines for conduct- Proceedings of the Proceedings of the International Workshop Empirical Software Engineering Software Process Improvement and Practice ¨ ¨ oter, A., Zimmermann, T., Premraj, R., and Zeller, oter, A., Zimmermann, T., Premraj, R., and Zeller, sic motivations: Classic definitions and new directions. rithms. In on Mining Software Repositories NY, USA. ACM. in european embedded software development organ- isations: a survey onof the extreme actual programming use and scrum. and2(1):58–64. usefulness mation system evolution: the minimaledocology. method- ceedings of the pirical Software Engineering pages 18–20. of code. In buquerque, USA. IEEE Computer Society. gineering — Scenario-based Development of Human- large software systems: Concepts and techniques. In vention (WESCon), August 1970, pagespages 1-9) 328–338, Los Alamitos, CA,puter USA. Society IEEE Press. Com- ing and reporting case study researchneering. in software engi- Software Engineering Notes Symposium on Software Metrics Software Engineering (Reprinted from Proceedings of Proceedings of the Contemporary Educational Psychology (2006). Detecting similar java classes using tree algo- Computer Interaction A. (2006a). Where do bugs come from? A. (2006). If your bug database could talk... In Technical Papers of Western Electronic Show and Con- 164. 14:337–347. Schr Ryan, R. M. and Deci, E. L. (2000). Intrinsic and extrin- Sager, T., Bernstein, A., Pinzger, M., and Kiefer, C. Salo, O. and Abrahamsson, P. (2008). Agile methods Salvaneschi, P. (2009). Managing knowledge for infor- Schr Rosenberg, J. (1997). Some misconceptions about lines Rosson, M. B. and Carroll, J. M. (2002). Royce, W. W. (1987). Managing the development of Runeson, P. and H , IBLIOGRAPHY . Free X—B . Pragmatic , MSR. IEEE Mastering the Ship it! A Prac- International Confer- , ICSE, pages 541–550. Proceedings of the Ninth Eu- Computer Science - Research ¨ . Addison-Wesley, Upper Sad- andert die Bildungsforschung Diffusion of Innovations , pages 325–345. Springer-Verlag. Conference on Pattern Languages , 3(2):64–62 (sic!). , 27:95–111. th IEEE Transactions on Software Engi- Proceedings of the International Work- 19 , PLoP, page (to appear). ACM. , 28(12):1135–1145. ACM Queue Technologie ver 2(4):1–17. trol of xml documents. and Development dle River, NJ, second edition. Press, NY, USA, fifth edition. repository. In shop on Mining Software Repositories Computer Society Washington, DC, USA. for evolution research. pages 155–164,CA, Los USA. Alamitos, IEEE Computer Society. Open source software peer review practices:study of A the case apache server.ence In on Software Engineering F. D. (2002). Explainingtance software of developer methodologies: accep- A comparisonretical models. of five theo- neering gineering. In of Programs tical Guide to Successful Software Projects Bookshelf. Gellersen, H., Schmidt,and K., Mackay, Beaudouin-Lafon, W., editors, M., ropean Conference on Computer-Supported Coopera- tive Work (ECSCW) code. in a heterogeneous organizational environment. In Requirements Process (2012). An approach to evolutionary design pattern en- way. ACM. ¨ onnau, S. and Borghoff, U. (2012). Xcc: change con- 244 Rohs, M. (2007). Qualitative forschung auf dem wiki R Robertson, S. and Robertson, J. (1999). Rogers, E. M. (2003). Robbes, R. and Lanza, M. (2005). Versioning systems Robbes, R. (2007). Mining a change-based software Riemenschneider, C. K., Hardgrave, B. C., , and Davis, Rigby, P. C., German, D. M., and Storey, M.-A. (2008). Richardson, J. and Gwaltney, W. (2005). Reiners, R., Halvorsrud, R., Eide, A. W., and Pohl, D. Raskin, J. (2005). Comments are more important than Reichling, T. and Veith, M. (2005). Expertise sharing

X—BIBLIOGRAPHY ehe,B 20) h araeo w rcs worlds. process two of marriage The (2009). B. Sechser, empirical An (1997). R. V. Basili, and B. C. Seaman, Schr mt,A (1776). A. Smith, (2008). F. J. Smart, a of bit a was It (2012b). K. Schneider, and L. Singer, the Influencing (2012a). K. Schneider, and L. Singer, Travas- C., J. Maldonado, J., Carver, V., Bisili, F., Shull, anyone? documentation, Agile (2009). B. Selic, (2009). P. Seibel, (1997). J. Seddon, gis S 9000 ISO Against rf fProgramming of Craft otaeEngineering Software Software Practice and Improvement Process Software London. nations of wealth the of edition. first the In of control. version of Gamification race: IEEE. 1325–1328. Engineering pages Software on Conference tional In so- software. using cial methods engineering software of adoption Society. puter Soft- Empirical Engineering on ware Symposium International the In of problem. knowledge tacit the Ad- dressing experiments: engineering software Mendon Replicating H., G. sos, Ireland. 2, Dublin Street, Merrion Lower 354. Press. Engineering ware the In of inspections. ceedings code in communication of study ACM. USA. NY, York, New Engineering Software pirical the In of time. ceedings design at failures component Predicting tr . imran . n elr .(2006b). A. Zeller, and T., Zimmermann, A., oter, ¨ 2 nd 26(6):11–12. , nentoa okhpo ae and Games on Workshop International 19 niqiyit h aueadcauses and nature the into inquiry An 5 oesa ok eetoso the on Reflections Work: at Coders th a rePes ero Building, Merrion Press, Tree Oak . aaPwrTools Power Java CE e ok Y S.ACM USA. NY, York, New ICSE, , th SS,pgs71.IE Com- IEEE 7–16. pages ISESE, , nPruto ult:TeCase The Quality: of Persuit In rceig fthe of Proceedings nentoa ypsu nEm- on Symposium International nentoa ofrneo Soft- on Conference International ç S,pgs5–8. pages GSE, , Apress. . ,M,adFbr,S (2002). S. Fabbri, and M., a, .SrhnadT Cadell, T. and Strahan W. . SS,pgs18–27, pages ISESE, , ’elyMedia, O’Reilly . 34 Proceedings Proceedings th 14:349– , - uuework Future — X-2 Interna- ICSE, , IEEE Pro- Pro- plk,J (2004). J. Spolsky, (2003). D. Spinellis, tcy .D (1996). D. R. Stacey, edoc. elyts (2011). D. Spinellis, documentation. Code (2010). D. Spinellis, (2006). D. Spinellis, development integrated An Gide: (2005). D. Smith, ily .R 19) ouetn-ntelrevs. Documenting-in-the-large (1993). R. S. Tilley, Correctness (2008). M. Holcombe, and C. Thomson, (2004). A. Hunt, and D. Thomas, (2001). S. A. Tanenbaum, prod- new new The (1986). I. Nonaka, and H. Takeuchi, visible. work Making (1995). L. Suchman, (2003). D. Rosenberg, and M. Stephens, (2012). D. R. Stacey, 103. 112–119. ofrneo optn Education Computing on Conference olbrtv eerh(CASCON) research Collaborative Perspective Perspective Repositories ganizations ware In feedback. student and methodologies programming agile on focusing environment B Press. on IBM Studies Advanced for Centre the of conference In documenting-in-the-small. ACM. USA. Software Mining In on Conference Working cvs. ternational from mined data of buchverlag. CVS mit Versionsverwaltung - mieren Hall. Prentice 64(1):137–146. game. development uct ACM the of tions edition. XP first Against Case The Refactored: gramming complex- of ity challenge the Meeting management: and Routledge. . 27:18–19. , dio Wesley. Addison . dio Wesley. Addison . Berrett-Koehler. . S,pgs1710 e ok NY, York, New 117–120, pages MSR, , 38:56–64. , olo Software on Joel ol n ehiuso leadership of Techniques and Tools oeRaig–TeOe Source Open The – Reading Code oeQaiy h pnSource Open The Quality: Code opeiyadCetvt nOr- in Creativity and Complexity oenOeaigSystems Operating Modern avr uiesReview Business Harvard rceig fte1993 the of Proceedings EESoftware IEEE rceig fteIn- the of Proceedings rgaic program- Pragmatisch Apress. . etr Canadian Western ae 1083–1090. pages , CC,pages WCCCE, , asrFach- Hanser . xrm Pro- Extreme Communica- EESoft- IEEE 28:104– , Apress, . 245 , .

X—BIBLIOGRAPHY . Pro- Pro- Inter- , Ubi- http: th , IWDSC, 11 , pages 194– Proceedings http://en. , pages 59–65, Los , chapter 17, pages , Washington, DC, . Dynamic Psychology ¨ ottingen. Proceedings of the 13th Conference on Software VLDB Conference International Conference on Soft- . th , ICSE, page 273, Los Alamitos, th Case Study Research: Design and 24 16 , 27:50–55. . Applied Social Research Methods Series. Sage Publications, third edition. USA. IEEE Computer Society. neering education. In Columbia University Press. H., and Nakajima, T.technologies to (2009). create economic incentives Applyingconsumer that pervasive alter behavior. In national Conference on Ubiquitous Computing Comp, pages 175–184, New York, NY, USA. ACM. pages 1–2. ceedings of the ware Engineering CA, USA. IEEE Computer Society Press. tative analysis and performance study forsearch similarity- methods in high-dimensional spaces.ceedings In of the 205. ACM Press. torings from source-code changes.of the In 21st IEEE/ACM International Conference on IEEE Software Lehrbuch der Personalpsychologie wikipedia.org Engineering Education & Training 481–507. Hogrefe Verlag, G Alamitos, CA, USA. IEEE Computer Society. Methods //www.geekandpoke.com/ Automated Software Engineering Workshop on Detection of Software Clones ”pair-pressure” and ”pair-learning” on software engi- Yin, R. K. (2003). Widder, O. (2012). Geek and poke. Online. Wikipedia.org (2012).Williams, Online. L. and Kessler, B. (2000). The effects of Woodworth, R. S. (1918). Wray, S. (2009). How pair programming really works. Yamabe, T., Lehdonvirta, V., Ito, H., Soma, H., Kimura, Wasmund, M. (1994). Reuse facts and myths. In Weber, R., Schek, H.-J., and Blott, S. (1998). A quanti- Wegge, J. (2001). Gruppenarbeit. In Schuler, H., editor, Weißgerber, P. and Diehl, S. (2006). Identifying refac- , IBLIOGRAPHY . Delft IEEE X—B . Wiley. , 2:201–207. International International rd nd 3 2 Journal of the ACM Lecture Notes in Com- . Faculteit Industrieel Ontwer- Work and Motivation Special publication of the Jacquard Proceedings of the Proceedings of the , 2418:153–165. , volume 158, pages 487–495. Springer. , pages 6–7. , 26(6):92–94. Empirical Software Engineering , RCIS, pages 313–324. IEEE Computer Soci- to-string correction problem. 21(1):168–173. evaluation can be improved: Ideasretrieval. from information In engineering. In ety. mated codehttp://www.javaworld.com/javaworld/ reviews withjw-11-2008/jw-11-checkstyle2.html checkstyle, part 2. and Kitchenham, B.industrially-based multiple A. case studies (2009). in software Guidelines for pen. tions of Computation Theory, Lecture Notesputer in Science Com- nance. ture clash or common cause? puter Science ﬂuss klassischer und agiler techniken auf den erfolg studie. Technical report, oose Innovative Informatik GmbH. iment: Impact of documentation on mainte- ing more mature. programme Science Software von it-projekten — 1. ergebniszusammenfassung der Proceeding of the International Conference Founda- Conference on Research Challenges in Information Design Guide - Part 2 Aligning perspectives and other best practices. 10.1023/A:1009778023863. 246 Wagner, R. A. and Fischer, M. J. (1974). The string- Walenstein, A. and Lakhotia, A. (2003). Clone detector Vroom, V. H. (1964). Verner, J. M., Sampson, J., Tosic, V., Bakar, N. A. A., van Boeijen, A. and Daalhuizen, J., editors (2010). Vashishtha, S. and Gupta, A. (2008). Auto- Ukkonen, E. (1983). On approximate string matching. In Umarji, M. and Shull, F. (2009). Measuring developers: Turner, R. and Jain, A. (2002). Agile meets cmmi: Cul- Tryggeseth, E. (1997). Report from an exper- Tolido, R. (2008-2009). Software engineering is becom- Toth, S., Vigenshow, U., and Wittwer, M. (2009). Ein-

X—BIBLIOGRAPHY o,L n ofe,M .(05.Uigoii anal- origin Using (2005). W. M. Godfrey, and L. Zou, distributed globally for factors Success (2009). S. Zopf, Zeller, and S., Diehl, P., Weibgerber, T., Zimmermann, (2003). J. R. Gerrig, and G. P. Zimbardo, quality and code Improving (2011). A. M. Ziesemer, 14:355–359. .(04.Mnn eso itre ogiesoft- guide to In histories changes. version ware Mining (2004). A. online. checkstyle. with sst eetmrigadsltigo orecode source of splitting and merging detect to ysis Engineering Software entities. projects. Society. Computer edition. siebte Springer, EETas ot.Eng. Softw. Trans. IEEE otaePoesIpoeetadPractice and Improvement Process Software 26 th CE ae 6–7.IEEE 563–572. pages ICSE, , nentoa ofrneon Conference International 31(2):166–181. , Psychologie - uuework Future — X-2 , . 247

X—BIBLIOGRAPHY ywhe l 20) 1 232 71, (2008), al. et Ayewah 89 dimension, reward AvP, 130 128, class, AuthorManager, 130 class, Author, Author Artifact 232 75, 73, (2006), al. et Anvik 232 73, (2007), Murphy and 232 Anvik 42, (2010), Cheshire and 232 Antin 117, (2008), Indyk and Andoni 232 71, (2009), Visser and Alves 232 58, (2010), 232 Al-Qutaish 89, (2007), al. et Ahlgren 168– 166, 165, 162, 154–156, 141, 104, 100, AgileLab, 232 190, 154, 134, 131, 128, 123, 104, (2012), 232 Agaraj 91, (2008), al. et 232 Adler 47, (2000), Huberman and 232 Adar 56, (2010), al. et Abrahamsson 248 232 91, (2009), al. et Bacon 232 56, (2009), Babar 69 audit, assets 54 article, archeology 110 arbiter, appreciation 58 analyzability, 146 evaluation, analytical 56 manifesto, agile agile 82 agent, Index ls,126 class, 88 motivation, 126 class, 79 75, software, 88 motivation, 56 development, 7,19 8,17 8,10 9,17 201, 197, 196, 190, 188, 187, 183, 179, 171, 0,26 1–1,221–223 210–215, 206, 205, rw ta.(00,2,11 8,21 233 221, 189, 161, 23, (2010), al. et Brown 233 62, 59, 233 (2003), 87, Brooks 85, 14, 13, 233 (2005), 89, Brockhaus 83, (1998), Page and 233 Brin 71, (2000), al. et Briand 233 85, 48, Br 47, (2010), Melvin and 233 Boyes 75, (1999), Holt and Bowman 233 147, (2005), Bortz 99 40–42, 233 Bora, 69–71, 233 (2006), 66, Moonen 65, and 57, Boogerd 56, 45, 29, (2002), Boehm 233 99, 56, (1986), 233 Boehm 58, (1976), al. et Boehm 233 14, 233 (2005), 28, Bodendorf 26, (2009), al. et 233 Boden 32, (2007), al. et Boden 233 87, (1981), Boal 14 blueprint, 36 stone, of out blood 114 73, svn, blame, 233 148 55, design, 53, between-group 28, (2010), al. et 232 Bertram 13, (1979), al. et 232 Bersoff 69, (2010), al. et 232 Bernhart 40, (2008), al. 232 et 159, Beecham 78, (2007), Jarzabek and Basit 232 125, (1964), Barankin 232 77, (2004), al. et Bar-Yossef 60 28, mud, 232 of 53, ball 25, 16, 13, (1997), Bailin 116, 80, 79, 77, (1999), Ribeiro-Neto and Baeza-Yates ueadWie 21b,6,234 63, (2010b), Weimer and 233 Buse 194, 37, (2010a), Weimer and Buse 233 44, (2010), Buschmann 49 factor, bus 233 26, 25, (2010), 233 Bullinger 82, (1988), Johansen 65, and 60, Bullen 55, 53, 22, 17, 15, (2000), Dutoit and Bruegge h 19) 6 233 56, (1995), ohl ¨ 232 8 233 68,

X—INDEX oesel 63 smell, code 48 ownership, code 29 hero, code 14 code, 234 86, (1960), Coase 234 65, 39, (2010), CMMI 65 CMMI, 65 CMM, 77 detection, clone 234 53, (2004), Clark 234 46, (2000), Clark 71 234 ckjm, 69, (2003), al. et 234 Ciolkowski 71, (1994), Kemerer and Chidamber 166, 163, 156, 137, 127, 110, 91, 72, 71, 40, 234 Checkstyle, 78, (2008), Mockus 234 and 14, Chang 13, (2007), Fuggetta and 234 Cerri 26, (2009), 234 Nambiar 121, and 28, Cataldo 26, (2009), Richardson and Casey 234 82, 54 46, CASE, 26, (2001), Agarwal and Carmel 234 64 77, capability, 76, (2007), al. et Canfora 191 C30, owy(98,18 5,234 159, 108, (1968), Conway 159 Law, Conway’s 147 group, control 149 contributor, 68 integration, continuous consistency 234 29, (1999), Connell 55 management, conﬁguration conﬁdence 146 evaluation, concept 14 compiler, 122 community, 69 commit-then-review, 49 ownership, code collective 234 86, (2009), Collard 95 prototype, CollabReview 95 concept, CollabReview 107– 103–105, 101, 95–99, 79, 12, iii, CollabReview, nweg ae 15 base, knowledge 88 motivation, 6,13 6,18 7–7,1811 185– 178–181, 171–174, 168, 167, 163, 161, 160, 153–157, 149–151, 138, 145–147, 137, 140–142, 132–134, 122–130, 120, 116, 113, 214, 212, 211, 204–207, 202, 201, 198, 197, 9,21 0–0,2027 220–223 210–217, 203–208, 201, 197, 215 - uuework Future — X-2 eceae l 20) 2,235 123, (2009), al. et Dencheva 235 223, 235 221, 39, 88, (2000), 85, Deming 84, 79, 64, (1994), Deming 235 28, (1992), 235 Dekleva 38, (1979), Pozefsky and 235 Deimel 18, (2007), al. et Decker 14 234 data, 55, 42, (2004), Voss and Danowski 71 Complexity, Cyclomatic 234 14, (2004), 14924-1 Std. CWA 89 dimension, 234 reward 234 66, CvA, 82, 65, 61, (1995), 29, al. 26, et 21, Curtis 16, (1988), al. et Curtis learning and curiosity 122 curiosity, 234 23, (1992), Cunningham culture 82 CSCW, 149 cross-validation, 234 60, (1980), Crosby 150 84, 29, coding, cowboy 119 covering, 77 corruption, correlation 80 rejection, 234 correct 78, (2004), al. et Cordy 234 71, (2005), Copeland 85 coopetition, 234 82, (2007), Cook G 19) 2 235 62, (1995), DGQ 235 148, (2002), 235 Vaus 20, de (2006), al. et 234 Souza 79, de (2008), al. et Alwis de 56 methodology, development 109 specialization, developer 15 developer, 106 235 develop, 86, (2011), al. et Deterding 100 99, 56, 41, Dendrodoc, 235 131, 128, 107, (2010), 173, Dencheva 153, 123, 112, 55, 43, 42, (2011), al. et Dencheva oiain 88 motivation, 84 organizational, 149 k-fold, 148 strength, 148 Spearman, 148 80, Pearson, 80 Matthews, 7,17 235 177, 175, 249

X—INDEX 236 software, 75 club, 48 common, 48 private, 48 public, 48 ¨ ohlich (2002), 84, 236 ırba et al. (2005), 75, 108, 236 ˆ Farmer and Glass (2010), 83, 109, 112, 113, 122, 123, Fenton and Pfleeger (1997), 70,Festinger 236 (1957), 184, 236 File-sharing drives, 54 FindBugs, 71 Flail, 41, 42 Fluri et al. (2007), 78,Flyvbjerg 236 (2006), 32, 236 Foote and Yoder (1997), 28, 60,forensics 236 fork, 16 Forward and Lethbridge (2002), 21,Forward 26, (2002), 28, 62, 29, 82, 62, 236 236 Fowler (1999), 21, 63, 236 Fowler (2006), 48, 49, 236 Fowler (2009), 23, 24, 236 FP6 (2011), 33, 236 Fr Franke and Kaul (1978), 89,free 90, riding, 236 47, 88, 220 FreeCol, 117, 151, 152, 157–160,Frey 162, and 164–166, Jegen 208 (2000), 87,Friendly 236 (1995), 55, 236 functionality, 58 G Galin (2004), 60, 61, 236 game theory, 85 gamification, 86, 220 Garvin (1984), 58, 236 Geek and Poke, 29 Geiger (1988), 58, 236 GG (1949), 47, 236 gold standard, 147 Golden Triangle, 59 Goldman and Gabriel (2005), 57,good 152, 236 Gordon (1954), 50, 237 Graham (2004), 20, 27, 50,Gray 51, et 53, al. 237 (1997), 75,Grechanik 237 (2009), 39, 237 Grier (2010), 19, 237 Fagan (1976), 65, 69, 236 Farlex (2012), 82, 107, 236 NDEX X—I information-exchange, 47 curse of, 116 string edit, 110 as communication, 82 external, 20 inherent, 19 internal, 20 string, 77 tree, 77 VIE model, 87 ideal, 147 250 dimensionality Dingsøyr et al. (2006), 56,DiNucci 235 (1999), 82, 235 DiPalantino and Vojnovic (2009), 87, 91,distance 235 documentation, 17 Doxygen, 18, 55, 56 Dromey (1995), 58, 235 Drommert (2011), 22, 235 Dubochet (2009), 19, 235 Ducasse et al. (1999), 119,DvC, 235 reward dimension, 89 edit distance, 77 Edwards (2003), 71, 91, 236 efficiency, 58 Eisenhardt (1989), 84, 236 Eiser (1986), 167, 236 El Classico, 107 Elster (1989), 111, 236 empirical evaluation, 146 employee, 150 equilibrium, 83 Erdogmus (2010), 31, 236 evaluation, 145 excludability, 47 expectancy experiment, 147 experimental group, 147 external documentation, 20 external quality, 58 external validity, 147 externality, 85 extreme programming, 56 Diamond (2005), 47, 235 Didrich and Klein (1996), 19,Dig 55, et 235 al. (2007), 75,dilemma 235

X—INDEX oeee n uh(04,6,238 63, (2004), Pugh and Hovemeyer 123 HotW, 238 27, (2009), Horeis 238 107, (2004), Hoppe 238 91, 47, 238 (2007), 91, Hoisl 42, (2007), al. et Hoisl 238 89, 83, (2005), 238 Hirsch 76, (2008), al. et 238 Hindle 56, (2001), Cockburn 178 and 153, Highsmith 123, week, the of 238 Hero 14, (1990), Kuhlen and 238 Herget 26, (2001), al. 238 et 29, Herbsleb 26, (2001), Moitra and 238 Herbsleb 81, (1999b), Grinter and 238 Herbsleb 26, (1999a), Grinter 237 and 122, Herbsleb 88, (1995), Hippel and 237 Heneman 87, (1987), Gollwitzer and Heckhausen 89 effect, 237 Hawthorne 186, (1998), 237 Katz 75, and 73, Hauser 48, (2010), al. 237 et 108, Hattori 75, (2009), Lanza and 237 Hattori 75, (2011), al. 237 et 64, Hata 59, (2005), Schroeder 237 and 185, Harry 147, (2004), List and Harrison 237 14, (2010), Harman 13 hardware, 237 47, (1968), Hardin 79 artifact, 237 halo, 71, (2010), al. et Haderer 89 h-index, 89 dimension, reward 237 GvL, 19, (2000), Sommer 237 and 108, Gumm 75, 49, 48, (2011), Grossbart 237 71, (2009), Großmann 237 25, (1993), Griss 237 237 37, 82, (1996), 81, Grinter 26, (1999), al. 237 et 148, Grinter 147, (2011), al. et Grinnell EESd 2 19) 5 8 2 238 82, 68, 65, (1998), 829 238 Std. 54, IEEE 53, (1990), 238 610.12 82, Std. 18, IEEE 17, (2006), 1074 Std. 238 IEEE 69, (2008), 1028 Std. IEEE 238 27, (2012), Programme Work ICT 238 238 20, 223, (2003), 221, Thomas 122, and 111, Hunt 88, 29, 16, (1997), Humphrey 238 65, 46, 238 (1988), 61, Humphrey 238 60, 194, (2006), 37, Boehm 29, and 25, Huang 20, 19, (2011), Starke and Hruschka - uuework Future — X-2 ono 20) 5 239 85, (2007), 239 Johnson 69, (1997), Tjahjono and Johnson 239 Jim 71, (2009), al. et Jay 239 45, (2005), 239 Jalote 90, (2010), al. et 239 Jahn 83, (2007), al. et Jøsang aa-lxne 21) 5 239 15, (2012), Kanat-Alexander 239 29, 28, (2001), Kajko-Mattsson 89 dimension, reward IvG, 89 dimension, reward IvC, 96 design, Iterative 55 tracker, issue 239 239 229, 71, 155, (2001), 71, 9126-3 65–68, ISO 59, 58, 239 (2001), 92, 9126-1 62–65, ISO 57, 18, (2005), 9000 239 ISO 59, 58, (2005), 25000 239 ISO 65, 64, (2004), 15504-1 ISO 239 23, (2006), 14764 ISO 238 35, (1999), 13407 ISO Triangle Iron 85 hand, invisible 80 index, inverted 86 internalized, 147 validity, internal 22 quality, internal 20 documentation, internal 24 interest, instrumentality 69 238 inspection, 111, (1985), al. et Insko 19 documentation, inherent 14 information, 103 89, space, incentives 130 class, ImportWorker, 238 59, (2009), Ika imreadCes(07,2,4,239 47, 29, (2007), Cress and Kimmerle 239 28, 26, (2003), Kiel 239 83, (2003), Schiff and Kennes 83 karma, 157–160, 151–153, 119, 117, 103, 44, 31–40, Karakos, nze l 20) 1 239 81, (2009), al. et enez ´ ult n rdciiy 5 63 45, productivity, and Quality 59 management, project 87 model, VIE outsoe 112 score, robust 222 112, score, open-ended 222 112, score, closed-range 6,15 6,13 208 193, 166, 165, 162, 251

X—INDEX 210–215, 221 133, 152, 161, 174, 212 154, 168–171, 173–179, 188, 190–194, 196, ¨ om and Johannesson (2008), 87, 241 agile, 56 plan-driven, 56 knowledge base, 15 crowding, 87 extrinsic, 87 intrinsic, 87 quality, 33 software quality, 33, 38 technical, 33 agile, 56 MediaWiki, 54, 55, 73, 76, 77, 79, 117, 128, 130,Mellstr 131, Merel (1999), 89, 241 meritocracy, 89 metadata, 78 methodology micro edit, 193 MicroEditCastigator, 104, 193 minimality miss, 80 Moknowpedia, 42, 43, 104, 133, 135–137, 140, 152– Mockus and Herbsleb (2002), 73,Mockus 81, et 241 al. (2002), 48,mongolian 241 hoards, 53 motivation, 86, 87 Nalebuff and Brandenburger (1996), 85, 241 Naumann and Jenkins (1982), 15,Naur 56, and 241 Randell (1969), 53,Navarro 241 (2001), 77, 241 Majchrzak et al. (2006), 18,Malik 241 and Choudhary (2008), 60,manager 241 manifesto Mantei and Teorey (1988), 96, 241 Manuel et al. (2011), 147,markup, 241 79 Martin and McClure (1983), 24,Marwedel 28, (2011), 241 145, 241 maturity, 64 McAdam and McCreedy (2000), 42,McCabe 241 (1976), 71, 241 McCall et al. (1977), 58,McLachlan 241 et al. (2004), 149,measurement, 241 70 MeasurementsManager, class, 131 mechanism design, 85 NDEX X—I 252 King et al. (1997), 63,Kitchenham 137, and 155, Pﬂeeger 239 (1996), 58,Kittur 70, and 239 Kraut (2010), 42,Kneuper 239 (2007), 65, 239 knowledge, 14 knowledge base, 15 Knuth (1992), 19, 55, 239 Koch (2005a), 45, 239 Koch (2005b), 45, 239 Koch (2006), 45, 240 Kogut and Metiu (2001), 27,Kontogiannis 240 (1997), 78, 240 Kozlov et al. (2007), 23, 240 Kraut and Streeter (1995), 28,Kraut 81, et 240 al. (2010), 42,Kreps 240 and Wilson (1982), 83,Krogstie 240 and Sølvberg (1994), 28,Krumbach-Mollenhauer 240 and Lehment (2007), 88, 240 Lampe and Johnston (2005), 91,Landin 240 (1964), 19, 240 Laramee (2010), 28, 240 LARI, 152, 156 learning, 122 Lehner (1994), 62, 240 Leonard-Barton (1987), 29, 38, 46,Lethbridge 240 et al. (2003), 29,Leuf 62, and 240 Cunningham (2001), 55,Levenshtein 240 distance, 77 Levenshtein (1966), 77, 240 Lewis (2005), 15, 25, 67,Lientz 68, and 240 Swanson (1981), 28,Lincke 240 (2009), 57, 240 line, 109 lines of code, 70 Lings et al. (2006), 35,literate 240 programming, 55 Liukkunen et al. (2010), 26,Lloyd 240 et al. (2008), 24,LOC, 240 70 Log4j, 162, 165, 166 looking over the shoulder, 69 Louridas (2006), 18, 241 Loveland (2009), 63, 71, 91,Lowrance 241 and Wagner (1975), 77, 241 Lynex and Layzell (1998), 50, 241 magic number, 156 maintainability, 58 maintenance, 23

X—INDEX arpesr,69 pressure, pair 89 PageRank, 65 P-CMM, 242 146, 83, 17, (1994), Dictionary Oxford 85 obligates, ownership 242 ownership 75, (2010), al. et Ostrand 84 organization, 242 29, (2007), Oram 56 development, 242 source 75, open 74, (2008), Maruyama and Omori 110 105, artifact, obsolete, 242 84, 75, 63, 49, 48, 28, (2003), Nordberg 242 96, (1993), Nielsen 242 86, (2001), Nerdinger 241 15, (2011), 242 Nehaniv 15, (2006), al. et Nehaniv 116 problem, neighbor 241 nearest 77, (2005), al. et Neamtiu otblt,58 portability, 71 PMD, 242 88, 82, (2008), PMBOK 242 148, 147, (2011), Plante 156 poker, planning 56 development, plan-driven Pl Pl 86 tax, 242 Pigovian 28, (2004), April and Pigoski 242 69, (2007), 242 Phongpaibul 70, (1994), al. et Perry penalty 148 80, correlation, Pearson pattern 242 20, (2010), Parnin 242 213–215 29, 166, 21, 165, (2010), 160, Papadimoulis 104, 69, 66, programming, pair she l 20) 2 242 22, (2007), al. 242 et 72, osch 71, (2008), al. et osch ¨ ¨ oe ek 49 weak, code, 48 strong, code, 49 no, code, 49 collective, code, 48 code, eaie 86 negative, 122 individual, 132 visitor, 60 code, throwaway 132 131, publish-subscribe, 60 28, mud, of ball - uuework Future — X-2 uc it,43 dirty, & quick quantity 63 planning, quality 63 management, quality 58 use, in quality 63 control, quality 64 assurance, quality 57 quality, 89 dimension, reward PvR, 89 dimension, reward PvD, 89 dimension, reward PvC, 15 prototype, 53 programming, 16 programmer, process 84 problem, principal-agent 243 62, (2001), Pressman 243 80 60, precision, 24, 17, 16, 243 (2011), 157, Prause 118–120, 115, 113, 242 (2009), 99, Prause 57, 243 55, 179, 17, 167, (2006), 155, Prause 154, (2012), al. et 243 Prause 243 43, 122, (2011), 121, al. 89, et 88, Prause 19, (2010b), al. 243 et 40, Prause 39, (2010a), al. et 243 Prause 54, (2008), al. et 243 Prause 79, 242 (2007b), 100, al. 79, et 56, Prause 41, (2007a), al. et 243 Prause 31, (2012), Zimmermann and 243 Prause 62, (2011), 243 Reiners 167, and 153, Prause 152, (2012), Eisenhauer 243 and 101, Prause 85, (2008), Eisenhauer and 173, Prause 172, 121, 66, 56, 31, 29, 243 (2012), 140, Durdik 106, and 103, Prause 101, (2008), Apelt and Prause 73 svn, praise, 242 80, (2007), Powers shape to power 242 84, (2006), Powell 242 89, (1995), Poulin am cr,112 score, karma 60 spot, sweet 112 score, karma 22 internal, 58 views, elementary 59 cost, 58 characteristics, 84 business, 88 motivation, 243 253

X—INDEX 221 motivation, 88 correlation, 148 evolutionary, 77 motivation, 88 master, 197 ¨ ¨ oter et al. (2006a), 75,oter 244 et al. (2006b), 79, 163, 165, 166, 244 ScrumLab, 100, 137, 156, 196, 197, 201, 207, 211–216, Seaman and Basili (1997), 111,Sechser 245 (2009), 46, 245 Seddon (1997), 64, 245 Seibel (2009), 28, 53, 63,self-explanatory, 155, 19 245 self-fulfillment self-organization, 84, 93 self-realization, 122 Selic (2009), 21, 22, 29,shelfware, 245 62 Shull et al. (2002), 32,significance 245 Silicate, 43 similarity Singer and Schneider (2012a), 28,Singer 40, and 46, Schneider 91, (2012b), 245 91,six 193, sigma, 245 64, 82 slow death, 28 Smart (2008), 71, 245 smell, 63 Smith (1776), 12, 51, 85,Smith 245 (2005), 91, 245 social facilitation, 88 social loafing, 87 social relationships software, 13 software crisis, 28, 220 software engineering, 53 software process, 17 software process artifact, 18 Sonar, 37, 71, 72, 189 source code, 14, 19 Spearman correlation, 148 specialization, 49 SPICE, 65 spillover, 85 Spinellis (2003), 73, 245 Spinellis (2006), 58, 245 Spinellis (2010), 19, 21, 29, 36, 245 Schr Schr SCM, 55 scrum, 56 NDEX X—I ¨ ost (2009), 32, 244 hidden, 38 open, 38 class, 126 ¨ oter et al. (2006), 161, 162, 166, 244 ¨ onnau and Borghoff (2012), 77, 244 254 Ryan and Deci (2000), 87, 244 Sager et al. (2006), 77,Salo 244 and Abrahamsson (2008), 56,Salvaneschi 244 (2009), 55, 244 Schr Rahman and Devanbu (2010), 75, 243 Rahman and Devanbu (2011), 83, 243 Rakitin (2001), 56, 243 ranking list, 177 Raskin (2005), 19–21, 25, 28,rat 55, race, 99, 98, 186, 101, 243 172 recall, 80 refactoring, 63 Reichling and Veith (2005), 42, 244 Reiners et al. (2012), 223,rejection, 244 103, 127 reliability, 58 Repository, class, 130, 131 reputation, 83 reputation statement, 83 reputation system, 83 ReputationForge, 104, 190 ReputationMetricManager, class, 130, 131 requirements process, 18 responsibility, 107 responsible, 107 Review review, 109 review-then-commit, 69 revision control, 55 revision repository, 105 reward dimensions, 89 Richardson and Gwaltney (2005), 18,Riemenschneider 37, et 49, al. 160, (2002), 244 46,Rigby 244 et al. (2008), 69,rivalrousness, 244 47 Robbes and Lanza (2005), 75,Robbes 244 (2007), 75, 244 Robertson and Robertson (1999), 18,Rogers 82, (2003), 244 46, 244 Rohs (2007), 42, 244 Rosenberg (1997), 71, 244 Rosson and Carroll (2002), 146,Royce 244 (1987), 56, 244 Runeson and H R

X—INDEX hmsadHn 20) 5 245 55, (2004), Hunt and Thomas 28 Bob’s, Redevelopment, of Theory 245 223, 69, (2001), Tanenbaum 245 56, (1986), Nonaka and Takeuchi etn,67 testing, 68 coverage, test 58 23, debt, technical 26 teamness, 105 team, 19 sugar, syntactic 60 44, spot, sweet 89 dimension, reward SvT, 88 effect, sucker 245 32, (1995), Suchman 106 substring, 150 subject, 71 StyleCop, 146 study, 150 student, 48 ownership, code strong matching string 106 string, 89 245 society, 26, stone 21, (2003), Rosenberg and Stephens status 68 analysis, static 149 stakeholder, 245 84, (2012), Stacey 245 84, (1996), Stacey 84 diagram, 245 Stacey 223, 26, 22–24, (2004), Spolsky 56 model, spiral 245 155, 63, (2011), Spinellis uvrin 7 8 9 4 5 3 4 6 7 9 117, 79, 77, 76, 74, 73, 55, 54, 49, 38, 37, Subversion, adtcs 68 yardsticks, 68 unit, 68 system, 68 static, 67 manual, 68 integration, 68 dynamic, 67 box, black/white 67 automatic, 73 praise, 73 blame, 77 approximate, 88 motivation, 6,10 9,28 1,214 212, 208, 163, 196, 156–161, 170, 154, 168, 153, 151, 131, 130, 126, - uuework Future — X-2 amn 19) 1 246 61, (1994), Wasmund 246 78, (2003), Lakhotia 246 and Walenstein 115, 77, (1974), Fischer and Wagner ekcd wesi,49 ownership, code weak 65 56, model, waterfall 65 56, waste, 210–212 190–193, 154, 153, 104, Vyrygh, 89 dimension, reward 246 VvR, 216, 186, 87, (1964), Vroom 87 model, VIE 246 32, (2009), al. et Verner 246 71, (2008), Gupta and Vashishtha 65 XT, V-model oio(09,2,3,245 35, 27, (2009), Tolido 245 20, (1993), 245 Tilley 76, (2008), Holcombe and Thomson v,rwr ieso,89 dimension, reward 246 TvO, 66, 45, (2002), 246 Jain 62, and 56, Turner 25, 24, 16, (1997), Tryggeseth 64 TQM, 246 56, (2009), al. et Toth 168– 166, 165, 162, 152, 141, 122, 104, 100, TornBond, oe 110 vote, 18 viewer, 18 view, 77 model, vector 246 146, variable (2010), Daalhuizen and Boeijen van validity 145 validation, valence 56 v-model, ieies 1,127 110, timeliness, sr 149 user, 58 usability, 58 246 understandable, 70, (2009), Shull and Umarji 246 77, (1983), Ukkonen 49 number, truck 220 47, commons, of tragedy needn,147 independent, 147 dependent, 147 internal, 147 external, 87 model, VIE 47 digital, 7,17 8,21 1,24 221 214, 213, 211, 189, 187, 171, 255

X—INDEX NDEX X—I motivation, 88 256 zero-sum game, 85 Ziesemer (2011), 71, 246 Zimbardo and Gerrig (2003), 85,Zimmermann 247 et al. (2004), 73,Zopf 75, (2009), 79, 26, 247 28, 247 Zou and Godfrey (2005), 76, 78, 247 XP, 56 wiki, 54 within-group design, 148 work packages, 33 workers’ pride Wray (2009), 29, 68, 69, 246 Web 2.0, 82 Weber et al. (1998), 116, 246 Wegge (2001), 87, 167, 246 Weißgerber and Diehl (2006), 78, 246 Widder (2012), 30, 101, 246 Wikipedia.org (2012), 54, 246 WikiTrust, 91 Williams and Kessler (2000), 69, 246 Woodworth (1918), 87, 246 Yamabe et al. (2009), 90, 246 Yin (2003), 32, 246

X—INDEX