Early praise for Your Code as a Crime Scene This book casts a surprising light on an unexpected place—my own code. I feel like I’ve found a secret treasure chest of completely unexpected methods. Useful for programmers, the book provides a powerful tool to smart testers, too. ➤ James Bach Author, Lessons Learned in Software Testing You think you know your code. After all, you and your fellow programmers have been sweating over it for years now. Adam Tornhill uses thinking in psychology together with hands-on tools to show you the bad parts. This book is a red pill. Are you ready? ➤ Björn Granvik Competence manager Adam Tornhill presents code as it exists in the real world—tightly coupled, un- wieldy, and full of danger zones even when past developers had the best of inten- tions. His forensic techniques for analyzing and improving both the technical and the social aspects of a code base are a godsend for developers working with legacy systems. I found this book extremely useful to my own work and highly recommend it! ➤ Nell Shamrell-Harrington Lead developer, PhishMe By enlisting simple heuristics and data from everyday tools, Adam shows you how to fight bad code and its cohorts—all interleaved with intriguing anecdotes from forensic psychology. Highly recommended! ➤ Jonas Lundberg Senior programmer and team leader, Netset AB After reading this book, you will never look at code in the same way again! ➤ Patrik Falk Agile developer and coach Do you have a large pile of code, and mysterious and unpleasant bugs seem to appear out of nothing? This book lets you profile and find out the weak areas in your code and how to fight them. This is the essence of combining business with pleasure! ➤ Jimmy Rosenskog Senior consultant, software developer Adam manages to marry source code analysis with forensic psychology in a unique and structured way. The book presents tools and techniques to apply this concept to your own projects to predict the future of your codebase. Brilliant! ➤ Mattias Larsson Senior software consultant, Webstep Your Code as a Crime Scene Use Forensic Techniques to Arrest Defects, Bottlenecks, and Bad Design in Your Programs Adam Tornhill The Pragmatic Bookshelf Dallas, Texas • Raleigh, North Carolina Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and The Pragmatic Programmers, LLC was aware of a trademark claim, the designations have been printed in initial capital letters or in all capitals. The Pragmatic Starter Kit, The Pragmatic Programmer, Pragmatic Programming, Pragmatic Bookshelf, PragProg and the linking g device are trade- marks of The Pragmatic Programmers, LLC. Every precaution was taken in the preparation of this book. However, the publisher assumes no responsibility for errors or omissions, or for damages that may result from the use of information (including program listings) contained herein. Our Pragmatic courses, workshops, and other products can help you and your team create better software and have more fun. For more information, as well as the latest Pragmatic titles, please visit us at https://pragprog.com. The team that produced this book includes: Fahmida Y. Rashid (editor) Potomac Indexing, LLC (indexer) Cathleen Small (copyeditor) Dave Thomas (typesetter) Janet Furlow (producer) Ellie Callahan (support) For international rights, please contact [email protected]. Copyright © 2015 The Pragmatic Programmers, LLC. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form, or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior consent of the publisher. Printed in the United States of America. ISBN-13: 978-1-68050-038-7 Encoded using the finest acid-free high-entropy binary digits. Book version: P1.0—March 2015 Contents Foreword by Michael Feathers . ix Acknowledgments . xi 1. Welcome! ..............1 About This Book 1 Optimize for Understanding 2 How to Read This Book 4 Toward a New Approach 6 Get Your Investigative Tools 7 Part I — Evolving Software 2. Code as a Crime Scene . 13 Meet the Problems of Scale 13 Get a Crash Course in Offender Profiling 15 Profiling the Ripper 16 Apply Geographical Offender Profiling to Code 17 Learn from the Spatial Movement of Programmers 18 Find Your Own Hotspots 22 3. Creating an Offender Profile . 23 Mining Evolutionary Data 23 Automated Mining with Code Maat 26 Add the Complexity Dimension 28 Merge Complexity and Effort 30 Limitations of the Hotspot Criteria 30 Use Hotspots as a Guide 31 Dig Deeper 33 Contents • vi 4. Analyze Hotspots in Large-Scale Systems . 35 Analyze a Large Codebase 35 Visualize Hotspots 38 Explore the Visualization 40 Study the Distribution of Hotspots 41 Differentiate Between True Problems and False Positives 45 5. Judge Hotspots with the Power of Names . 47 Know the Cognitive Advantages of Good Names 47 Investigate a Hotspot by Its Name 50 Understand the Limitations of Heuristics 52 6. Calculate Complexity Trends from Your Code’s Shape . 55 Complexity by the Visual Shape of Programs 55 Learn About the Negative Space in Code 57 Analyze Complexity Trends in Hotspots 59 Evaluate the Growth Patterns 63 From Individual Hotspots to Architectures 64 Part II — Dissect Your Architecture 7. Treat Your Code As a Cooperative Witness . 67 Know How Your Brain Deceives You 68 Learn the Modus Operandi of a Code Change 71 Use Temporal Coupling to Reduce Bias 72 Prepare to Analyze Temporal Coupling 76 8. Detect Architectural Decay . 77 Support Your Redesigns with Data 77 Analyze Temporal Coupling 78 Catch Architectural Decay 83 React to Structural Trends 87 Scale to System Architectures 89 9. Build a Safety Net for Your Architecture . 91 Know What’s in an Architecture 91 Analyze the Evolution on a System Level 93 Differentiate Between the Level of Tests 94 Create a Safety Net for Your Automated Tests 99 Know the Costs of Automation Gone Wrong 103 Contents • vii 10. Use Beauty as a Guiding Principle . 105 Learn Why Attractiveness Matters 105 Write Beautiful Code 107 Avoid Surprises in Your Architecture 108 Analyze Layered Architectures 111 Find Surprising Change Patterns 113 Expand Your Analyses 116 Part III — Master the Social Aspects of Code 11. Norms, Groups, and False Serial Killers . 121 Learn Why the Right People Don’t Speak Up 122 Understand Pluralistic Ignorance 124 Witness Groupthink in Action 127 Discover Your Team’s Modus Operandi 128 Mine Organizational Metrics from Code 132 12. Discover Organizational Metrics in Your Codebase . 133 Let’s Work in the Communication Business 133 Find the Social Problems of Scale 135 Measure Temporal Coupling over Organizational Boundaries 138 Evaluate Communication Costs 141 Take It Step by Step 145 13. Build a Knowledge Map of Your System . 147 Know Your Knowledge Distribution 147 Grow Your Mental Maps 152 Investigate Knowledge in the Scala Repository 155 Visualize Knowledge Loss 158 Get More Details with Code Churn 161 14. Dive Deeper with Code Churn . 163 Cure the Disease, Not the Symptoms 163 Discover Your Process Loss from Code 164 Investigate the Disposal Sites of Killers and Code 168 Predict Defects 171 Time to Move On 174 15. Toward the Future . 175 Let Your Questions Guide Your Analysis 175 Take Other Approaches 177 Contents • viii Let’s Look into the Future 181 Write to Evolve 182 A1. Refactoring Hotspots . 183 Refactor Guided by Names 183 Bibliography . 187 Index . 191 Foreword by Michael Feathers It’s easy to look at code and think that there is nothing more than what we see. When we look at it, we see operators, identifiers, and other language structure, but that is all surface. We forget the depth. We spend so much time looking at the current state of the code that we forget its history and all of the forces that influenced it along its path toward the present. We pay for this myopia. Many code changes are incredibly shortsighted, both in terms of our vision of what the code will be in the future, and the way that it got to be the way that it is. Years ago, I was struck by the fact that we use version-control systems to keep track of our projects’ histories, but we hardly ever revert to previous versions. Those versions exist as an insurance policy, and we’re lucky when we never have to file a claim. It’s easy, then, to look at that record of changes and see it as waste. Do we really need more than the last few versions? The naive answer is no, but we have a real opportunity when we enthusiastically say yes—we can mine our source code history to learn more about us, our environment, and how we work. That information is real power. I think that we are at the beginning of a new era of awareness about how software changes. We’re abandoning the static view of code and seeing it as a verb—a constantly changing medium that reacts to its immediate and extended environment. Your Code as a Crime Scene is the first book I’ve encountered that takes that view and runs with it. Adam presents tools, techniques, and insight that will change the way you develop software. You can’t unread this information, and you will see software differently. Dig in. Michael Feathers report erratum • discuss Acknowledgments My writing experience has been fun, challenging, and rewarding. I owe that to all the amazing people who helped me out. First of all, I’d like to thank the Pragmatic Bookshelf for this opportunity. In particular, I’d like to thank my editor, Fahmida Y. Rashid, who made this a much better book than what I could’ve done on my own.