Schedule

1. Introduction Software Reengineering There are OO legacy systems too ! 2. Reverse Engineering & Evolution How to understand your code 3. Visualization Scalable approach 4. Dynamic Analysis To be really certain 5. Restructuring Serge Demeyer How to Refactor Your Code Stéphane Ducasse 6. Code Duplication Oscar Nierstrasz The most typical problems 7. Software Evolution Learn from the past January 2019 8. Going Agile Continuous Integration 9. Conclusion http://scg.unibe.ch/download/oorp/ © S. Demeyer, S. Ducasse, O. Nierstrasz Software Reengineering and Evolution.2

Goals What is a Legacy System ?

We will try to convince you: “legacy” A sum of money, or a specified article, given to another by • Yes, Virginia, there are object-oriented legacy systems too! will; anything handed down by an ancestor or predecessor. • Reverse engineering and reengineering are essential — Oxford English Dictionary activities in the lifecycle of any successful software system. A legacy system is a piece of Typical problems with legacy systems: (And especially OO ones!) software that: • original developers not available • you have inherited, and • outdated development methods used • There is a large set of lightweight tools and techniques to • is valuable to you. • extensive patches and modifications have help you with reengineering. been made • missing or outdated documentation • Despite these tools and techniques, people must do job and they represent the most valuable resource.

Þ so, further evolution and development may be prohibitively expensive

© S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.3 © S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.4 Software Maintenance - Cost Continuous Development

Relative Cost Relative Maintenance Effort of Fixing Mistakes data from [Lien78a] Between 50% and 75% of x 200 17.4% Corrective (fixing reported errors) global effort is spent on “maintenance” ! 60.3% Perfective x 20 (new functionality) x 10 18.2% Adaptive Solution ? (new platforms or OS) • Better requirements engineering? x 5 • Better software methods & tools 4.1% Other (database schemas, CASE-tools, objects, x 1 components, …)? The bulk of the maintenance cost is due to new functionality requirement coding delivery Þ even with better requirements, it is hard to predict new functions design testing

© S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.5 © S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.6

Modern Methods & Tools ? Lehman's Laws

[Glas98a] quoting empirical study from Sasa Dekleva (1992) A classic study by Lehman and Belady [Lehm85a] identified several “laws” of system change. • Modern methods(*) lead to more reliable software

• Modern methods lead to less frequent software repair Continuing change • and ... • A program that is used in a real-world environment must change, or • Modern methods lead to more total maintenance time become progressively less useful in that environment.

Contradiction ? No! Increasing complexity • modern methods make it easier to change • As a program evolves, it becomes more complex, and extra resources ... this capacity is used to enhance functionality! are needed to preserve and simplify its structure.

(*) process-oriented structured methods, information engineering, data-oriented methods, prototyping, CASE-tools – not OO ! Those laws are still applicable…

© S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.7 © S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.8 What about Objects ? What about Components ?

Object-oriented legacy systems • = successful OO systems whose architecture and design no longer responds to changing requirements

Compared to traditional legacy systems • The symptoms and the source of the problems are the same • The technical details and solutions may differ

OO techniques promise better • flexibility, • reusability, Þ they do not come for free • maintainability Components are very brittle … After a while one inevitably resorts to glue :) • …

© S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.9 © S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.10

Soccer Field Metaphor How to deal with Legacy ?

• Assume 10 lines of code New or changing requirements will gradually degrade original design = 40 tiles of 1 x 1 cm … unless extra development effort is spent to adapt the structure • 12.5 million lines of code » 40 soccer fields New Functionality

Hack it in ?

Imagine 400 developers concurrently • duplicated code First … moving tiles around on 40 soccer fields • complex conditionals • refactor … • abusive inheritance • restructure • large classes/methods • reengineer

A. van Deursen, De software-evolutieparadox Take a loan on your software Investment for the future Intreerede TU Delft, 23 feb 2005 Þ pay back via reengineering Þ paid back during maintenance

©© S. A. Demeyer, Van Deursen S. Ducasse, O. Nierstrasz Reengineering Legacy Systems.11 © S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.12 Common Symptoms The Reengineering Life-Cycle

Lack of Knowledge Process symptoms (0) requirement Requirements • obsolete or no documentation • too long to turn things over to analysis • departure of the original production (2) problem developers or users • need for constant bug fixes (3) problem detection • disappearance of inside • maintenance dependencies resolution knowledge about the system • difficulties separating products Designs • limited understanding of entire Þsimple changes take too long system • people centric (1) model • lightweight Þmissing tests capture Code symptoms • duplicated code • code smells Code Þbig build times (4) program transformation

© S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.13 © S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.14

A Map of Reengineering Patterns 2. Reverse Engineering

• What and Why Tests: Your Life Insurance • First Contact Detailed Model Capture Migration Strategies ☞ Interview during Demo • Initial Understanding

Initial Understanding Detecting Duplicated Code

Redistribute First Contact Responsibilities

Transform Setting Direction Conditionals to Polymorphism

© S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.15 © S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.16 What and Why ? The Reengineering Life-Cycle

Definition (0) requirement Reverse Engineering is the process of analysing a subject system Requirements analysis ☞ to identify the system’s components and their interrelationships and ☞ create representations of the system in another form or at a higher level of (2) problem (3) problem abstraction. — Chikofsky & Cross, ’90 detection resolution

(0) req. analysis Motivation Designs (1) model capture Understanding other people’s code issues (cfr. newcomers in the team, code reviewing, (1) model • scale original developers left, ...) capture • speed • accuracy • politics Code Generating UML diagrams is NOT reverse engineering ... but it is a valuable support tool (4) program transformation

© S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.17 © S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.18

First Contact First Project Plan

feasibility assessment Use standard templates, including: System experts (one week time) • project scope Talk with Talk with ☞ see "Setting Direction" end users developers • opportunities Chat with the Interview Talk about it ☞ e.g., skilled maintainers, readable source-code, documentation during Demo Maintainers • risks ☞ e.g., absent test-suites, missing libraries, … ☞ record likelihood (unlikely, possible, likely) Software System Verify what & impact (high, moderate, low) for causing problems you hear Read Read it Compile it • go/no-go decision about it • activities Read All the Code Skim the Do a Mock ☞ fish-eye view in One Hour Documentation Installation

© S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.19 © S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.20 Interview during Demo Initial Understanding Problem: What are the typical usage Top down scenarios? Recover design

Solution: Ask the user! Speculate about Design • Solution: interview during demo - select several users understand Þ ITERATION - demo puts a user in a positive mindset higher-level model Study the - demo steers the interview Analyze the Exceptional • ... however Persistent Data Entities ☞ Which user ? ☞ Users complain Recover Identify database problems ☞ What should you ask ? Bottom up

© S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.21 © S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.22

3. Software Visualization The Reengineering Life-cycle

• Introduction (0) requirement ☞ The Reengineering life-cycle Requirements analysis • Examples (2) problem (3) problem • Lightweight Approaches detection ☞ CodeCrawler resolution • Dynamic Analysis Designs (2) problem detection ☞ Key Concept Identification issues ☞ Feature Location (1) model • Tool support • Conclusion capture • Scalability • Efficiency

Code

(4) program transformation

© S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.23 © S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.24 Visualising Hierarchies Bottom Up Visualisation

• Euclidean cones ☞ Pros: All program entities and • More info than 2D relations ☞ Cons: • Lack of depth • Navigation • Hyperbolic trees ☞ Pros: • Good focus • Dynamic ☞ Cons: Filter • Copyright

© S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.25 © S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.26

A lightweight approach System Complexity View • A combination of metrics and software visualization Entity ☞ Visualize software using Relationship colored rectangles for the entities and edges for the relationships ☞Render up to five metrics on one node: X Coordinate Nodes: Classes • Size (1+2) Y Coordinate Edges: Inheritance Relationships Color tone Width: Number of attributes • Color (3) Height Height: Number of methods • Position (4+5) Color: Number of lines of code Width

© S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.27 © S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.28 Inheritance Classification View Data Storage Class Detection View

Boxes: Classes Edges: Inheritance Width: Number of Methods Added Height: Number of Methods Overridden Boxes: Classes Color: Number of Method Extended Width: Number of Methods Height: Lines of Code Color: Lines of Code

© S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.29 © S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.30

Industrial Validation State of the Art Tooling

Personal experience 1. source{d} 2-3 days to get something https://sourced.tech https://github.com/src-d/engine Nokia (C++ 1.2 MLOC >2300 classes) Nokia (++/Java 120 kLOC >400 classes) MGeniX (Smalltalk 600 kLOC >2100classes) Bedag (COBOL 40 kLOC) 2. teamscale ... https://www.cqse.eu/ Used by developers + Consultants https://github.com/cqse

3. codescene https://codescene.io https://github.com/empear-analytics

© S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.31 © S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.32 4. Dynamic Analysis Key Concept Identification Ant docs Ant mining web I C _ C

- • Extract run-time coupling • Key Concept Identification C ’

+ • Apply datamining (“google”)

• Feature Location Class • Experiment with documented Project √ √ open-source cases (Ant, JMeter) UnknownElement √ √ ☞ recall: +- 90 % Task √ √ ☞ precision: +- 60 % Main √ √ IntrospectionHelper √ √ ProjectHelper √ √ RuntimeConfigurable √ √ Target √ √ ElementHandler √ √ TaskContainer × √ Recall (%) 90 - Precision (%) 60 - © S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.33 © S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.34

Feature Location 5. Restructuring

T. Eisenbarth, R. Koschke, and D. Redistribute Responsibilities Simon. Locating features in source code. IEEE Transactions ☞ Move Behaviour Close to Data on Software Engineering, ☞ Eliminate Navigation Code 29(3):210–224, March 2003. ☞ Split up God Class ☞ Empirical Validation

Replication is not supported, industrial cases are rare, …. In order to help the discipline mature, we think that more systematic empirical evaluation is needed. [To n e l l a et. Al, in Empirical Software Engineering]

© S. Demeyer, S. Ducasse, O. Nierstrasz Reengineering Legacy Systems.35 © S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.36 Redistribute Responsibilities Move Behavior Close to Data (example 1/2) * Employee Payroll * +telephoneNrs +printEmployeeLabel() Monster client +name(): String of data containers Chains of +address(): String TelephoneGuide data containers +printEmployeeTelephones()

Split Up God Class System.out.println(currentEmployee.name() ); System.out.println(currentEmployee.address() ); Eliminate Navigation Code for (int i=0; i < currentEmployee.telephoneNumbers.length; i++) { System.out.print(currentEmployee.telephoneNumbers[i]); System.out.print(" "); Data containers } System.out.println(""); … for … Move Behaviour Close to Data System.out.print(" -- "); …

© S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.37 © S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.38

Move Behavior Close to Data (example 2/2) Eliminate Navigation Code * Car Carburetor Employee Payroll Engine -engine * +fuelValveOpen - telephoneNrs +printEmployeeLabel() +carburetor +increaseSpeed() - name(): String - address(): String TelephoneGuide … +printLabel(String) +printEmployeeTelephones() engine.carburetor.fuelValveOpen = true

Engine Car … Carburetor -carburetor -engine public void printLabel (String separator) {emp.printLabel(" "); +fuelValveOpen +speedUp() +increaseSpeed() System.out.println(_name); … System.out.println(_address); … for (int i=0; i < telephoneNumbers.length; i++) { carburetor.fuelValveOpen = true engine.speedUp() System.out.print(telephoneNumbers[i]); System.out.print(separator); Carburetor Engine Car } … -fuelValveOpen -carburetor -engine System.out.println("");} emp.printLabel(" -- "); +openFuelValve() +speedUp() +increaseSpeed() … fuelValveOpen = true carburetor.openFuelValve() © S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.39 © S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.40 Split Up God Class Split Up God Class: 5 variants Problem: Break a class which monopolizes control? Mail client filters incoming mail Solution: Incrementally eliminate navigation code FilterActionD • Detection: A Filter1 Extract Controller ☞ measuring size Controller behavioral class Filter2 ☞ class names containing Manager, System, Root, Controller MailHeader ☞ the class that all maintainers are avoiding Extract C Filter1 • How: behavioral class Controller Extract ☞ move behaviour close to data + eliminate navigation code Filter2 data class ☞ remove or deprecate façade MailHeader E FilterAction • However: B Extract Filter1 ☞ If God Class is stable, then don't split Controller Filter1 data class Filter2 Þ shield client classes from the god class Controller NameValuePair Filter2 MailHeader

© S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.41 © S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.42

Empirical Validation Interpretation of Results

• Controlled experiment with 63 last- • “Optimal decomposition” differs with respect to training year master-level students (CS and ICT) ☞ Computer science: preference towards C-E ☞ ICT-electronics: preference towards A-C

IndependentVariables DependentVariables • Advanced OO training can induce a preference towards Accuracy Institution particular styles of decomposition Experimental ☞ Consistent with [Arisholm et al. 2004] task 9 3 Time 6 God class decomposition

© S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.43 © S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.44 6. Code Duplication The Reengineering Life-Cycle

a.k.a. Software Cloning, Copy&Paste (0) requirement Programming Requirements analysis

• Code Duplication (2) problem (3) problem detection ☞ What is it? resolution ☞ Why is it harmful? • Detecting Code Duplication Designs (2) Problem detection • Approaches • A Lightweight Approach (1) model issues • Visualization (dotplots) capture • Scale • Duploc • Unknown a priori • Recent trends Code

(2) Problem detection

© S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.45 © S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.46

Code is Copied How Much Code is Duplicated?

Small Example from the Mozilla Distribution (Milestone 9) Usual estimates: 8 to 12% in normal industrial code Extract from /dom/src/base/nsLocation.cpp 15 to 25 % is already a lot!

Duplication with Case Study LOC without comments comments gcc 460’000 8.7% 5.6% Database Server 245’000 36.4% 23.3% Payroll 40’000 59.3% 25.4% Message Board 6’500 29.4% 17.4%

© S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.47 © S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.48 Copied Code Problems Code Duplication Detection Nontrivial problem: • General negative effect: • No a priori knowledge about which code has been copied ☞ Code bloat • How to find all clone pairs among all possible pairs of segments? • Negative effects on Software Maintenance ☞ Copied Defects ☞ Changes take double, triple, quadruple, ... Work ☞ Dead code ☞ Add to the cognitive load of future maintainers • Copying as additional source of defects ☞ Errors in the systematic renaming produce unintended aliasing • Metaphorically speaking: ☞ Software Aging, “hardening of the arteries”, Type I ☞ “Software Entropy” increases even small design changes become very Type II difficult to effect (& Type III) Type IV

© S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.49 © S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.50

General Schema of Detection Process Simple Detection Approach (i)

• Assumption: • Code segments are just copied and changed at a few places • Code Transformation Step • remove white space, comments • remove lines that contain uninteresting code elements (e.g., just ‘else’ or ‘}’)

… Comparison … Author Level Transformed Code //assign same fastid as container Technique fastid = NULL; fastid=NULL; [John94a] Lexical Substrings String-Matching const char* fidptr = get_fastid(); constchar*fidptr=get_fastid(); [Duca99a] Lexical Normalized Strings String-Matching if(fidptr != NULL) { if(fidptr!=NULL) [Bake95a] Syntactical Parameterized Strings String-Matching int l = strlen(fidptr); intl=strlen(fidptr) fastid = newchar[ l + 1 ]; [Mayr96a] Syntactical Metric Tuples Discrete comparison fastid = newchar[l+] [Kont97a] Syntactical Metric Tuples Euclidean distance [Baxt98a] Syntactical AST Tree-Matching

© S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.51 © S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.52 Simple Detection Approach (ii) A script for C++ (1/2)

• Code Comparison Step ☞ Line based comparison (Assumption: Layout did not change during copying) ☞ Compare each line with each other line. ☞ Reduce search space by hashing: 1. Preprocessing: Compute the hash value for each line 2. Actual Comparison: Compare all lines in the same hash bucket • Evaluation of the Approach ☞ Advantages: Simple, language independent ☞ Disadvantages: Difficult interpretation

© S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.53 © S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.54

A Perl script for C++ (2/2) Output Sample

Lines: create_property(pd,pnImplObjects,stReference,false,*iImplObjects); create_property(pd,pnElttype,stReference,true,*iEltType); • Handles multiple files create_property(pd,pnMinelt,stInteger,true,*iMinelt); create_property(pd,pnMaxelt,stInteger,true,*iMaxelt); • Removes comments create_property(pd,pnOwnership,stBool,true,*iOwnership); and white spaces Locations: 6178/6179/6180/6181/6182 6198/6199/6200/6201/6202 • Controls noise (if, {,) Lines: create_property(pd,pnSupertype,stReference,true,*iSupertype); • Granularity (number of create_property(pd,pnImplObjects,stReference,false,*iImplObjects); lines) create_property(pd,pnElttype,stReference,true,*iEltType); create_property(pd,pMinelt,stInteger,true,*iMinelt); • Possible to remove create_property(pd,pnMaxelt,stInteger,true,*iMaxelt); Locations: 6177/6178 keywords 6229/6230 Lines = duplicated lines Locations = file names and line number

© S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.55 © S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.56 Visualization of Duplicated Code Visualization of Copied Code Sequences

•Visualization provides insights into the duplication situation •A simple version can be implemented in three days Detected Problem •Scalability issue File A contains two copies of a piece of code •Dotplots — Technique from DNA Analysis • Code is put on vertical as well as horizontal axis File B contains another copy of • A match between two elements is a dot in the matrix this code

Possible Solution Extract Method

All examples are made using Duploc from an industrial case study (1 Mio LOC C++ System) © S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.57 © S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.58

Visualization of Repetitive Structures Visualization of Cloned Classes

Detected Problem 4 Object factory clones: a switch Detected Problem: statement over a type variable is Class A is an edited copy used to call individual construction of class B. Editing & Insertion code Possible Solution Possible Solution Subclassing … Strategy Method

© S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.59 © S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.60 Recent Visualization of Clone Families Trends Overview Detail

Clone Detection Inside

20 Classes implementing lists for different data types © S. Demeyer, S. Ducasse, O. Nierstrasz © S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.61 Object-Oriented Reengineering.62

7. Software Evolution The Reengineering Life-Cycle

• Exploiting the Version Control System (0) requirement ☞ Visualizing CVS changes Requirements analysis • The Evolution Matrix (2) problem • Test History (3) problem detection resolution

Designs (2) Problem detection It is not age that turns a piece of software into a legacy system, (1) model Issues capture • scale but the rate at which it has been developed and adapted without being reengineered. [Demeyer, Ducasse and Nierstrasz: Object-Oriented Reengineering Patterns] Code

(2) Problem detection

© S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.63 © S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.64 Ownership Map: Analyse CVS changes 1) Vertical lines = Frequent Changers Developer Activity

Edit Takeover

3) Triangle = Core Reduces

2) Horizontal line = Shotgun Surgery 4) Block Shift = Design Change

© S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.65 Monologue© S. Demeyer, S. Ducasse, O.Familiarization Nierstrasz DialogueObject-Oriented Reengineering.66

Data from Windows Vista and Windows 7 What to (re)test ? The Evolution Matrix

Software components with a Removed Classes Last Version high level of ownership will have fewer failures than components with lower top ownership levels. First Version

Added Classes

Major Leap Software components with many minor contributors will have more failures than software components that have fewer. Growth Stabilisation TIME (Versions)

© S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.67 © S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.68 System under study = checkstyle Example: MooseFinder (38 Versions) Test history integration tests

… affect unit tests … affect unit tests

unit tests

phased testing © S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.69 single© S. Demeyer, test S. Ducasse, O. Nierstrasz Reengineering Legacy Systems.70

Git repositories of the XWiki, OpenLMIS and Atlas Selenium Tests © Laurent Christophe (Vrije Universiteit Brussel) 8. Going Agile

• Continuous Integration / Deployment

version developer scenario deploy to measure & build deploy control tests tests production validate

<>

Avoid Magic Constants !!

© S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.71 © S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.72 Traditional Release Cycle Rapid Release Cycle 8.0 Mining Software Repositories 1.0 1.5 2.0 3.0 3.5 3.6 4.0 5.0 7.0 9.0

The Mining Repositories (MSR) field analyzes the rich data available in software repositories to uncover interesting and actionable information about software (a) Time Line of Major Versions of FireFox systems and projects.

Conferences Hall of Fame — Mining Challenge 2018—15th edition, Gothenburg, Sweden 2018 — IDE Event Stream (JetBrains) 2017—14th edition, Buenos Aires, Argentina 2017 — TravisTorrent (Github) 2016—13th edition, Austin, Texas 2016 — BOA (SourceForge & Github) 2015—12th edition, Florence, Italy 2015 — StackOverflow 2014—11th edition, Hyderabad, India 2014 — GitHub 2013—10th edition, San Francisco, USA 2013 — StackOverflow (b) Time Line of Minor Versions of FireFox 2012—9th edition, Zürich, CH 2012 — Android Figure 1. Timeline of FireFox versions. 2011—8th edition, Honolulu, HI, USA 2011 — Netbeans+Eclipse 2010—7th edition, Cape Town, ZAF 2010 — GNOME Projects channels are respectively 100,000 for NIGHTLY, 1 million by bug triaging developers and assigned for fixing. When 2009—6th edition, Vancouver, CAN 2009 — GNOME project for AURORA, 10 million for BETA and 100+ millions for a developer fixes a bug, he typically submits a patch to 2008—5th edition, Leipzig, DEU 2008 — Eclipse a major Firefox version [11].[Khom2014] NIGHTLY reaches Khomh, Firefox F. Adams,Bugzilla. B, Dhaliwal, Once approved, T and the Zou, patch Y code is integrated into 2007—4th edition, Minneapolis, MN, USA 2007 — Eclipse Developer developers and contributors, while other channels (i.e., AU- the source code of Firefox on the corresponding channel and 2006—3rd edition, Shanghai, CHN 2006 — PostgreSQL RORA and BETA) recruitUnderstanding external users for testing.the Impact The ofmigrated Rapid through Releases the other on Software channels for Quality: release. Bugs that 2005—2nd edition, Saint Luis, MO, USA source code on AURORA isThe tested Case by web of developersFirefox, Empirical who take Software too long to get Engineering, fixed and hence Springer. miss a scheduled release 2004—1st edition, Edinburgh, UK are interested in the latest standards,http://link.springer.com/article/10.1007/s10664 and by Firefox add-on are picked up by the next- release’s014-9308 channel.-x developers who are willing to experiment with new browser © S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.73 APIs. The BETA channel is tested by Firefox’s regular beta III. STUDY DESIGN testers. This section presents the design of our case study, which Each version of Firefox in any channel embeds an auto- aims to address the following three research questions: mated crash reporting tool, i.e., the Mozilla Crash Reporter, 1) Does the length of the release cycle affect the software to monitor the quality of Firefox across all four channels. quality? Whenever Firefox crashes on a user’s machine, the Mozilla 2) Does the length of the release cycle affect the fixing Crash Reporter [12] collects information about the event of bugs? and sends a detailed crash report to the Socorro crash 3) Does the length of the release cycle affect software report server. Such a crash-report includes the stack trace updates? of the failing thread and other informationRecommender about a user Systems environment, such as the operating system, the version of A. Data Collection Firefox, the installation time, and a list of plug-ins installed. Socorro groups similar crash-reports into crash-types. In this study, we analyze all versions of Firefox that were These crash-types are then ranked by their frequency of released in the period from January 01, 2010 to December occurrence by the Mozilla quality assurance teams. For the 21, 2011. In total, we study 25 alpha versions, 25 beta top crash-types, testers file bugs in Bugzilla and link them to versions, 29 minor versions and 7 major versions that were the corresponding crash-type in the Socorro server. Multiple released within a period of one year before or after the bugs can be filed for a single crash-type andMisclassified multiple crash-bug reportsmove ? to a rapid release model. Firefox 3.6, Firefox 4 and types can be associated with the same bug. For each crash- their subsequent minor versions were developed following type, the Socorro server provides a crash-type summary, i.e., a traditional release cycle with an average cycle time of a list of the crash-reports of the crash-type and a set of bugs 52 weeks between the major version releases and 4 weeks that have been filed for the crash-type. Who to fixbetween ? How the long minor to fix version ? releases. Firefox 5, 6, 7, 8, 9 Firefox users can also submit bug reports in Bugzilla and their subsequent minor versions followed a rapid release manually. A bug report contains detailed semantic infor- Descriptionmodel with antext average mining release time interval of 6 weeks mation about a bug, such as the bug open date, the last between the major releases and 2 weeks between the minor modification date, and the bug status. The bugs are triaged releases. Table I shows additional descriptive statistics of the different versions. bugs are fixed faster [Khom2014] Khomh, F. Adams, B, Dhaliwal, T and Zou, Y Understanding the Impact of (butRapid … Releases harder on bugs Software propagated Quality: toThe later Case releases) of Firefox, Stack Trace link to source code Empiricalamount Software of pre Engineering,- & post-release Springer. bugs ± the same http://link.springer.com/article/10.1007/s10664the program crashes earlier -014-9308-x (perhaps due to recent features) © S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.76 9. Conclusion Goals

1. Introduction We will try to convince you: There are OO legacy systems too ! • Yes, Virginia, there are object-oriented legacy systems too! 2. Reverse Engineering How to understand your code ☞ … actually, that's a sign of health 3. Visualization • Reverse engineering and reengineering are essential Scalable approach activities in the lifecycle of any successful software 4. Dynamic Analysis system. (And especially OO ones!) To be really certain ☞ … consequently, do not consider it second class work 5. Restructuring How to Refactor Your Code • There is a large set of lightweight tools and 6. Code Duplication techniques to help you with reengineering. The most typical problems ☞ … check our book, but remember the list is growing 7. Software Evolution • Despite these tools and techniques, Learn from the past people must do job and represent the most valuable 8. Going Agile resource. Continuous Integration 9. Conclusion ☞ … pick them carefully and reward them properly

Þ Did we convince you ? © S. Demeyer, S. Ducasse, O. NierstraszQ§ Object-Oriented Reengineering.77 © S. Demeyer, S. Ducasse, O. Nierstrasz Object-Oriented Reengineering.78