Patterns for Evolving Agile Architecture

5/3/2017

Deliver FAST “with confidence” Joseph W. Yoder Twitter: @metayoda [email protected] The Refactory http://www.refactory.com Teams That Innovate http://www.teamsthatinnovate.com

Introducing Joseph

Founder and Architect, The Refactory, Inc. Pattern enthusiast, author and Hillside Board President Author of the Big Ball of Mud Pattern Adaptive Systems expert (programs adaptive software, consults on adaptive architectures, author of adaptive architecture patterns, metatdata maven, website: adaptiveobjectmodel.com) Agile enthusiast and practitioner Business owner (leads a world class development company) Consults and trains top companies on design, refactoring, pragmatic testing Amateur photographer, motorcycle enthusiast, enjoys dancing samba!!! Loves Sushi, Ramen, Taiko Drums 

1 5/3/2017

Confidence

2 5/3/2017

Kano Model

architecture quality can be invisible

www.kanomodel.com

3 5/3/2017

architecture quality can be invisible

…especially when the spotlight is on

FEATURES

4 5/3/2017

Mobile New Products Version EBiz Features

The Solution

5 5/3/2017

What’s below the waterline?

Reliability all those “ilities” Scalability we can’t ignore … Performance Stability Maintainability

6 5/3/2017

Agile Myths

 Simple solutions are always best  We can easily adapt to changing requirements (new requirements)  Scrum/TDD will ensure good Design/Architecture  Good architecture simply emerges from “good” development practice  You always go fast when doing agile  Make significant architecture changes at the last moment “www.agilemyths.com” Sustaining Your Architecture

Big Ball of Mud Alias: Shantytown, Spaghetti Code

A BIG BALL OF MUD is haphazardly structured, sprawling, sloppy, duct-tape and bailing wire, spaghetti code jungle. The de-facto standard software architecture. Why is the gap between what we preach and what we practice so large?

We preach we want to build high quality systems but why are BBoMs so prevalent?

Sustaining Your Architecture

7 5/3/2017

Big Ball of Mud Alias: Shantytown, Spaghetti Code

A BIG BALL OF MUD is haphazardly Lisa Pollack structured, sprawling, sloppy, duct-tape and bailing wire, spaghetti code jungle. Brazilian architect Oscar Niemeyer The de-facto standard software architecture. Why is the gap between what we preach and what we practice so large?

We preach we want to build high quality systems but why are BBoMs so prevalent?

Sustaining Your Architecture

Complex vs Complicated Systems (Cynefin Framework)

"Cynefin as of 1st June 2014" by Snowded - Own work. Licensed under CC BY-SA 3.0 via Commons - https://commons.wikimedia.org/wiki/File:Cynefin_as_of_1st_June_2014.png#/media/File:Cynefin_as_of_1st_June_2014.png

8 5/3/2017

How can I be more confident

Linda Masters Northrop © Can Stock Photo /

Values Drive Practices

9 5/3/2017

Agile/Lean Design Values

 Core values:

 Design Simplicity

 Quick Feedback

 Frequent Releases

 Continuous Improvement

 Teamwork/Trust

 Satisfying stakeholder needs

 Building Quality Software  Keep Learning  Sustainable Development

10 5/3/2017

Delivery Size???

Delivery Size is Key

Large Delivery Size can cause many issues

Issues:  More potential defects  Longer time to get feedback  Slower adjust time  Harder to experiment  Bugs take a long time to fix

11 5/3/2017

Small Deliveries Quick Feedback

Chris Richardson

12 5/3/2017

What about Quality?

Code Bad Smells Have you ever looked at a piece of software that doesn't smell very nice?

A code smell is any symptom in the source code that can indicate a problem!

13 5/3/2017

Neglect Is Contagious

.Disorder increases and software rots over time .Don’t tolerate a broken window

http://www.pragmaticprogrammer.com/ppbook/extracts/no_broken_windows.html

Is it better to Or to let dirt clean little by and mess little? accumulate?

14 5/3/2017

Some dirt becomes very hard to clean if you do not clean it right away!

Technical Debt?

Clean Code Doesn't Just Happen

•You have to craft it •You have to maintain it •You have to make a professional commitment

“Any fool can write code that a computer can understand. Good programmers write code that humans can understand.” – Martin Fowler

15 5/3/2017

Refactoring “If you value clean code…”

Refactorings Behavior Preserving Program Transformations

• Rename Instance Variable • Promote Method to Superclass • Move Method to Component Always done for a reason!!! Refactoring is key and integral TDD to most Agile processes!!! Sustaining Your Architecture

16 5/3/2017

Two Refactoring Types*

Floss Refactorings—frequent, small changes, intermingled with other programming (daily health)

Root canal refactorings—infrequent, protracted refactoring, during which programmers do nothing else (major repair)

* Emerson Murphy-Hill and Andrew Black in “Refactoring Tools: Fitness for Purpose” http://web.cecs.pdx.edu/~black/publications/IEEESoftwareRefact.pdf

Ten Most Putrid List 0) Magic Numbers, 1) Sloppy Layout, 2) Dead Code, 3) Lame Names, 4) Commented Code, 5) Duplicated Code, 6) Feature Envy, 7) Inappropriate Intimacy, 8) Long Methods & Large Class, 9) Primitive Obsession & Long Parameter List, 10) Switch Statement & Conditional Complexity …

17 5/3/2017

Safe Refactorings  Rename is always safe!!!  New Abstract Class moving common features up  Extract Method (always safe)  Extract Interface / Extract Constant  Pull Up / Push Down  Create common component for shared internal methods

 Fairly safe but can be harder to share Sustaining Your Architecture

You Must Test

When you find smelly code, you often apply refactorings to clean your code.

Testing is a key principle for safe refactoring!

Sustaining Your Architecture

18 5/3/2017

Common Wisdom

Work refactoring into your daily routine…

“In almost all cases, I’m opposed to setting aside time for refactoring. In my view refactoring is not an activity you set aside time to do. Refactoring is something you do all the time in little bursts.” — Martin Fowler

Sustaining Your Architecture

But We Don’t Have Time!

19 5/3/2017

PAUSE POINTS HELP

Professional Responsibility There’s no time to wash hands, get to the next patient!

http://en.wikipedia.org/wiki/Image:I_Semmelweis.jpg

20 5/3/2017

Professionalism

Make it your responsibility to create software: Delivers business value Is clean Is tested Is simple Good design principles

When working with existing code: If you break it, you fix it You never make it worse than it was You always make it better

Quality Focused Checklists

• Release Checklists* – Agreed upon checklist for quality and major architecture concerns • Use at pause points – sprint planning, release planning, …

*Thanks, James Thorpe for sharing your company’s checklist

21 5/3/2017

Two Kinds of Checklists 1.Read-review 2.Do-confirm

Checklists at MozaicWorks*

*Thanks, Alex Balboaca for sharing

22 5/3/2017

Slack Time Key Principle: Need Slack time to improve

Ways to get slack time…  Monitor and Make Visible  Reduce Waste (Muda)  Inject time into process (retros, daily cleanup, …)

Try little experiments…

Kaizen

The Sino-Japanese word "kaizen" simply means "change for better", with no inherent meaning of either "continuous" or "philosophy" in Japanese dictionaries or in everyday use. The word refers to any improvement, one-time or continuous, large or small, in the same sense as the English word "improvement". (Wikipedia)

Most view it as Continuous Improvement…

23 5/3/2017

Continuous Improvement “Retrospectives are Key!!!”

Small Steps we can take - next sprint!!!

Spotify: Innovation

24 5/3/2017

Regular Practices

Allocate time for dealing with tech debt Team building and education sessions Evaluate and Reflect…small changes

HOW SYSTEM QUALITY WORK CAN FIT INTO YOUR RHYTHMS

25 5/3/2017

Build architectural quality into your project rhythms

“QUALITY IS NOT AN ACT, IT IS A HABIT.” —ARISTOTLE

Some decisions are too important to leave until The Last Responsible Moment

SO CHOOSE THE MOST RESPONSIBLE MOMENT

26 5/3/2017

Qualify the Roadmap “All you need is the plan, the roadmap, and the courage to press on to your destination” — Earl Nightingale

LOW HIGH TBD RISK RISK NORMAL Qualify the Roadmap 2017 2018 Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar

MOBILE WEB v1 MOBILE WEB v2 RICH MOBILE WEB APPS

PC PLATFORM v1 PC PLATFORM v2 ONGOING RELEASES

DEVELOPMENT MOBILE RESEARCH ANDROID v1 IOS v1 RESPONSIVE DESIGN

PERSISTENCE FRAMEWORK MOBILE GENERIC SERVICES SYBASE TO ORACLE MIGRATION

LOAD BALANCING MOBILE SECURITY PLATFORM STABILITY

CLOUD RESEARCH MICROSERVICES NO SQL / BIG DATA v1 NO SQL / BIG DATA v2 ENTEPRISE ARCHITECTURE

DELIVERY BUDGET RESOURCE ARCHITECTURE DEPENDENCIES RISKS ISSUES ON RADAR Delays Budget will All resources Persistence Partnerships expected to need on track. Framework and services COMPETITOR DELIVERY AUG 2017 Version 1 bolstering in Load- all in place E Corp – new Tech issues New mobile Q2 2017 Balancing. and on track. product. opportunity Cloud ARCHITECTURE ARCHITECTURE OCT 2017

DASHBOARD Performance Migration Re-evaluate Platform stability Security NO SQL strategy

27 5/3/2017

Qualify the Backlog

You can add backlog items for technical debt and quality-related architecture work… yes, you can

Make Architecture Work Visible and Explicit

Visible Invisible

Positive Value Invisible Architectural Visible Feature Feature

Negative Value Visible Defect Technical Debt

Color your backlog—Phillipe Kruchten http://philippe.kruchten.com/2013/12/11/the-missing-value-of-software-architecture/

28 5/3/2017

Question:

HOW CAN I FIND AND DESCRIBE QUALITIES WHILE BEING AGILE?

Find Essential Qualities

System qualities often overlooked or simplified until late in the development process causing delays and extensive refactoring and rework …

 How can agile teams understand & prioritize essential qualities?

29 5/3/2017

Quality Scenarios, Quality Stories, and Fold-Out Qualities

“Users initiate 1,000 order transactions per minute under normal operations; transactions are processed with an average latency of 2 seconds”

Example Performance Scenario

Source of Stimulus Artifact Response Response Stimulus Measure Initiate Users System Transactions Average order processed latency of 2 Environment seconds Full load during busy part of order processing

“Users initiate 1,000 order transactions per minute under busy part of day; transactions are processed with an average latency of 2 seconds”

30 5/3/2017

Possible Performance Scenario Values

Portion of Possible Values Scenario Source External systems, users, components, databases Stimulus Periodic events, sporadic or random events (or a combination) Artifact The system’s services, data, or other resources Environment The state the system can be in: normal, overloaded, partial operation, emergency mode… Response Process the event or event sequence and possibly change the level of service Response Times it takes to process the arriving events (latency or Measure deadline by which event must be processed), the variation in this time, the number of events that can be processed within a particular time interval, or a characterization of events that cannot be processed (missed rate, data loss)

Fold-out Qualities

Quality-related acceptance criteria attached to user stories

Security: HowUse 256is credit bit SSL information securely transmittedencryption….? Security: Is credit information retained? Do I have control over this?

Performance: How fast can I place an order and receive confirmation? Performance: WhenOrder timethere < are 2 seconds lots of users?

Usability: Can I cancel my order? When?

“Acceptable means done with quality”

31 5/3/2017

How Quality Fits Into An Agile Process

Incorporate Feedback Functional Quality Acceptance Testing Testing Can Include Quality Daily Review Items Include relevant Product quality Envisioning tasks / Roadmap Deploy to Identify: Stakeholders Architecture Risks Develop Key Quality Scenarios Plan a Sprint Run a Sprint and Manage Landing Zone Criteria the Backlog

Test Driven Development

32 5/3/2017

ONGOING QUALITY ACTIVITIES

Visibility is Important

33 5/3/2017

Monitor System Qualities— Build An Operational Dashboard

Continuous Inspection

Asian PLoP 2014 Paper

CODE SMELL DETECTION METRICS (TEST COVERAGE, CYCLOMATIC COMPLEXITY, TECHNICAL DEBT, SIZES, …) APPLICATION SECURITY CHECKS ARCHITECTURAL CONFORMANCE

AUTOMATE WHERE YOU CAN!!!

Sustaining Your Architecture

34 5/3/2017

Continuous Inspection

Sustaining Your Architecture

Define Architecture Triggers

• Conditions that cause architecture investigation/ tasks – Quality target no longer met – Code quality metrics violations – … • Have broad system impact

35 5/3/2017

Periodically Re-Evaluate Architecture Risks

Iteration Delivery and Planning Feedback Continuous Improvement Architecture Quality

Implementation

Agile Values Can Drive Architectural Practices  Do something. Don’t debate or discuss architecture too long  Do something that buys you information  Prove your architecture ideas Do  Reduce risks something!  Make it testable  Prototype realistic scenarios Prove & that answer specific questions  Incrementally refine Refine. your architecture  Defer architectural decisions that don’t need to be immediately made

36 5/3/2017

Patterns for Evolving Agile Architecture

USA PLoP 2015

Patterns for Evolving Agile Architecture

Asian PLoP 2015

37 5/3/2017

Patterns for Being Agile at Quality

Core Patterns Breaking Down Barriers Integrate Quality

Becoming Agile at Quality Identifying Qualities Making Qualities Visible Whole Team Finding the Qualities System Quality Quality Focused Sprints Agile Quality Scenarios Dashboard Product Quality Champion Quality Stories System Quality Radiator Agile Quality Specialist Measureable Qualify the Roadmap Spread the System Qualities Qualify the Backlog Quality Workload Fold-out Qualities Automate First Agile Landing Zone Shadow the Quality Expert Quality Checklists Pair with a Recalibrate the Quality Advocate Landing Zone Agree on Quality Targets

QA to AQ: Patterns about transitioning from Quality Assurance to Agile Quality, AsianPLoP 2014 QA to AQ QA to AQ Part Two: Shifting from Quality Assurance to Patterns about transitioning from Agile Quality, PLoP 2014 Quality Assurance to Agile Quality Joseph W. Yoder 1, Rebecca Wirfs-Brock2, Ademar Aguiar3 QA to AQ Part Three: Shifting from Quality Assurance 1 The Refactory, Inc., to Agile Quality “Tearing Down the Walls”, SugarLoafPLoP 2014 2Wirfs-Brock Associates, Inc.

3 FEUP QA to AQ Part Four: Shifting from Quality Assurance [email protected], [email protected], [email protected] to Agile Quality “Prioritizing Qualities and Making them Visible”, PLoP 2015 Abstract. As organizations transition from waterfall to agile processes, Quality Assurance (QA) activities and roles need to evolve. Traditionally, QA activities QA to AQ Part Five: Being Agile At Quality “Growing have occurred late in the process, after the software is fully functioning. As a Quality Awareness and Expertise”, AsianPLoP 2016 consequence, QA departments have been “quality gatekeepers” rather than actively engaged in the ongoing development and delivery of quality software. Agile teams incrementally deliver working software. Incremental delivery provides an QA to AQ Part Sox: Being Agile At Quality “Enabling opportunity to engage in QA activities much earlier, ensuring that both and Infusing Quality”, AsianPLoP 2016 functionality and important system qualities are addressed just in time, rather than too late. Agile teams embrace a “whole team” approach. Even though special skills may be required to perform certain development and Quality Assurance tasks, Patterns to Develop and Evolve Architecture in an everyone on the team is focused on the delivery of quality software. This paper Agile Project, PLoP 2016, outlines 21 patterns for transitioning from a traditional QA practice to a more agile process. Six of the patterns are completely presented that focus on where quality is addressed earlier in the process and QA plays a more integral role. Continuous Inspection, AsianPLoP 2016

Categories and Subject Descriptors D.1.5 [Programming Techniques]: NEED TO ADD HERE

General Terms Agile, Quality Assurance, Patterns, Testing …PATTERNS FOR TRANSITIONING Keywords Agile Quality, Quality Assurance, Testing

FROM TRADITIONAL TO AGILE QA Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission. Preliminary versions of these papers were presented in a writers' workshop at the 3rd Asian Conference on Pattern Languages of Programs (AsianPLoP). AsianPLoP'2014, March 5-7, Tokyo, Japan. Copyright 2014 is held by the author(s). ACM 978-1-XXXX- Copies available off our XXXX-X. AND AGILE ARCHITECTURE websites.

38 5/3/2017

Where do you start? • Visibility - Monitor Qualities • Small quick releases • Pick some low hanging fruit – Make goals visible – Colorize your backlog – Create quality-related checklists • Spread attention to code quality throughout teams • Depends on where you are © Can Stock Photo Inc. / iqoncept and where the pain is…

How Much Architecture Risk do you Have?

• New architecture, new product, new market, new technologies Higher • Transforming an existing product • Evolving a product • Feature extensions on a “stable” architecture Lower

“the more risk, the more attention you need to pay to architecture”

39 5/3/2017

How Big is your Project? Small v. Large Projects

Small Projects Large Projects • 6-8 people • Multiple teams • Non-life critical • Known domain • Known domain but tackling a big problem • “Naturally” emerging architecture can architecture typically evolves OK without much attention reflect organization structure • Significant risks, challenges, unknowns, lots of coordination

architecture needs explicit attention

Indicators You’ve Paid Enough Attention to Architecture

 Defects are localized  Stable interfaces  Consistency  Developers can easily add new functionality  New functionality doesn’t “break” existing architecture  Few areas that developers avoid because they are too difficult to work in  Able to incrementally integrate new functionality

Sustaining Your Architecture

40 5/3/2017

Other Techniques for Improving Quality Steve McConnell http://kev.inburke.com/kevin/the-best-ways-to-find-bugs-in-your-code/

Average is 40% for any one technique!

Combining techniques gives you quality (> 90%)

Visibility Visibility

VALUES DRIVE PRACTICE CALL TO ACTION

Daily Practices Sustainable Development (CC) by muffinn on Flickr

41 5/3/2017

“We are uncovering easier ways of developing valuable products by doing it and helping others to do it. Through this work we have come to value:”

Relaxed Lazy Manifesto “We are uncovering easier ways of developing valuable products by doing it and helping others to do it. Through this work we have come to value:” Keeping slack over being busy all the time Small high quality software over large complex software

Doing only what is necessary over Harada Kiro exhaustively discovering all tasks Doing less to deliver the same over doing more to deliver less That is, while there is value in the items on the right, we value the items on the left more…

42 5/3/2017

Relaxed Principles of Lazy Manifesto

“We follow these principles when they don’t add work:”

Doing nothing is always an option. We seek to minimize the number of backlog items while keeping the value of the backlog. We believe to keep increasing velocity is not always good. We try to eliminate tasks that generate no value. We try to combine tasks to reduce latency and rework. We try to rearrange tasks to find problems early. We try to simplify all tasks as much as possible Harada Kiro We are not afraid of eliminating our own tasks / processes by continuously acquiring new skills / capabilities. We expand capabilities over increasing capacities. We only work hard to make our work easier and safer. We always look to get help while we provide help to others with minimum effort. We never try to add an unnecessary principle simply to match with the other manifesto :)

BeingBeing vs Agile Doing

Agile Mindset

43 5/3/2017

Dogmatic

Synonyms: bullheaded, dictative, doctrinaire, fanatical, intolerant

Antonyms: amenable, flexible, manageable

Pragmatic Synonyms: common, commonsense, logical, practical, rational, realistic, sensible

Antonyms: idealistic, unrealistic

Being Pragmatic

Rough No Planning Adaptive Lot’s of Plan (changing) Upfront No Design Planning Right Balance or Architecture of Design Lot of Design & Architecture Sometimes & Architecture called Agile Being Agile Traditional  or Waterfall   Balance Between…

44 5/3/2017

It is a Journey Commitment Follow-through Deliberate practices Slack Time to Improve Paying attention Continuous Learning

Takeaways

• Values drive practice • Deal with Technical Debt • Code Quality and Delivery Size • Continues Improvement and Learning • Many Practices can help with confidence • Call To Action (small steps you can take)

45 5/3/2017

Additional Resources Kaizen

• The Hillside Group (patterns community): Hillside.net • Being Agile at System Qualities workshop: – www.adaptiveobjectmodel.com/2015/04/ qa-to-aq-shifting-towards-agile-quality • Agile Myths: agilemyths.com • The Refactory (www.refactory.com) • Teams That Innovate (www.teamsthatinnovate.com) • Pragmatic TDD : refactory.com/training/test-driven-development http://adaptiveobjectmodel.com/2012/01/ what-is-pragmatic-tdd

Thanks!!!

Joe’s cool photo goes here!!! [email protected] www.joeyoder.com Twitter: @metayoda www.refactory.com