Architecture, structure, infrastructure October 24, 2014
3-tures: architecture, structure, infrastructure Philippe Kruchten Agile Vancouver October 26, 2014
Philippe Kruchten, Ph.D., P.Eng., CSDP
Professor of So ware Engineering NSERC Chair in Design Engineering Department of Electrical and Computer Engineering University of Bri sh Columbia Vancouver, BC Canada [email protected]
Founder and president Kruchten Engineering Services Ltd Vancouver, BC Canada [email protected]
Copyright © 2014 Philippe Kruchten 2
Philippe Kruchten 1 Architecture, structure, infrastructure October 24, 2014
Outline • Architecture • Architects • Agile and architecture (!?) • Scaling agile • Socio-technical congruence • Con nuous delivery • The advent of DevOps • Reducing fric on
3
Architecture
Yes, but what flavour?
• System architecture • Enterprise architecture • Business architecture • So ware architecture • Service-oriented architecture • Informa on architecture • Solu on architecture
4
Philippe Kruchten 2 Architecture, structure, infrastructure October 24, 2014
Business Architecture
• A subset of the enterprise architecture that defines an organiza on’s current and future state, including its strategy, its goals and objec ves, the internal environment through a process or func onal view, the external environment in which the business operates, and the stakeholders affected by the organiza on’s ac vi es. BABOK v2 2009
5
Enterprise Architecture
• Enterprise architecture is a descrip on of an organiza on’s business processes, IT so ware and hardware, people, opera ons and projects, and the rela onships between them.
Source BABOK v2 2009
6
Philippe Kruchten 3 Architecture, structure, infrastructure October 24, 2014
Solu on architecture
• System architecture
• So ware architecture
7
Two Main Cultures Solu on Architect Enterprise Architect • Authority • Advisor / Consultant • Technical Decision Maker • Building Bridges • Requirements è Architecture • Business / IT Alignment • Single “problem” • Governance over mul ple • “Building Design” “problems” • “City Planning”
• References: – SEI: ATAM, CBAM, QAW • References: – RUP: 4+1 Views – Zachman – Fowler: Architectus Oryzus – TOGAF, DODAF – IEEE 1471 – DYA, IAF, GEM, BASIC,… – IEEE 1471 Source Eltjo Poort
8
Philippe Kruchten 4 Architecture, structure, infrastructure October 24, 2014
So ware Architecture
System Enterprise Architecture Architecture
Source Eltjo Poort
9
Architectural Styles, Genres, Pa erns, Mechanisms,…
• Layered architecture • Client-server architecture • Service-oriented architecture
10
Philippe Kruchten 5 Architecture, structure, infrastructure October 24, 2014
So ware Architecture: A Defini on
“It’s the hard stuff.”
M.Fowler, cited by J. Highsmith 11
So ware Architecture: A Defini on
“It’s the hard stuff.” “It’s the stuff that will be hard to change”
M.Fowler, cited by J. Highsmith 12
Philippe Kruchten 6 Architecture, structure, infrastructure October 24, 2014
ISO/IEC 42010
Architecture: the fundamental concepts or proper es of a system in its environment embodied in its elements, their rela onships, and in the principles of its design and evolu on
13
So ware Architecture So ware architecture encompasses the set of significant decisions about • the organiza on of a so ware system, • the selec on of the structural elements and their interfaces by which the system is composed together with their behavior as specified in the collabora on among those elements, • the composi on of these elements into progressively larger subsystems,
Grady Booch, Philippe Kruchten, Rich Reitman, Kurt Bi ner; Ra onal, circa 1995 (derived from Mary Shaw) 14
Philippe Kruchten 7 Architecture, structure, infrastructure October 24, 2014
So ware Architecture (cont.) … • the architectural style that guides this organiza on, these elements and their interfaces, their collabora ons, and their composi on. • So ware architecture is not only concerned with structure and behavior, but also with usage, func onality, performance, resilience, reuse, comprehensibility, economic and technological constraints and tradeoffs, and aesthe cs.
15
Func ons of the so ware architect
Defini on of the architecture Delivery of the architecture • Architecture defini on • Ownership of the big picture • Technology selec on • Leadership • Architectural evalua on • Coaching and mentoring • Design, development and • Management of non Tes ng func onal requirements • Architecture collabora on • Quality assurance
Brown 2010
16
Philippe Kruchten 8 Architecture, structure, infrastructure October 24, 2014
Func ons of the so ware architect
Defini on of the architecture Delivery of the architecture • Architecture defini on • Ownership of the big picture • Technology selec on • Leadership • Architectural evalua on • Coaching and mentoring • Design, development and • Management of non Tes ng func onal requirements • Architecture collabora on • Quality assurance
Brown 2010
17
3 main boundaries
B. A.
Architect
Project Developers manager
18
Philippe Kruchten 9 Architecture, structure, infrastructure October 24, 2014
Three main cons tuencies Customers Domain experts Requirements eng. Product & marke ng managers Legislators
Analysts
Architect
Project Developers manager Top management Subcontractors Subcontractors management Technology vendors Airlines, unions, etc. 19
Perceived Tensions Agility- Architecture
Adapta on versus An cipa on
Highsmith 2000
20
Philippe Kruchten 10 Architecture, structure, infrastructure October 24, 2014
Scaling agile
Scope, team size, dura on • Larger system • Larger teams • Distributed teams • More frequent releases • Over longer meframes
21
Wrong Problem
• Can we scale up agile prac ces?
22
Philippe Kruchten 11 Architecture, structure, infrastructure October 24, 2014
Wrong Problem
• Can we scale up agile prac ces?
23
Problems
• Can the architecture of the product support mul ple waves of enhancements, to accommodate a constant flow of new needs? • Can the architecture evolve con nuously to support enhancements? • Does the architecture allow teams to organize the work so that they feel as if they were in the “agile sweet spot” and allow them to take advantage of agile prac ces? • Can the organiza on avoid the extra work generated by repe ve handover from a development team to an opera ons group?
24
Philippe Kruchten 12 Architecture, structure, infrastructure October 24, 2014
Problem (in other words)
1. Agile architecture – Flexible, evolvable architecture, over me 2. Agile architec ng – Can we “be agile” while crea ng, evolving an architecture
Note that (1) does not imply (2), nor vice versa
25
Scale needs architecture
• Work par on – decoupling • Conceptual integrity – Common vocabulary, etc… • Guide for release planning, configura on management • Address major “ili es” • Reuse, product line, por olio, etc.
26
Philippe Kruchten 13 Architecture, structure, infrastructure October 24, 2014
Architecture benefits from agility
• Itera ve development
• Small increments • Pace decisions • Validate early arch decisions with running testable code, and by building features on it
• Some arch elements may be stubbed
27
Architecture to the rescue
• Common vocabulary • Common culture The original metaphor as architecture from XP • Control dependencies: code, data, ming, requirements • Keep technical debt in check • Guide release planning and configura on management
28
Philippe Kruchten 14 Architecture, structure, infrastructure October 24, 2014
Early or late binding decisions?
• Upfront vs. emergent?
• Key ques ons: – How early is “upfront” – How much can you defer?
• Is architecture part of development? – Architecture conjecture (or architecture planning) /= architectural development
29
Early or late binding decisions? and the specter of technical debt
Source: Kruchten, Ozkaya, Nord., 2012
30
Philippe Kruchten 15 Architecture, structure, infrastructure October 24, 2014
Zipper Model
• From requirements derive: – Architectural requirements – Func onal requirements • Establish – Dependencies – Cost • Plan interleaving: – Func onal increments – Architectural increments
31
Weaving func onal and architectural chunks
32
Philippe Kruchten 16 Architecture, structure, infrastructure October 24, 2014
Benefits
• Gradual emergence of architecture – Deliberate, not accidental (“architecture owner”) • Valida on of architecture with actual func onality (not mere hypotheses) • Early enough to support development – Time pacing
• Not just BUFD • No YAGNI effect
33
Weaving func onal and architectural chunks
34
Philippe Kruchten 17 Architecture, structure, infrastructure October 24, 2014
A more complex story
Architecture of the System
Structure of the Produc on Development Organiza on Infrastructure
36
Philippe Kruchten 18 Architecture, structure, infrastructure October 24, 2014
3 tures
• The Architecture (A) of the system under design, development, or refinement, what we have called the tradi onal system or so ware architecture • The Structure (S) of the organiza on: teams, partners, subcontractors, and others • The Produc on infrastructure (P) used to develop and deploy the system, the last ac vity being especially important in contexts where the development and opera ons are combined and the system is deployed more or less con nuously
37
Three evolving structures
Architecture of the System
Structure of the Produc on Development Organiza on Infrastructure
38
Philippe Kruchten 19 Architecture, structure, infrastructure October 24, 2014
Socio-technical congruence
Architecture of the System
Socio-technical congruence
Structure of the Produc on Development Organiza on Infrastructure
39
Socio-technical congruence
• A measure of the alignment between technical rela onships (in the product) and the social rela onships (in the development organiza on)
• Conway’s law (1968): “...organiza ons which design systems ... are constrained to produce designs which are copies of the communica on structures of these organiza ons.”
40
Philippe Kruchten 20 Architecture, structure, infrastructure October 24, 2014
Architecture ßà Structure • If architecture (of the system) and the structure (of the organiza on) are at odds, the structure of the organiza on wins • Architecture and structure must co-evolve • if they don't, Conway's Law warns us that the organiza on form will trump intended designs that go "cross-grain" to the organiza on warp. • System architects (the “architects”) and business/ organiza on architects (the “managers”) should not work as if one has no impact on the other.
41
Three evolving structures
Architecture of the System
Socio-technical congruence
Structure of the Produc on Development Organiza on DevOps Infrastructure
42
Philippe Kruchten 21 Architecture, structure, infrastructure October 24, 2014
DevOps
43
DevOps
• DevOps is the prac ce of opera ons and development engineers par cipa ng together in the en re service lifecycle, from design through the development process to produc on support. h p://theagileadmin.com/what-is-devops/
• Kill the silos Patrick Debois h p://www.jedi.be/blog/2010/02/12/what-is-this-devops-thing-anyway/
44
Philippe Kruchten 22 Architecture, structure, infrastructure October 24, 2014
DevOps
• Test environment on development not representa ve of the deployment environment • Data conversion and transfer • Con nuity of opera ons • Reten on of personnel, of knowledge • Con nuous release
45
DevOps and Architecture
• Opera onal concerns considered early • Break down system in small independent services – Micro-service oriented architecture (?)
46
Philippe Kruchten 23 Architecture, structure, infrastructure October 24, 2014
Con nuous architecture to emable con nuous delivery 1. Architect Products – not Projects 2. Focus on Quality A ributes – not on Func onal Requirements 3. Delay design decisions un l they are absolutely necessary 4. Architect for Change – Leverage “The Power of Small” 5. Architect for Build, Test and Deploy 6. Model the organiza on of your teams a er the design of the system you are working on
Source: Pureur & Erder h p://pgppgp.wordpress.com/ 47
Tac cs
• An architectural tac c is a design op on for the architect • Each tac c is an hypothesis – To be validated by implementa on, prototyping, tes ng • Pa erns and frameworks package tac cs
48
Philippe Kruchten 24 Architecture, structure, infrastructure October 24, 2014
Example
• Need high availability • Possible tac c: ac ve redundancy • Does ac ve redundancy give me the adequate level of availability? • A pa ern that supports availability will ikly use some form of redundancy
49
Tac cs Catalog
• Availability • Interoperability • Modifiability • Performance • Security • Testability • Usability
50
Philippe Kruchten 25 Architecture, structure, infrastructure October 24, 2014
Deployability tac cs
• Parameteriza on • Self-monitoring • Self-ini ated version update
51
52
Philippe Kruchten 26 Architecture, structure, infrastructure October 24, 2014
Fric on
“There is s ll much fric on in the process of cra ing complex so ware; the goal of crea ng quality so ware in a repeatable and sustainable manner remains elusive to many organiza ons, especially those who are driven to develop in Internet me.”
53
Fric on
“Fric on: the resistance that one surface or object encounters when moving over another.”
In so ware development, fric on is the set of phenomena that limits or constraints our progress, therefore reduces our velocity (or produc vity).
Technical debt causes fric on.
54
Philippe Kruchten 27 Architecture, structure, infrastructure October 24, 2014
Fric on and Debt
Technical Debt Fric on Social Debt Reduced velocity Defects Delays …
55
Social debt
• Social debt is a state of a development project which is the result of the accumula on over me of decisions about the way the development team (or community) communicates, collaborates and coordinates.
Tamburri et al. 2013
56
Philippe Kruchten 28 Architecture, structure, infrastructure October 24, 2014
Social debt
• In other words, decisions about : – the organiza onal structure, – the process, – the governance, – the social interac ons, • or some elements inherited through the people: – their knowledge, personality, working style, etc.
Tamburri et al. 2013 57
Parallel Technical & Social Debt
Visible Invisible Posi ve New features Architectural, Added Structural Value func onality features Nega ve Defects Technical Value Debt
58
Philippe Kruchten 29 Architecture, structure, infrastructure October 24, 2014
Social debt
Visible Invisible Posi ve Community Community Features Structure Value Nega ve Community Social Defects Value Debt
Tamburri et al. 2013
59
To agilely architect an agile architecture • Architecture /= Big Design Up Front – Architecture is development • Design architecture incrementally … but not in isola on from the rest of the system • Re-pay architectural technical debt – As you go, or as needed to not hinder future evolu on • Align the organiza on to match the architecture – Not the opposite • And use all the agile prac ces you see fit
60
Philippe Kruchten 30 Architecture, structure, infrastructure October 24, 2014
Reduce fric on - Keep them aligned
Architecture of the System
Structure of the Produc on Development Organiza on Infrastructure
61
62
Philippe Kruchten 31 Architecture, structure, infrastructure October 24, 2014
References
• S. Bellomo, P. Kruchten, R.L. Nord, and I. Ozkaya, "How to agilely architect an agile architecture?," Cu er IT Journal, vol. 27, no. 2, 2014, pp. 12-17 • P. Kruchten, R. Nord, and I. Ozkaya, "Technical debt: from metaphor to theory and prac ce," IEEE So ware, vol. 29, no. 6, 2012, pp. 18-21. • R. Kazman and P. Kruchten, Design approaches for taming complexity, in IEEE Interna onal Systems Conference, Alain Beaulieu and George Foster (eds). 2012, IEEE, pp. 519-524. • P. Kruchten, "What do so ware architects really do?," Journal of Systems & So ware, vol. 81, no. 12, 2008, pp. 2413-2416. • C. Hofmeister, P. Kruchten, R. Nord, H. Obbink, A. Ran, and P. America, "A general model of so ware architecture design derived from five industrial approaches," Journal of Systems & So ware, vol. 80, 2007, pp. 106-126. DOI:10.1016/j.jss.2006.05.024
63
References (cont.)
• D. Tamburri, P. Lago, P. Kruchten, and H. van vliet, "What is social debt in so ware engineering?," presented at CHASE Workshop (part of ICSE), San Francisco, 2013, IEEE Computer Society, pp.93-96DOI 10.1109/CHASE. 2013.6614739 • Ruth Malan, A trace in the sand, Blog at h p://www.ruthmalan.com/ journal/journalcurrent.htm • Patrick Debois, DevOps blog at, h p://www.jedi.be/blog/ • Pierre Pureur, Murat Erder, Con nuous architecture Blog at h p:// pgppgp.wordpress.com/ • M. E. Conway, "How do commi ees invent?," Datama on, vol. 14(4), pp. 28-31, 1968.
64
Philippe Kruchten 32