Dr. Ralf S. Engelschall Architecture Fundamentals AF Goal Focus 00.0
Step 1: Your Insight (Believe) Concepts Methods Technologies ReproductionUnauthorized Prohibited. Ralf All Rights S. Engelschall
understand München (TUM) Universität Science lecture contexts in Computer only. reproduction for Technische Licensed to form to be handy in practice. task of trainee in practice apply
AN ARCHITECT Step 3: Your Application 1. THINKING LIKE 2. BEING GOOD AT Concepts can be applied in practice both CONCEPTUALIZATION proactive/constructive and reactive/analytical.
Scope Type Focus Content
Computer Literature Industry Theory knows about knows about Science more things more things Diagrams Statements (written) Abstraction Generalization Software (Conceptual) Trainer Architecture Rationales Model Theory Practice (verbal) Engineering Fundamentals
Software & Examples Trainer Systems (verbal) Practice Architecture the most Instantiation relevant concepts Specialization Software Engineering Disciplines AF ANALYTICAL CONSTRUCTIVE STEERING 01.1 REQ Requirements ENV Environment RES Resources
Intellectual Content: Intellectual Version 1.0.7 (2010-07-21), Authored 2006-2010 by Dr. S. Engelschall, inspired Ralf Uni ed Process by Rational (RUP) Graphical Illustration: Graphical Illustration: Version 1.0.9 (2019-06-28), Copyright © 2007-2019 Dr. S. Engelschall
Systematically and structurally elicit, Design all external application Provision all human resources, organize, document and agree on architecture aspects to allow seamless environments and toolings Who? scope, functionality and constraints integration into its environment to drive the project Why? How? Where?
ANA Analysis DES Design PRJ Project PLANNING
Licensed to Technische for reproduction in Computer only. lecture contexts Science Universität (TUM) München Systematically and structurally analyze Design all internal application Plan all time, costs and scope and document the functionality into a architecture aspects to allow an e cient aspects to allow an e ective and Who? binding and complete speci cation and e ective implementation e cient execution of project What? How? When?
DBG Debugging IMP Implementation EXE Execution
Locate the origin of a functional Craft all application code and data Continuously ensure that at any time all or non-functional problem within according to the requirements, analysis project members know about and can the application code or data and design speci cations perform their assigned work packages What?
TST Test ASM Assembly CTL Controlling
Test application at all its structural Automate hierarchical building of all Continuously compare planned and layers against de ned functional code and data artifacts and automate actual status of time, costs and scope and non-functional test cases deployment & operations of application and adjust project accordingly
MSM Measurement DOC Documentation DLV Delivery EXECUTIONAL Measure static and dynamic aspects Craft documentation for the target Plan and control the continuous of application and project and provide audiences manager, developer, user change and delivery process of the resulting key performance indicators and administrator application across releases
Project = Disciplines x INS Inspection TRN Training CFG Con guration Process
Review and audit all aspects of the Create and perform presentations Consistently identify and version all software engineering process of the and workshops based on the revisions of all project artifacts across project and judge current quality application and its documentation all storage types and their branches
secondary / support disciplines primary / focus / ”king” disciplines domain Analysis domain Design domain Development domain Documentation domain Analytics domain Con guration domain Management Software Deployment AF AMA Bare Amalgamation CON Container Image 13.1 Manually deploy all applications into a single, Bundle an application with its stripped-down shared, and unmanaged flesystem location. Application OS dependencies and run-time environment Application Applic ReproductionUnauthorized Prohibited. Ralf All Rights S. Engelschall
Pro: simple deployment Amalgamation Pro: independent, simple deployment Container Run-Time Con: incompatibilities, hard uninstallation Con: less variant possibilities, no depend.
HEA Unmanaged Heap STK Package/Container Stack München (TUM) Universität Science lecture contexts in Computer only. reproduction for Technische Licensed to Manually deploy all applications into Establish an application out of multiple Application multiple, distinct, and unmanaged flesystem Managed Packages. Examples: OpenPKG locations. Dependencies are resolved Applic Applic Applic Stack, Docker Compose. Package/ Package/ manually. Examples: macOS *.app, OpenPKG ation ation ation Container Container LSYNC. Heap Heap Heap Pro: independent, fexible Stack Pro: simple deployment, easy uninstallation Con: overhead Con: no repair mechanism Package/Container Manager
INS Managed Heap IMG Virtual Machine Image
Let individual installers deploy applications Bundle an application with its full OS into multiple, distinct, and managed dependencies and run-time environment Application flesystem locations. Dependencies are Applic Applic Applic into a virtual machine image and deploy and OS (guest) manually resolved or bundled. Examples: ation ation ation execute this as a virtual machine. Examples: Virtual Machine Image macOS *.pkg, Window MSI, InnoSetup. Heap Heap Heap VirtualBox, VMWare, HyperV, Parallels, QEMU. Pro: easy uninstallation, repairable Installer Installer Installer Pro: all-in-one, independent Virtual Machine Con: requires installer, diversity, no depend. Con: overhead, sealed, unfexible
PKG Managed Package APP Solution Appliance
Let a central package manager deploy all Bundle an application with its full OS applications into a single, shared, and Applic Applic Applic dependencies, run-time environment and Software Application managed flesystem location. Dependencies ation ation underlying hardware. Examples: AVM Fritz! ation OS are automatically resolved. Examples: DPKG/ Box, SAP HANA. APT, OpenPKG RPM, FreeBSD pkg, MacPorts. Package Package Package Pro: all-in-one, independent Hardware Pro: easy uninstall., repairable, dependencies Package Manager Con: expensive, sealed, unfexible Solution Appliance Con: requires package manager Cloud Computing Resources AF 13.2 ON-DEMAND MULTI-TENANCY Public Cloud Provider-Hosted Unauthorized ReproductionUnauthorized Prohibited. Ralf All Rights S. Engelschall
SELF-SERVICE MEASURED SERVICE Community Cloud On-Premises outside-inside relationship outside-inside outside-inside relationship outside-inside logically / chronologically related logically / chronologically related Licensed to Technische Universität München (TUM) Universität Science lecture contexts in Computer only. reproduction for Technische Licensed to AUTOMATABLE ELASTICITY Private Cloud
NETWORK ACCESS PAY-PER-USE Hybrid Cloud
Cloud Characteristics Cloud Scope Cloud Location
Service Application Service Mesh Service Events technical layers Application Runtime aka “Serverless Tools & Libraries Computing” Container Runtime Operating System SaaS FaaS PaaS CaaS IaaS Computer Software Function Platform Container Infrastructure Network as a Service as a Service as a Service as a Service as a Service
Duration: Duration: Duration: User Session Function Call Continuously
Cloud Approaches Cloud-Native Architecture AF Systems of Engagement Users Systems of Record Developers 13.3 Standard Cluster Setup (5+1+N Machines):
M G I T P C D ReproductionUnauthorized Prohibited. Ralf All Rights S. Engelschall
Management [M] Gateway [G] Integration [I] Delivery [D] M G I T P C Name Caching API Authentication Enterprise Version Resolution Proxy Gateway & Authorization Integration Control M G I T P C (BIND) (Squid) (GoBetween+Traefk) (Dex + SPIRE + Opa) (WSO2 ESB) (Gitea + Git) Licensed to Technische Universität München (TUM) Universität Science lecture contexts in Computer only. reproduction for Technische Licensed to Service Continuous M G I T P C Registry Delivery (Consul) (Drone) M G I T P C Authentication Process Microservice Microservice Microservice Artifact & Authorization (Frontend) (Frontend) (Cross) Manager (SPIRE Agent) Repository 1 2 3 … N (Nomad + Dkron) (Nexus)
Service Microservice Microservice Microservice Registry (Core) (Core) (Cross) Partitioned Cluster Setup (5x2+1+N Machines): Communication [C] (Consul) Tracing [T]
Message Metric M G I T P C D Queue Process Store (NATS + NATS Microservice Microservice Microservice (InfuxDB-Relay + Streaming Server) Manager (Backend) (Backend) (Cross) InfuxDB) (Nomad) M G I T P C Session Event Store Store M G I T P C (Corvus + Redis) (Jaeger)
M G I T P C
Credential Entity Graph Full-Text Blockchain BLOB File-Tree M G I T P C Store Store Store Store Store Store Store (Vault + Notary) (CockroachDB) (Neo4J) (ElasticSearch) (Tendermint) (Minio) (GlusterFS + XFS)
Persistence [P] 1 2 3 … N
Common Group Platform Application (HA) Major Approach Idea: Major Design Criterias: Microservice Group Foreign Application (non-HA) With Cloud-Native Architecture one maximizes the 1. Targets DevOps approach. leverage of PaaS-like, high-available, and scalable Cloud 2. Targets Continuous Delivery process. (Virtual) Machine Microservice Application (HA) services at the level of Software- and Systems-Architecture 3. Targets Microservice Architecture. for a whole set of applications. 4. Targets Container Image deployment. 5. Targets Service Mesh communication. 6. Targets Server Cluster setup. 7. Provides High-Availability of Service Platform 8. Provides High-Availability of Application Microservices. 9. Provides Scalability of Application Microservices. Version Control Architecture AF M.N.K branch
M.N branch 14.1 M.N.K.K
M.N.K.1 ReproductionUnauthorized Prohibited. © 2008-2012 Ralf All Rights 1.2.1 (2012-09-19), Copyright S. Engelschall
M.N.K[.0] M.N.0.K Licensed to Technische Universität München (TUM) Universität Science lecture contexts in Computer only. reproduction for Technische Licensed to M.N.0.1 M.N.1 feature bugfx development maintenance merging merges M.NrcK M.NbK-S- M.N.0 YYYYMMDD M.Nrc1 M.Na1 M.NaK M.Nb1 M.NbK trunk (M branch)
M.NbK-D
upstream vendor tracking branch
development alpha beta release release maintenance phase phase phase candidate phase phase phase product diversity product focus product stability product freeze access unlimited access unlimited access controlled access restricted VCS commit activity
time Assembly Process Architecture AF
1 2 mode Development: (Source) Software Developer BUILD Artifact DS Distribution AR Repository 1 2 3 Site Release Engineer PATTERNS lossless lossy Build 14.2
Examples: 1 2 3 generation editing from scratch Standard Process Continuous Integration System making random data
Examples: 4 5 6 duplication downloading Continuous Deployment System uploading source:upload source:download artifact:download artifact:upload Examples: Examples: fragmentation unpacking archive object extraction Unauthorized ReproductionUnauthorized Prohibited. 1.3. Version Graphical Illustration: 1.3. Version Intellectual Content:
Examples: Examples: format changing source compilation transformation recoding parsing operation
Examples: Examples: integration packing archive linking objects A A B A B Examples: Archived elimination cleaning up
stage3:test source:track source:pack source:unpack stage2:test stage4:test stage5:test link:test track pack unpack compile:test bundle:test package:test test 1 1 (2019-06- stage0[:build] stage1[:build] stage3[:build] stage4[:build] stage5[:build] (2019-06- source:checkout stage2[:build] bootstrap setup link bundle package:pack
checkout compile München (TUM) Universität Science lecture contexts in Computer only. reproduction for Technische Licensed to generate confgure build all package 2 2 7), Authored 2009-2019 by 2009-2019 by 7), Authored VCS © 2009-2019 7), Copyright F B S C L B P package:upload stage0:clean stage1:clean stage2:clean Version source:checkin stage3:clean stage4:clean stage5:clean Fresh bootstrap:clean Bootstrapped setup:clean Setup compile:clean Compiled Linked Bundled Packaged Control checkin link:clean bundle:clean package:clean System realclean distclean clean
stage2:lint stage0:lint stage1:lint stage3:lint stage4:lint stage5:lint source:lint compile:lint package:meta-write bootstrap:lint setup:lint link:lint bundle:lint package:lint lint Dr. Dr. Dr. Dr. Ralf All Rights S. Engelschall
Packager (Binary) 1 2 3 1. edit 2. compile source A Deployer 4 5 3. start editing 4. stop DS Operator 6 (Binary) Distribution DEVELOPER LOOP Site SC1: Developer Short-Circuit Short-Circuit SC1: Developer Transition SC2: Developer Short-Circuit Short-Circuit SC2: Developer Transition Short-Circuit SC3: Developer Transition Short-Circuit SC4: Developer Transition Subversion, Git, Mercurial, Maven, Gradle, Make, Maven, Gradle, Ant/Ivy, NSIS, InnoSetup, Tar, InfoZip, B 1 PREPARATION NPM, YARN, UPD, Autoconf, Automake, CMake, OpenPKG RPM 2 BUILDING NPM, YARN, Docker Build, OpenPKG RPM 3 DISTRIBUTION NPM, Maven, Gradle, Docker, OpenPKG Build
Operations: Deployment runtime:backup runtime:restore install:upgrade install:update package:meta-read Standard Process backup restore upgrade update
ROUND TRIP: every forward- runtime:test runtime:start C / L / B install:install test start install only activity should have a Compiled/Linked corresponding backward activity CONSISTENCY: directly triggering an activity causes runtime:status runtime:reconfgure install:confgure package:unpack package:download intermediate activities (between status R reconfgure confgure I B P old and new state) to be implicitly triggered frst Bundled Packaged Running Installed DEVELOPMENT: intentionally, there are short-circuit transitions runtime:reload runtime:stop install:uninstall GLOBAL RULES between states reload stop uninstall
ACTIVITY NAMING: each activity has at least an ofcial runtime:scrub install:migrate install:validate install:repair name and zero or more calling aliases scrub migrate validate repair
OpenRC, SystemD, Init, SupervisorD, APT, dpkg, RPM, APT, dpkg, RPM, 6 RUNTIME OpenPKG RC, Docker 5 INSTALLATION OpenPKG RPM, Docker 4 RESOLUTION OpenPKG Build, Docker
X State Transition Process Activity External State/Resource XXX State Transition (short-circuit) (semi-automated or automated) X Process State State Transition (external) Process Activity XXX (manually or semi-automated) Software Release Management AF Evolution Stages (ES) Version Number (VN) Maturity Levels (ML) Points-In-Time (PiT) 15.2 WF Wireframe WS Walking Skeleton M Major Version A Alpha R Release DEV Development Distraction-free low-fdelity illustration Realization of all technical, fundamental Major Version of solution. Usually Early version of the solution with Final version of the solution with Arbitrary permanent points-in-time of the solution and its base features, aspects of the solution, ensuring the bumped on major technical or domain- incomplete and unstable functionalities complete and stable functionalities, during development. This is the default displaying its pure structure and core domain specifc aspects can be realized specifc changes only. A bump resets to get feedback on product. Usually available for production use. Usually tag for the source code. Intended for no ReproductionUnauthorized Prohibited. Ralf All Rights S. Engelschall
Minimum-Viable PT Prototype MVP Product N Minor Version B Beta RU Release Update SNP Snapshot High-fdelity mostly interactive sample, Early version of solution with just Minor Version of the solution within the Early version of the solution with Updated version of the solution with Distinct temporary point-in-time for a mockup, model or simulation of the enough functionality to enable full turn Major Version. Usually bumped on new complete but still unstable complete and stable functionalities, release of the current version without a solution and its base features, show- of Build-Measure-Learn loop with features. A bump resets the Revision, functionalities to stabilise product. available for production use. Usually version increase. Intended for limited casing its structure and functionality. minimal amount of efort and time. too. Usually tagged as “M.NbR” (R > 0). tagged as “M.N.R” (R > 0). availability releases.
PoC Proof of Concept FP Full Product R Revision RC Release Candidate REL Release München (TUM) Universität Science lecture contexts in Computer only. reproduction for Technische Licensed to Pure realization of most-risky aspects of Final version of the solution with all The Revision of the Maturity Level Mature version of solution with Distinct temporary point-in-time for a the solution, proofng their feasibilities. intended functionality and targeting within Major and Minor Version. Usually complete and stable functionalities to release of the current version with a Might still be based on a diferent the mainstream market. bumped for every A/B/RC/R level and catch last-minute problems. Usually version increase. Intended for early and technology than WS, MVP and FP. for bug and security fxes on Updates. tagged as “M.NrcR” (R > 0) around RTM. general availability releases.
Product Editions (PE) Availability Scopes (AS) Distribution Channels (DC)
Community Standard No Early Bleed Stable CE Edition STD Edition XA Availability EA Availability BLEED Channel STABLE Channel Edition of the solution for the Open Edition of the solution with just the No public availability of solution at all. Early public availability of solution for Distribution channel for all daily Distribution channel for all quarterly Source Community. Contains just the standard functionality and regular The scope for all Development and early market. Usually for Beta or Release snapshots (“YYYY.MM.DD”) with releases (“YYYY.QN”) with experimental base functionality and has limited support. sometimes Snapshot point-in-times. Candidate levels or for Release and experimental features turned on. features turned of. Intended for fast volunteering support. initial Release Update levels. Intended for testing purposes only. mainstream market and production use.
Enterprise Professional Limited General Edge Solid (LTS) EE Edition PRO Edition LA Availability GA Availability EDGE Channel SOLID Channel Edition of the solution for the Enterprise Edition of the solution with both the Limited public availability of solution. Late public availability of solution for Distribution channel for all monthly Distribution channel for all (half-)year market. Contains the base and standard and extra functionalities and Usually for releases after the End-of- mainstream market. Usually for Release releases (“YYYY.MM”) with experimental releases (“YYYY[.N]”) with experimental additional functionality and has full extended support. Life-Announcement (EOLA) or for and sometimes just for Release Update features turned on. Intended for early features turned of. Intended for slow commercial support. releases with specifc customer features. levels. market or testing purposes. mainstream market and production use.
Version Control (VC) Version Scheme (VS) Product Life-Cycle (PLC)
SOLID
Unit Unauthorized ReproductionUnauthorized Prohibited. Ralf All Rights S. Engelschall
Integration STG DRY PRD FOV INT Integration IT Testing München (TUM) Universität Science lecture contexts in Computer only. reproduction for Technische Licensed to (Potentially destructive) Whitebox testing the outer Environment for integrating all interplay of individual units artifacts of the system, usually (components) against the located centrally to the developers technical design of the solution. and driven automatically. Specialization: Smoke Testing
System SBX Sandbox TST Test ST Testing Destructive environment mirroring Environment for testing the Blackbox testing the system as a the Test environment for system as a whole in a production- whole against the functional/ demonstration and resembling context. domain and non-functional/ experimentation purposes, quality requirements of the including trainings and showcases. solution.
Operations Environments
(User) Acceptance DRY Dry-Run STG Staging UAT Testing Destructive environment mirroring Environment for staging the Blackbox testing and formal the Staging environment for system as a whole in a production- approval of the system as a whole experimentation purposes, equal context in order to deploy it against the end-to-end including operating system and to the Production environment functionality and user experience major dependency upgrades. subsequently. requirements. Environment Mirroring Operation FOV Failover PRD Production OT Testing Destructive environment mirroring Environment for running the Blackbox testing the system as a the Production environment for system as a whole in production. whole against the availability and failover situation, including proper operation of the intended disaster recovery situations. services. Specialization: Service Monitoring Quality Promise, Bugfx Efort
Replicated Environments Original Environments DevOps Toolchain AF Development Environment Integration Environment Test Environment Staging Environment Production Environment 15.4 ED Actor-Store Pattern DevOps Pipeline Pattern Development Operations Unauthorized ReproductionUnauthorized Prohibited. Ralf All Rights S. Engelschall
SC DS SC DS München (TUM) Universität Science lecture contexts in Computer only. reproduction for Technische Licensed to BS TS BS TS CI Rules Rule 1: Actors are linked by either Store-connected artifact fows or plain triggers Rule 2: The confguration of all Actors and Stores have to be kept as small as possible by just IC IC performing orchestration and retrieving the actual commands via external scripts. Rule 3: On each Version Control (VC) commit, the application should have to be automatically redeployed as an updated version on the Run-Times (RT). BX BX Rule 4: Development and Operations can be split via two synchronised Artifact Repositories Administrator (AR), because of network topology constraints, or act on a single common one. Rule 5: Deployment on Staging Environment can be both automatically and manually. TC DS IC TC DS IC TS TS DC AR AR
TC DS TC TC DS TC DS TC DS TC DS TS TS CD TS CD TS CD IA
DX DX DX DX DX
TC TC TC TC TC Actors Stores Flows Artifacts
ED: Editor / IDE WC: Working Copy Artifact Flow SC: Source Component RT CI: Continuous Integration RT VC: Version Control RT Event Flow RT IC: Interm. Component RT BX: Build Execution DC: Distribution Copy Control Flow TC: Target Component CD: Continuous Deployment AR: Artifact Repository BS: Build Script DX: Deployment Execution RT: Run-Time DS: Deployment Script IA: Infrastructure Automat. TS: Test Script
Developers Users Testers
Development Operations