"Right-Sized Architecture: Integrity for Emerging Designs"

AW7 Concurrent Session 11/7/2012 2:15 PM

"Right-sized Architecture: Integrity for Emerging Designs"

Presented by:

Ken Kubo Northrop Grumman Corporation

Brought to you by:

340 Corporate Way, Suite 300, Orange Park, FL 32073 888‐268‐8770 ∙ 904‐278‐0524 ∙ [email protected] ∙ www.sqe.com

Ken Kubo Northrop Grumman Electronic Systems

Ken Kubo is director of software engineering with twenty years of service at Northrop Grumman Electronic Systems, Intelligence, Surveillance, Reconnaissance, and Targeting Systems Division, Azusa campus. His work has focused on the development of satellite ground systems, building the bigger picture from individual bits of data. Information radiators are Ken’s natural focus in agile development. A certified Lean-Agile ScrumMaster and Certified ICAgile Professional, he developed and teaches a Lean-Agile Development overview course for NGES. Ken firmly believes that lean-agile and government acquisition processes are not completely incompatible.

Right-Sized Architecture

Integrity for Emerging Designs AW7 - 7 November 2012 – 2:15 PM

Ken Kubo, James Yoshimori, Jason Liu

Agenda

• Software Engineering and Lean-Agile

• Right-Sized Architecture

• Collaborative Design

• Extending the Metaphor

• Retrospective Acknowledgements

• The team gratefully acknowledges the support and interest of the Sensor Exploitation Systems/SAIG business area leadership.

But First…A Word From Our Sponsor

• Northrop Grumman Corporation (NYSE: NOC) is a leading global security company providing innovative systems, products and solutions in aerospace, electronics, information systems, and technical services to government and commercial customers worldwide. The company has over 120,000 employees in all 50 states and 25 countries around the world.

• Our core competencies are aligned with the current and future needs of our customers and address emerging global security challenges in key areas, such as unmanned systems, cyber-security, C4ISR, and logistics that are critical to the defense of the nation and its allies.

• Our Electronic Syy,g,stems sector is a leader in airborne radar, navigation, electronic countermeasures, precision weapons, airspace management, space payloads, marine and naval systems, communications, biodefense, and government systems. The Need for Agility

Increase in Software in DoD Systems

Software Content of Sample Major DoD Weapon Systems 1960 - 2020

18 FCS 16 F-35 Aircraft 14 and Ground 12 10 DDX 8 ESLOC in Millions 6 Virginia SSN SBIRS F-22 4 Patriot Aegis System PAC-3 2 Polaris A3 ACS 0 1960 1970 1980 1990 2000 2010 2020 Sea Systems Missiles/Space Ground Systems Aircraft Sources: CARD Data, SEI, CSIS Analysis

The Need for Agility

• Defense Science Board Recommends More agile Acquisition Process (March 2009) “A new acqu i si ti on pr ocess f or inf orm ati on • National Defense Authorizationtechnology Act shouldfor Fiscal be Yeardeveloped— 2010 (H.R.2647)modeled on (Became Public Law No: 111-84 October 2009) successful commercial practices, for the rapid • AFEI Task Force 804 Reportacquisition – June and 2010 continuous upgrade and Recommended (Amongimprovement Other Things): of IT capabilities. The process “Institute continuous, iterative, development, test, and certificationshould processes be agile that and drive geared the to delivering commercial IT statemeaningful of the art increments to deliver more of capability in trusted, standard, off-the-shelf building blocks” approximately 18 months or less—increments that are prioritized based on need and technical readiness.”

6 Software Engineering (and other oxymorons)

• Definitions: – Software: a set of instructions which define and enable the operation of a computer syypstem to perform a desired task – Engineering: the discipline and profession of applying technical, scientific, and mathematical knowledge to practical problems

• Historical endpoints – 1968 NATO Software Engineering Conference – Software Engineering Body of Knowledge (SWEBOK) (IEEE, 2004)

Software Engineering Practices

• Systemic view and development lifecycle (requirements analysis, design, development, testing, deployment, maintenance)

• Risk analysis

• Best practices – Lessons learned – Design patterns

• Certification – Certified Software Development Associate/Professional (CSDA/P) – PMI-ACP, ICAgile Certified Agile Professional, role certification etc.

8 The Agile Transition

Predictive

Develop FS A

RA HLD DD Develop FS B I&T V&V PDS Develop FS C FS A,B,C

Adaptive

RA HLD … PDS

FS A FS B FS D FS C

Agile Execution

Analysis “Desirement Analysis” with iteration planning AAhittrchitecture and backlog (work queue) grooming. Also used for risk identification/mitigation.

Design Traditional engineering activities all still occur, Development but note that they (mostly) happen collaboratively, not sequentially.

I&TI & T

10 When Engineering Goes Bad…

Traditional predictive practices and “ilities” can commit too early

• Lock-down architecture at the beginning of the project (Big Design Up Front)

• Only discuss design with customer prior to development

• Design for all possible customer requests, enhancements, and expandability

Wide focus can be uninformed 11

When Agile Goes Bad…

Agile methodology focus can incur technical debt

• Agile methodology impacts – Iteration focus can emphasize coding activity – de-emphasizing others – “Just in time” design can devalue architecture

• Technical debt – Neglecting the “big picture” – Decoupling architecture – Losing sight of system design integrity

Tight focus can prove short-sighted 12 Agile as a Philosophy

• Execute with lean agility – Avoid waste – Creating shared knowledge > creating documentation

• Leverage social interaction – Use a common “information radiator” • Artifacts and processes are open to view • Artifacts are easily accessible • Single source creates common understanding – Build a social environment for development • Encourage developers to work closely with one another • Strengthen trust within the community – Establish common understanding of artifacts being constructed

Shared understanding drives efficiency 13

Implementing Agile Architecture

•Avoid waste –just enough (George Fairbanks) “You‘ll be smarter later”

• The Zen of “just enough” – Avoid big design up front (BDUF) – Decide on the vision for your framework – Pick your focal points (risk and ignorance) – Make design a convenient part of the workflow – Emphasize creating knowledge, not creating diagrams • Encourage an open team dynamic so team members can say when they have enough information to code – Prefer the collaboration dynamic to automation tools – Don’t force documentation of design sessions – Be disciplined but not rigid

Just Enough Architecture A Little Before Its Time 14 Roles and Responsibilities

• Initial Design Objective: identify interfaces, processing partitions/components, general system sketch, build the framework for the developers – Architect: maintains big picture, helps identify and define interfaces, keeps focus of the meeting, controls documentation – Product Owner: clarifies objectives and done-ness, fills in unstated (“wasn’t that obvious?”) requirements – Technical Lead(s): identify where development activity has to occur, help define interfaces

• Delta Design Objective: sanity check and impact analysis (not the search for the perfect design) – Developer(s): present (proposed) solution, highlight any changes/detailing to architecture – Architect: helps perform impact analysis, verifies design integrity, controls documentation – Technical Lead(s): help perform impact analysis

Eyes on the Big Picture

• Apply Agile philosophy – Understanding > documentation

•How? – Using my Agile eyes …

• Inspiration – CRC: class, responsibilities & collaborators (Ward Cunningham) • Informal grouping of index cards (4”x6”) • Simple, easily remembered rules • Visual and collaborative

16 Color Map Affinity Diagrams

• Use colored index cards and push-pins – advanced developers may use self-adhesive notes

• Assign colors to the various types of high level components you may have (more details to follow)

• Find sufficient vertical space to place the index cards

• Have fun ☺

Example

• Project: Support processing, archive, and display of satellite wideband data, substantial legacy code

• Team size: 5 members

• Platform: SGI and Linux

• Language: C/C++ and Java

18 Issues with Database to Display Interfaces

• Large amounts of data being transferred to displays on network – Intermingled with smaller data packets

• Noticing network loading “uncomfortably” increased for single source on a GigE network – Anticipating load to increase by 4 times current loading in a year – Load may increase by more than 18 times in 5 years

• Current display has noticeable lag in processing and rendering – Display is just one executable so performance degradation is not unexpected

New capabilities have outgrown old paradigm 19

Just Enough Architecture

• Risk driven

• Focus on data stores and display

• Decouple display – Do not directly attach to principal data stores – Get away from single executable for display – Create an infrastructure that supports “drop-in” views

• Find a smarter way to manage the data and their transmission over the network

• Pay more attention to modularity and scalability qualities

Prioritize what to rearchitect/refactor 20 Color Assignments

• Specific colors do not matter just be • Edge components consistent that perform I/O

• 4-5 is usually all I • Processing need components • Persistent • Coincidentally a component stack of colored index cards have • Controller 5l5 colors component • User Interface

High Level Tracking Architecture

Signal Event Data Ingest Tracking Processing Detection

Pipeline

Wideband Track Points Events

Display Simple approach has issues

22 Current Display Architecture

Track Point

View

Data Events Service View

View View

Wide- band

Initial Update to Display Architecture

Track Track Points Point Service

View View

Events Events Service View

View Wide- Wide- band band Service

24 Updated Display Architecture

Blade Workstation

Track Track Points Points Service View View

Events Events Service View

View Wide- Wide- band band Service We use blue painters tape to identify boundaries 25

Updated Display Architecture

Blade Workstation

Track Track Track Points Points Point Service Controller View View Events Events Events Service Controller View

View Wide- Wide- Wide- band band band Service Controller

26 Display Architecture

Blade Workstation

Track Track Track Points Points View Point Service Controller View

View Events Events Events Service Controller

Exec

Wide- View Wide- Wide- band Setup band band Controller Service View

Display Architecture

Blade Workstation

Track Track Track Points Point Points View App Service Controller View App

Events Events View App Events Cache Service Controller

Wide- Exec band Wide- Wide- Controller View App Setup band band Service View App

28 Display Architecture

Blade Workstation

Track Track Track Point Points Points View App Service Controller View App View App

Events Events Events Service Controller View App View App Wide- band Track Wide- Wide- Controller Events Point band band Service Exec Cache Setup Wide- band

Display Architecture

Blade Workstation

Track PitPoints View A pp Track Track Points Controller Points Service View App Events Controller View App

Wideband Event Events Controller View App Service View App Exec

Wide- Wideband band Service Track Config Wide- Event Points File band 30 And Now For Something Different

• Story: As a mission operations monitor, I need an “always on” passive display that shows wideband data from the satellite point of view overlaid on map data to provide quick indication that data is flowing and line of sight is good.

• Approach: Add a new view application which can be standalone or “boxed” with the others which just provides a wideband data display.

Display Architecture

Blade Workstation

Track PitPoints View A pp Track Track Points Controller Points Service View App Events Controller View App

Wideband Event Events Controller View App Service View App Exec

View App Wide- Wideband band Service Track Config Wide- Event Points File band 32 Information Radiator

250 16

200 14

150 12

10 Ideal 100 ETC 8 Committed 50 6 Completed

0 4 109876543210 2

0 109876543210

• Can be physical or virtual

• Must be current

Information Radiator

• Iteration goal/vision

• Current iteration task/story status (in play and backlogged), iteration metrics

• Schedule (IMS, Release Plan, milestone calendar)

• Product vision, roadmap, Release Backlog

• Project metrics (velocity, defects found in I&T, bug rates, customer feedback)

• High-level architecture/system overview

• Process/workflow, team rules and standards

• Team member vacation/leave calendar

• Recognition/appreciation

Build shared knowledge and team direction Extending the Metaphor

• Team-based metaphors (and better note stock)

• Visual design patterns

• Component abstraction

• UML, SysML, MBE, … tools where value-added

• For legacy systems, practical default may be to adopt legacy standards

Team evolves “design language” 35

The Working System

• “My code works” is only a start; each developer must own the ppyerformance of the system as a whole

• “Responding to Change > Following a Plan” does not mean “Don’t Plan”

36 Retrospective

• Lean-Agile and good engineering practices are not opposed!

• Visual design methodologies can strengthen role of architectural design in Agile (and predictive!) methodologies

• Be creative (and Agile) – Focus on goals • Build shared system understanding • Incorporate design into workflow • Facilitate evolution of architecture over project life • Reduce integration rework – Right-sized = Do the minimum – Create a variant or use another technique – find what works better for your team

• Always keep looking for ways to do better

Keep an Agile eye on design 37

Useful References

• Scott L. Bain, Emergent Design: The Evolutionary Nature of Professional Software Development (Addison-Wesley, 2008)

• Kent Beck, Ward Cunningham, “A Laboratory for Teaching Object- Oriented Thinking” (OOPSLA 1989)

• George Fairbanks, Just Enough Software Architecture: A Risk-Driven Approach (Marshall & Brainerd, 2010)

38 Abstract

Lean-Agile development methodologies provide increased flexibility to deliver a more rapid return on customer investment in the form of working software. In Agile projects, system architecture ideally emerges over the course of development. However, if teams primarily focus on independent user stories, they risk losing sight of the Big Picture – the product’s vision and the integrity of well-thought-out architecture. This presentation shares techniques to improve the chances that a product’s design will emerge into a cohesive and coherent architecture that will support customer needs now and in the future. Incorporating contextual design principles and simple, visual techniques as part of “A Little Before Its Time (ALBeIT) Design” maintains a focus on architectural integrity. You can adopt these practices into your Agile workflow to maintain a shared team understanding of your product’s vision and the system’s emerging design.

40 Author Biography

• Ken Kubo is Director of Software Engineering at Northrop Grumman Electronic Systems, Intelligence, Surveillance, Reconnaissance, and Targeting Systems Division, Azusa campus with twenty years of service with NGES. His development work has focused on the development of satellite ground systems, building the bigger picture from individual bits of data. Information radiators are thus his natural focus in Agile development. He holds an MS in Management Science from Cal State Fullerton and a BS in Engineering/English from Harvey Mudd College. He is also a certified Lean-Agile Scrum Master and a Certified ICAgile Professional.

• James Yoshimori has over 30 years of experience in software development. His work has ranged from small embedded signal processing systems to very large radar imaging and infrared processing systems. He has served in various roles from software engineer to architect to program management. He currently serves as architect, team lead and Agile process maven for the Situational Awareness and Global Exploitation (SAGE) project. He holds an MS in Information and Computer Science and a BS in Civil Engineering from the University of Hawaii, Manoa.

• Jason Liu is a Software Engineer currently associated with the SAGE project. He has been with NGES at the Azusa campus since 2009. Jason holds an MS in Computer Science from UCLA and a BS in Information Computer Science from the University of California, Irvine.