Software Engineering On
Total Page:16
File Type:pdf, Size:1020Kb
Title page - CLEO Baltimore, May 9, 2007 Software Engineering Processes Used to Develop the NIF Integrated Computer Control System* A. Peter Ludwigsen 2007 International Conference on Accelerators and Large Experimental Physics Control Systems, Knoxville, TN, October 14, 2007 Lawrence Livermore National Laboratory, USA 1 * This work performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under Contract DE-AC52-07NA27344. NIF is a stadium-sized facility that will contain • 192-beam, 1.8-Megajoule, 500-Terawatt, ultraviolet laser system • 10-meter diameter target chamber with room for nearly 100 diagnostics 2 National Ignition Facility “Quads” (4 beams) and “Bundles” (8 beams) are the basic building blocks of the NIF 3 NIF’s 192 energetic laser beams will compress small deuterium-tritium fusion targets to conditions where they will ignite and burn, and achieve energy gain 4 The Integrated Computer Control System (ICCS) robustly operates a complex physics facility •• ControlControl 60,00060,000 pointspoints ¾¾ 1,500 processes 1,500 processes AutomaticAutomatic controlcontrol ofof 192-beam192-beam shotsshots ¾¾ 850 computers 850 computers areare overseenoverseen byby 1414 operatoroperator stationsstations ¾¾ 140,000140,000 distributeddistributed objectsobjects •• AlignAlign andand firefire 192192 laserslasers ¾¾ << 5050 micronsmicrons onon targettarget ¾¾ << 3030 psecpsec timing timing •• AutomaticAutomatic shotshot controlscontrols ¾¾ << 4-hours4-hours shot-to-shotshot-to-shot ¾¾ Hands-offHands-off operationoperation •• AssureAssure ¾¾ SituationalSituational awarenessawareness ¾¾ MachineMachine protectionprotection 5 ICCS is required to automatically align, fire and diagnose laser shots every 4 hours Fire pre- Fire main amplifier amplifiers S Amplifier cooling S S h h h o o o t Archive Shot Preparation t Verify t Archive 30-m 20-m 2-s 4 hours Automated Shot Cycle • Acquire campaign shot goals from laser physics model • Perform automatic alignment and wavefront correction • Configure diagnostics and laser performance settings • Conduct countdown (software:4-min and timing:2-sec) • Assess shot outcome and archive shot data 6 Summary of Key Technical Challenges Key Challenges Solutions Scale from single bundle to full NIF Partitioned into 24 independent bundles Align 192 beams on target in 30 Robust parallel image processing of minutes machine vision sensors Shots automated with model-based Fire laser shots in 4-hour cycles control software Incorporated shot verification and Assure machine protection situational awareness features Provide flexible controls capable of Created versatile plug-in software supporting NIF for decades framework using CORBA distribution 7 Software engineering and computer science were key to building NIF’s large-scale control system • Software engineering — Open distributed architecture based on industry tools — Formalized design process — Configuration Management — Product Integration — Formal Verification Testing • Computer science — Application frameworks developed in-house — Standardized coding techniques — Extensive peer reviews Goal: Build team with the necessary experience and engineering rigor 8 A segmented and layered architecture was implemented to decompose the control system 9 ICCS software framework is an object-oriented toolkit to build large-scale control systems 10 Open software architecture supports multiple environments and simplifies technology evolution Category Tool / Environment Usage - Sun (Solaris) - Servers, FEPs w/o hardware Computers / - PowerPC VME (VxWorks) - FEPs with hardware Operating - X86 (WinXP, LINUX) - Consoles, image processing System - X86 PC104 (WinCE) - Target diagnostic controllers - Java - GUIs, frameworks, commissioning tools - Ada - FEPs, servers and supervisors Languages - C - Embedded controllers - IDL - On-line data and image analysis - XML - Workflow models and scripts CORBA - JacORB - Java applications (open source) Distribution - ORBexpress - Ada applications - Eclipse - Java IDE (open source) - IBM Rational - Solaris, VXworks Ada IDE Software - Ada Core Technology - Windows, LINUX Ada IDE Development - ILOG - Graphics builder tools - UML - Object modeling Database - Oracle - Data-driven configuration, archives AsAs technologytechnology evolves,evolves, CORBACORBA allowsallows ICCSICCS toto migratemigrate withwith itit 11 ICCS architecture permits each bundle to be built and commissioned independently of the others A partitioned architecture helped ICCS eliminate scaling risks 12 Data-driven architecture adapts quickly in the field to suit operating conditions • Flexible and responsive — Nominal values deployed along with new capabilities — Data modified as components are commissioned — Operators update equipment calibration • Separate database instances map to the environment Development: Schema and initial data Integration: Validates data and schema Formal Test: Final verification Production: On-line use Aux. Facilities: Factory and maintenance facility support Software modeling and data-driven automation behavior supports flexible point-of-use commissioning and optimization 13 Shot Director Supervisor 14 15 Rigorous quality controls assure delivery of reliable software to the facility The QC process resolves software problems early, with more than 90% of problems found prior to commissioning or shot operations 16 Incremental development strategy delivered functionality, while also iteratively managing risk • Target high-level project milestones — Major release planned every 12 months — Over 60 minor releases and patches deployed per year • Release content driven by — Project goals and identified capabilities — Defect resolution — Improvements requested by operation staff — Framework enhancements — Perceived risks • Strict development lifecycle followed — Managers plan release and collect detailed requirements — Developers design, code, unit test and integrate — Separate test team performs offline and online tests Goal: Support the overall project schedule while maintaining a manageable release content 17 Software metrics demonstrate the team’s adaptability to deliver flexible releases Average Average Release Average Deployments Average Development Integration Type Changes per Year Test Time Time Time Major 540 1 1 y 1 m 1.5 m Minor 469 3 12 w 2.5 w 4 w Service 57 11 4 w 1 w 2.5 w Pack Patch 7 50 4 d <1 d 2 d Script & Several Hundreds NA NA NA Database per Day 18 Releases are actively managed by closely monitoring the change tracking system Tracking System Database — Releases contain subsystems that are individually tracked — Subsystem leads estimate the work in hours — Change states tracked: – Assigned, Completed, Desk-checked, Integrated, Released Active Management — Planning – Work assigned to each subsystem is analyzed – High levels of change requests indicate increased testing – Unrealistic developer workloads are replanned — Release – Progress is tracked on a daily basis – Trends predict completion dates – Progress is easily understood – Release content continues to be adjusted as needed The tool maintains data on past release performance, which is used to flag potentially unreasonable expectations 19 Status of Software Change Requests (SCR) predicts the release completion date 900 800 700 600 500 SCRs 400 300 200 100 0 11-Dec 18-Dec A 25-Dec ssi g 1-Jan ned 8-Jan 15-Jan 22-Jan 29-Jan 5-Feb 12-Feb 19-Feb 26-Feb 5-Mar 12-Mar 19-Mar 26-Mar 2-Apr Completed 9-Apr 16-Apr 23-Apr 30-Apr Desk Checked 7-May Integrated 14-May 21-May 28-May 4-Jun 11-Jun 18-Jun 25-Jun 2-Jul 9-Jul Integration 16-Jul Predicted to 23-Jul Start Formal Test 30-Jul 6-Aug 13-Aug 20-Aug 27-Aug 3-Sep 10-Sep Predicted 17-Sep 24-Sep to NIF 1-Oct 8-Oct 15-Oct 22-Oct 29-Oct 20 Peer reviews enforce design and coding standards during the development phase • Requirements Reviews — Held with hardware and operations Subject Matter Experts — Assures developers and testers understand work • Design Reviews — Mandatory for new systems, or for complex changes • Code Reviews — Desk Checks performed by another developer – Does code meet requirements? – Does code implement the design properly? — Walkthrough Inspections – Team review of critical code or key interfaces ¾ NIF’s 100% Desk Check policy catches 15% of total defects Goal: Find defects earlier in the process, preferably before testing when cost to repair is lowest 21 Efficient developer testing is a combination of unit testing followed by formal product integration • Unit Tests confirm code changes — Informal peer review — Developed to specific standards and practices — Automated tests are being developed • Integration Tests ensure products work together — Tests interfaces — Verifies proposed database changes — Applies various test levels – Change verification – Regression checks – Shot sequences – Scaling and loading tests 22 Software releases are formally integrated in an “extreme programming” mode • Led by the Product Integration Manager • Integration is started before all coding is completed — Gets the team into test and fix mode • Daily integration goals are published — Lower-level software functions verified first — Shot sequences evaluated last — Goals updated based on previous day’s results • Extreme programming works for NIF — All developers on call (via pagers) — Development code base used for integration — Defects fixed on the spot where possible — Integration defects carefully prioritized Goal: Produce a final integrated product with good quality 23 Independent verification demonstrates