The Measurable News

THE MEASURABLE NEWS

The Quarterly Magazine of the College of Performance Management 2017.04

mycpm.org

INSIDE 11 15 19 29 Intelligently Linking Project Decision Analysis Assessing Earned Value Trust, but Verify: An THIS Information for Process Management and Earned Improved Estimating Better Performance Schedule Forecasting Technique Using the ISSUE Management By Lev Virine Integrated Master Across Industry and By Walt Lipke Schedule (IMS) Government By Eric M. Lofgren By Gordon M. Kranz THE MEASURABLE NEWS

The Quarterly Magazine of the College of Performance Management 2017.04

2017 ISSUE 04 CONTENTS

05 Letter from the Editor(s)

09 Message from the President

11 Intelligently Linking Information for Better Performance Management Across Industry and Government Gordon M. Kranz

15 Project Decision Analysis Process Lev Virine

19 Assessing Earned Value Management and Earned Schedule Forecasting Walt Lipke

29 Trust, but Verify: An Improved Estimating Technique Using the Integrated Master Schedule (IMS) Eric M. Lofgren

37 Agile's Earned Schedule Baseline Robert Van De Velde

44 Integrated Program Performance Management Reading List Glen B. Alleman

46 Vendors/Services mycpm.org THE MEASURABLE NEWS IS AN OFFICIAL PUBLICATION OF THE THE COLLEGE OF THE MEASURABLE NEWS COLLEGE OF PERFORMANCE PERFORMANCE MANAGEMENT MANAGEMENT The Quarterly Magazine of the College of Performance Management EDITORIAL STAFF Publisher: College of Performance Management Story Editors: Glen B. Alleman and Rick Price Design: id365 Design + Communications 2017.04 Communications VP: Lisa Mathews 2017 BOARD & STAFF EDITORIAL COPY PRESIDENT Wayne Abba Editorial contributions, photos, and miscellaneous 703-658-1815 • [email protected] inquiries should be addressed and sent to the editor at the College of Performance Management (CPM) headquarters. Please follow the author guidelines EXECUTIVE VICE PRESIDENT posted on the CPM web site. Letters submitted to Kym Henderson the editor will be considered for publication unless +61 414 428 537 • [email protected] the writer requests otherwise. Letters are subject to editing for style, accuracy, space, and propriety. All VICE PRESIDENT OF FINANCE letters must be signed, and initials will be used on Brian Evans request only if you include your name. CPM reserves 703-201-3278 • [email protected] the right to refuse publication of any letter for any reason. We welcome articles relevant to project VICE PRESIDENT OF ADMINISTRATION management. The Measurable News does not pay for Lauren Bone submissions. Articles published in The Measurable +44 (0) 7766 97 40 63 • [email protected] News remain the property of the authors. VICE PRESIDENT OF CONFERENCE & EVENTS ADVERTISING Kathy Evans Advertising inquiries, submissions, and payments 703-201-3278 • [email protected] (check or money order made payable to the College of Performance Management) should be sent to CPM VICE PRESIDENT OF EDUCATION & CERTIFICATION headquarters. Advertising rates are $1000 for inside Bill Mathis front or back cover (full-page ad only), $800 for other 703-825-5588 • [email protected] full-page ads, $500 for half-page ads, and $300 for quarter-page ads. Issue sponsorships are available at $2,500 per issue. Business card ads are available VICE PRESIDENT OF RESEARCH & STANDARDS for $100 per issue (or free with full-page ad). Rates Marty Doucette are good from January 1, 2017 – December 31, 2017. 317-727-1237 • [email protected] College of Performance Management reserves the mycpm.org right to refuse publication of any ad for any reason. VICE PRESIDENT OF COMMUNICATIONS Lisa Mathews SUBSCRIPTIONS 301-802-0627 • [email protected] All College of Performance Management publications are produced as a benefit for College of Performance PAST PRESIDENT Management members. All change of address or Gary W. Troop membership inquiries should be directed to: 310-365-3876 • [email protected]

College of Performance Management EXECUTIVE DIRECTOR 11130 Sunrise Valley Drive, Suite 350 Don Kaiser Reston, VA 20191 703-370-7885 • [email protected] Ph 703-370-7885 • Fx 703-370-1757 www.mycpm.org

All articles and letters represent the view of the authors and not necessarily those of College of Performance Management. Advertising content does not signify endorsement by College of Performance Management. Please notify College of Performance Management for single copy or reproduction requests. Appropriate charges will apply.

THE MEASURABLE NEWS 2017.04

LETTER FROM THE EDITOR(S) Glen B. Alleman and Rick Price

This issue of The Measurable News has a new topic that was introduced at IPMW this last November – Building Information Modeling (BIM). BIM is a process to create and manage digital representations of the physical and functional characteristics of a physical structure that are built, analyzed, documented and assessed virtually, then revised iteratively until the optimal model results.

There is a story (urban legend perhaps) that is applicable here1

The automated steam engine is said to have been invented by a young child. In the early days of steam power, steam engines were used to simply push a piston connected to a rocking beam. Due to the simplicity of these machines it was necessary to manually open a valve at the end of each stroke to release the pressure on the piston. Young children were employed to open and close this valve, an occupation called the plug man (Deakins 2008).2 As the story goes, one who wanted to play stickball outside with some other children devised a mechanism from scrap steel to do his job. He connected one end of a rod to the rocking beam and the other end to the lever for the valve. As the beam rocked back and forth it opened and closed the valve.

Applying BIM to Defense acquisition, beyond buildings and facilities, is an opportunity to devise a more effective way to assess the performance of projects by connecting the dots between the tree structured WBS, the 3D model of a physical deliverable, the cost and schedule components of that model and its physical manifestation, and information about the interdependencies of the components of both the model and the physical product, and how these interdependencies drive the risk to cost, schedule, and technical performance.

In our current Integrated Program Performance Management world, the WBS and the Earned Value Management processes (IPMR Formats 1 and 3) fail to show these interdependencies and the probabilistic behaviors of cost and schedule that result from these interdependencies. Integrating knowledge about product development with the performance measures of the program developing that product has existed in the construction domain for decades.3 It’s time to start making the links between cost, schedule, technical performance, Measures of Effectiveness, Measures of Performance, Key Performance Parameters, Technical 1) “Steam Engine Familiarly Performance Measures, and Risk. Explained and Illustrated,” Edith Edition, Revised and Improved, The object modeling capabilities of BIM provides this circular integration4 of the measures Dionysius Lardener, 1851. needed to increase the probability of program success, interconnected with the program’s 2) Deakins, S. (2008) The Primitive technical, cost, and schedule attributes. Steam Engines University of Notre Dame, AME34104 Engineering Analysis. BIM is based on the aecXML which is a XML mark-up language using Industry Foundation 3) “Linking Knowledge-Based Classes to create a vendor-neutral means to access data generated by Building Information Systems to CAD Design Data Modeling. BIM has been developed for use in the architecture, engineering, construction with an Object-Oriented Building and facility management industries, in conjunction with BIM software (e.g. Autodesk Revit Product Model,” Kenji Ito, Building), and is trademarked by buildingSMART (the former International Alliance for Yasumasa Ueno, Raymond Levitt, Interoperability), which is a council of the National Institute of Building Sciences (NISB) and Adnan Darwiche, Center of (http://www.nibs.org/) NISB a non-profit, non-governmental organization bringing together Integrated Facility Engineering, Stanford University, Technical representatives of government, professions, industry, labor and consumer interests, and Report Number 17, August 1989. regulatory agencies to focus on the identification and resolution of problems and potential 4) “Circle Integration,” Martin problems that hamper the construction of safe, affordable structures for housing, commerce Fischer and John Kunz, CIFE and industry throughout the United States. Working Paper Number 20, Center for Integrated Facility Engineering, Sanford University, April 1993.

The Quarterly Magazine of the College of Performance Management | mycpm.org 05 The BIM model is a graphical design and is also a virtual database including technical and management data. The construction manager (CM) can use BIM as a real simulation of the actual project. One question is, “Could this notion of program management be transferred to the management of Defense Programs?”5

Let’s start with a framework for BIM – the Product Lifecycle Management paradigm. One place BIM has been established on the DOD side is Ship Building. In the standard BIM domain, the term model is the critical success factor. A BIM solution creates computable building information that enables the model of the building to be understood by a software system as a building. A wall, for example, "knows" what it is and how to react to the rest of the building. As such, it can be scheduled or quantified as a wall: a building assembly made of real materials. Computable building information supports numerous building design and construction activities: structural analysis, MEP (Mechanical, Electrical, Plumbing) system modeling, building energy analysis, and specification management.

Cost estimating of the building process benefits from computable building information as well. Designing a building is done by architects; assessing the cost to build it is done by cost estimators. When preparing cost estimates, estimators begin by digitizing the architect’s paper drawings, or importing their CAD drawings into a cost estimating package, or doing manual material takeoffs from their drawings. These methods introduce the potential for human error and propagate any inaccuracies there may be in the original drawings.

Using the BIM instead of drawings, the material takeoffs, counts, and measurements are generated directly from an underlying model. This information is consistent with the design, and when a change is made in the design – a smaller window size, for example – the change propagates to the related construction documentation and schedules, as well as all material takeoffs, counts, and measurements that are used by the estimator.

One important concept to connect our Defense procurement processes with building design and construction is the recognition that our domain is based on MIL-STD-881-C, where all the components of the product in the Program Performance Management domain are structured as a Tree with formal relationships. This formality is Mutually Exclusive, Collectively Exhaustive, which is a is a grouping principle for separating a set of items into subsets that are mutually exclusive (ME) and collectively exhaustive (CE). In this structure, the parent/ child relationship of the components of the systems are strictly enforced.

In BIM, the relationships between the components of the systems are networked. In the construction business, there are Five Big Ideas: 6

1. Collaboration throughout design, planning, and execution 2. Increase readiness among all project participants 3. Projects are networks of commitments 4. Projects are optimized not the pieces of the project 5. Learning is tightly coupled with action

This paradigm of the network is held in the BIM models, to be queried to retrieve information when needed. One important concept with BIM is the JSON (JavaScript Object Notation) data structure for BIM data exchange. JSON has started to enter the Earned Value Management data processes as well. The BIM model is guided by ifcJSON4 (http:// openbimstandards.org/).

In the building industry, building data such as objects and processes are described in Industry Foundation Classes (IFC) data model schema to support a neutral data exchange format for the BIM tools to interoperate.

5) “Building Information Modeling The opportunity to use JSON as the exchange basis between BIM and the EVMS is the next (BIM) and the Construction step to be explored in more detail. Management Body of Knowledge,” Mehmet Yalcimkaya and David Here’s a notional example of the difference between a WBS program structure and a BIM Arditi, The IFIP WG5.1 10th structure: While the BIM structure can include the WBS as a coded field, BIM is a multi- International Conference on dimensional framework representing the various views of the program, including documents, Product Lifecycle Management – products, classifications systems, building specification, CAD system models, engineering PLM13. calculations, facilities management systems, all coded in Industry Foundation Classes (IFC). 6) “Building Information Modeling: A Framework for Collaboration,” Howard W. Ashcraft, Construction Lawyer, Volume 28, Number 3, Summer 2008.

06 The Measurable News 2017.04 | mycpm.org

Figure 2 ‒ BIM is a structure of structures, which can include the WBS.

Figure 1 ‒ WBS is a structure tree with Mutually Exclusive Exhaustively Complete, tree with Parent Child relationships

In this issue of The Measurable News, we’ve included a Bibliography of BIM information. These

will further inform you how this powerful modeling can be adapted to DOD acquisition, including Ship Building.

As well, a paper by Gordan Kranz, Earl Carter, Jeff Gravatte, Cathi Hayes, Raul Gomez, and Michael Marcel presents the foundation of BIM and its applicability to Defense acquisition.

Figure 2 ‒ BIM is a structure of structures, which can include the WBS.

Figure 1 ‒ WBS is a structure tree with Mutually Exclusive Exhaustively Complete, tree with Parent Child relationships In this issue of The Measurable News, we’ve included a Bibliography of BIM information. In this issue of The Measurable NewsThese, we’vewill further included inform a youBibliography how this powerful of BIM information.modeling can Thesebe adapted to DOD acquisition, including Ship Building. will further inform you how this powerful modeling can be adapted to DOD acquisition, including Ship Building. As well, a paper by Gordan Kranz, Earl Carter, Jeff Gravatte, Cathi Hayes, Raul Gomez, and Michael Marcel presents the foundation of BIM and its applicability to Defense acquisition. As well, a paper by Gordan Kranz, Earl Carter, Jeff Gravatte, Cathi Hayes, Raul Gomez, and Michael Marcel presents the foundation of BIM and its applicability to Defense acquisition.

The Measurable News 2017.04 | mycpm.org 07 JOIN NOW!

The College of Performance Management (CPM), an independent entity, is the premier organization for earned value management (EVM) and project planning and controls. As an international non-pro t organization, CPM is dedicated to sustained improvement in project management and performance management.

Come join us and get involved in shaping the exciting future of the “new” CPM!!! Membership is $85/year. Register online at www.mycpm.org.

CPM Objectives • Promote Earned Value Management and Project Planning and Control: Foster the recognition and use of earned value management and other project planning and control techniques as integrating processes for project management • Disseminate Information: Provide opportunities for the exchange of ideas, information, solutions and applications • Improve Community: Encourage and enable the advancement of theory and application through research, standards, and education • Grow Professionals: Provide our diverse membership of project management professionals with growth

• Enhance Membership and Beneﬁts: continued development of a professional association Membership Beneﬁts • Discount to CPM Conferences • Access to latest information on performance management • Networking with other professionals, industry leaders, and academia • Quarterly magazine, The Measurable News • Access to CPM electronic library • Topical Webinars Conference • Earn PDUs for Project Management Professionals • Update skills • Network •

Active members, serving on a committee, as a speaker, volunteer or as a member of the Governing Board, gives you the greatest reward

For more information, email [email protected] or call (703) 234-4116

Join CPM now at www.mycpm.org

PMI is a registered mark of the Project Management Institute, Inc. THE MEASURABLE NEWS 2017.04

MESSAGE FROM THE PRESIDENT Wayne Abba

Department of Defense Instruction 7000.2, “Performance Measurement for Selected Acquisitions” was issued on December 22, 1967, marking the widespread adoption of Earned Value Management (EVM) for complex acquisitions. (This policy instruction was my responsibility from 1982 to 1999 as a civil servant.) Its principles remain intact in other forms a half century later – in government-wide acquisition and procurement regulations, in national and international standards, in professional and industrial associations, and in company management procedures. CPM’s professional education and certification programs, workshops and symposia support all these disparate needs and help to advance the state of the art of integrated program performance management using EVM and other techniques.

Rereading the instruction prompts a flood of career memories. My memories as well as those of many others are being captured and organized by CPM Vice President Marty Doucette and his team of volunteers, who are recording interviews, digitizing videotapes, scanning documents and incorporating all into an EVM Timeline that will document thoroughly EVM’s origins and evolution. We are the original professional association dedicated to integrated performance measurement using EVM. As we mark this milestone, it’s a good time to recall first principles and encourage our younger members to bear them in mind as they adapt the vision of our founders to today’s environment.

However, we should indulge a “chicken or egg” question. What came first, BCWS, BCWP and ACWP or their contemporary versions PV, EV and AC? Would it surprise you to learn it was the latter, as documented in a Performance Technology Corporation letter to the Air Force Ballistics Systems Division on March 21, 1965 (except that AC was “Actual Value”)? My point is that usually, simple is better. The letter forwarded drafts of the “Earned Value General Specification and Checklist” in fulfillment of an Air Force contract. These documents would evolve into the Air Force C/SPCS and DoD C/SCSC – and the rest is history.

That history, as with any government policy, has experienced ups and downs. In my experience, the “downs” occurred when overseers lost sight of the first principles. The original objectives of DoD Instruction 7000.2, as listed in the press release accompanying its release, were to:

“… provide an adequate basis for responsible decision-making by both contractor management and DoD components through contractors’ internal management control systems that provide data which (1) indicate work progress, (2) properly relate cost, schedule, and technical performance, (3) are valid, timely, and auditable, and (4) supply DoD managers with a practicable level of summarization.”

Finding the right balance to achieve these objectives follows a cyclical pattern that is influenced by many factors, which will be apparent from the timeline. For example, ratchet- like tightening of regulations in response to contract cost overruns, accompanied by ever more intrusive customer audits, is followed eventually by loosening of regulations – and so the pendulum swings. The danger, of course, is that the swings will be extreme and will compromise the value of earned value.

Criterion-based management using EVM was designed to support essential management needs that remain relevant today. Every manager needs to know how a contract/project/ program is performing. And the more complex the organization, the more compelling the need for consistency in management reports. As CPM leads the way into the next half century of management evolution, we all should use these first principles as touchstones, especially as we develop the practitioner and enterprise professional levels of our Integrated Program Performance Management certification.

The Quarterly Magazine of the College of Performance Management | mycpm.org 09 THE DATESAVE EVM World® 2018 INTEGRATED PROGRAM PERFORMANCE MANAGEMENT (IPPM) “Intelligently Linking Information for Better Program Management” 34th Annual International Workshop May 30 - June 1, 2018 • The Westin Fort Lauderdale, FL 954-467-1111 • www.westinftlauderdalebeach.com

• Update your skills with the latest Earned Value Management (EVM) trends, tools, and techniques

• Learn through training, practice symposia, as well as workshops

• Earn PDUs (for PMPs)

• Network with earned value professionals from around the world

For more information visit:

www.mycpm.org or www.evmworld.org

INTELLIGENTLY LINKING INFORMATION FOR BETTER PERFORMANCE MANAGEMENT ACROSS INDUSTRY AND GOVERNMENT By Gordon M. Kranz

ABSTRACT One of the constant concerns about the acceptance and implementation of Earned Value Management (EVM) is the perception that earned value metrics and trends do not tie well to the schedule and do not reflect the true technical status of a project. We believe that the aerospace and construction industries can learn to manage programs better by using Building Information Modeling (BIM).

In 2012, the U.S. Department of Defense published the Integrated Program Management Report (IPMR) Data Item Description (DID) which emphasized the integration of EVM data with the schedule and facilitates integrated analysis by requiring a common electronic format for the cost and schedule. The use of common tags such as Work Breakdown Structure, Control Account Numbers, Work Package Numbers and Organizational Breakdown Structure allow for electronic correlation and analysis using computers to increase the breadth and depth of the earned value analysis and establishing root cause.

At the center of EVM is the requirement to claim technical progress based on objective criteria. During the development phase of programs, objective criteria for progress claims are sometimes not straight forward. But if the cost and schedule could be electronically tagged to the system design artifacts, as the design matures the technical, cost, and schedule status would be naturally integrated. The vision of BIM is to address this current lack of data integration issue throughout the project lifecycle.

BIM was created as a process and concept in the early 2000s. The concept is to move the construction industry from building complex structures using 2D drawings to designing and building using a 3D interactive BIM approach. BIM is a process with an intelligent model at the core with time and cost can and serves as a virtual prototype, a complete model of all building with components of the building as discrete objects.

The IPMW 2017 held a first-ever BIM track to bring both construction experts and performance management experts together to foster collaboration and a working relationship to resolve our common goal to improve cost, schedule, and technical performance across our respective communities.

The BIM track consisted of 5 presentations which are summarized below:

BIM 01 OVERVIEW OF BUILDING INFORMATION MODELING (BIM) A Wiki definition of Building Information modeling (BIM): A process involving the generation and management of digital representations of physical and functional characteristics of places.

The BIM process allows teams to communicate ideas in a visual prototype manner allowing for collaboration across program management teams to investigate and interrogate a model, like never before, to guide owner decisions by access to accurate evaluation of construction status throughout the design and construction process and even into operations.

The BIM process provides for the integration of intelligent 3D design with schedule (4D) Gordon M. Kranz, Enlightened and cost (5D) and eventually to support the business operations and maintenance of the Integrated Program Management, system (building). LLC with key input from the presentation authors: Earl Carter, Arora Engineering, Inc., Jeff Gravatte, LEED AP, Cathi Hayes, Hexagon, Raul Gomez, Robins and Morton, and Michael Marcel, K2 Consulting

The Quarterly Magazine of the College of Performance Management | mycpm.org 11 alternative designs based on experience to address customer requirements. BIM will allow the ability to use a computer to produce hundreds or thousands of viable design alternatives for customers, BIM02architects DESIGN and builders AND to BIM choose (3D) from based on a set of desired design outcomes. 3D BIM provides a 3D interactive model of the system being designed and built for project stakeholdersThe 3D model to then communicate. continues toThe mature typical and approach gain fidelity in construction as the building today or is system for architects design matures. to manually develop a few alternative designs based on experience to address customer requirements.Keeping the 3D BIM model will allowup to thedate ability facilitates to use the a computer ability to workto produce through hundreds issues found or thousands during the design ofand viable build design process. alternatives This is particularly for customers, useful architects since complex and builders buildings to arechoose being from designed based and on built aconcurrently. set of desired design outcomes. The 3D model then continues to mature and gain fidelity as the building or system design matures.BIM 03 BIM Keeping Integration the 3D intmodelo Schedule up to date (4D) facilitates the ability to work through issues found during the design and build process. This is particularly useful since complex buildings are being4D BIM designed is the integration and built concurrently.of a high-fidelity 3D model over time. Build sequences and dependencies are built into the model allowing the team to visualize the time phased build and design of the system. BIMBeing 03 able BIM to visualize INTEGRATION the expected INTO maturity SCHEDULE of the design (4D) in the future can be very powerful in support 4Dof identifyingBIM is the integrationpotential resource of a high-fidelity conflicts, building3D model sequence over time. issues, Build and sequences the ability and to iterate on dependencies are built into the model allowing the team to visualize the time phased build andalternate design sequences of the system. to work Being around able unforeseento visualize theissues expected that arise maturity such as of weather the design or lainbor the strikes. future can be very powerful in support of identifying potential resource conflicts, building sequenceBIM 04 BIM issues, Integration and the intoability Cost to (5D)iterate on alternate sequences to work around unforeseen issues that arise such as weather or labor strikes. Simply stated, 5D BIM means adding the element of cost to models (3D) that have been linked to BIMschedule 04 BIMinformation INTEGRATION (4D). 5D BIM INTO includes COST the (5D)pre-construction planning of costs and performance Simplyagainst stated, budget, 5D while BIM alsomeans measuring adding the actual element costs ofand cost changes to models against (3D) plans that for have real been time project insight. linked to schedule information (4D). 5D BIM includes the pre-construction planning of costs andBringing performance together against cost, technical, budget, whileand schedule also measuring objects actualin both costs cloud and and changes mobile environmentsagainst enables plans for real time project insight. automatic analysis of key project indicators such as cost performance (CPI), schedule performance (SPI) Bringingand earned together value (EVM).cost, technical, The reports and scheduleare highly objects visual as in 3Dboth models cloud areand color mobile coded to communicate environmentsstatus, trade andenables location. automatic analysis of key project indicators such as cost performance (CPI), schedule performance (SPI) and earned value (EVM). The reports are highly visual as 3D models are color coded to communicate status, trade and location.

Figure 1: 5D Performance reportFigure (CPI) 1: 5D Performance report (CPI)

THE IMPORTANCE OF LINKING WITH 5D A key concept in 5D BIM is the "intelligent linking" of data. Intelligent linking of data is what makesThe importance real-time visibility of linking of with project 5D status and performance possible. The key is to embrace the 5D workflow during planning, so the cost, schedule and model structures can be mapped, or linked, to each other with the right level of detail from the beginning of the project. By breaking down siloes of information, empowering workers and reducing risks for owners and contractors, 5D BIM presents a real opportunity for the building construction industry.

BIM 05 BIM INTEGRATION WITH LIFECYCLE MANAGEMENT (6D) Extending the use of BIM into the sustainment phase of a project is where the lasting value can be obtained for the building lifecycle. The cost of operating and maintaining a building is 10 to 15 times the initial cost of the building. As buildings get “smarter” new capabilities can be integrated into the BIM model to provide building operators, users, and maintainers easy access to data allowing for more streamlined and predictable operations.

12 The Measurable News 2017.04 | mycpm.org BIM 06 THE FUTURE OF BIM PANEL DISCUSSION During the panel discussion, the presenters and members of CPM talked about the common issues we face in evaluating cost, schedule, and technical status throughout the project. The BIM process emphasizes the data integration of all aspects of the project to quickly and accurately evaluate status, evaluate alternatives to resolve issues, forecasting, and planning out future work.

The group agreed that there is synergy in the construction and BIM experts and the CPM performance management experts. Our plan is to continue the collaboration by forming a BIM interest group. The intent of the group will be to continue the collaboration and to look for opportunities to engage in both BIM forums and performance management forums.

Look for the next BIM track at EVM World 2018.

About the Author: Gordon M. Kranz, Enlightened Integrated Program Management, LLC with key input from the presentation authors: • Earl Carter, Arora Engineering, Inc. • Jeff Gravatte, LEED AP • Cathi Hayes, Hexagon • Raul Gomez, Robins and Morton • Michael Marcel, K2 Consulting

The Measurable News 2017.04 | mycpm.org 13

Humphreys & Associates offers a complete range of Proposal Support Humphreys & Associates offers a complete range of Proposal Support EVMSEVMS consultingconsulting services services. From. Fromproposal proposal preparation preparation Humphreysandand management & Associates system system gap offers gapanalysis analysis ato completemock to mock range of Proposal Support EVMS consultingcompliancecompliance reviewsservicesreviews or . or contractorFrom contractor proposal self- preparationself- EVMS Staff EVMS Staff EVMS Requirements EVMS Requirements Augmentation or Gap Analysis and managementsurveillancesurveillance systemand and third third gapparty party analysisvalidations, validations, to H&A mock is H&A is Augmentation or Gap Analysis the authority in EVMS. compliancethe authorityreviews in EVMS.or contractor self- EVMS Staff EVMS Requirements As the industry recognized leader in AugmHenutmatiponhreys & Associates or Gap Analysis surveillance and third party validations, H&A is EVMS Compliance EVMS Design and AsEVMS the trainingindustry, H&A recognized offers regularly leader in Review Preparation Humphreys & ADsoscuomecnitatiotnes the authority in EVMS. EVMS Compliance EVMS EVMS Design and EVMSscheduled training Earned , ValueH&A Managementoffers regularly Review PrepaCraotinonsulting Services EVMS Documentation scheduledSystem, Project Earned Scheduling, Value andManagement Advanced Consulting Services As the industry recognized leader in Integrated Baseline Humphreys & Associates Earned Value Management Techniques public EVMS Compliance EVMS EVMS Design and System, Project Scheduling, and Advanced Review (IBR) EVMS trainingworkshops. , H&A offers regularly Review Preparation EVMImSplementation Documentation Preparation Integrated Baseline Earned Value Management Techniques public EVMS scheduled Earned Value Management Review (IBR) H&A now offers a Control Account Consulting Services Implementation workshops. PreparatioEnVMS and System, ManagerProject (CAM)Scheduling, Certification and ProgramAdvanced – ideal for Project Management H&anyoneA lookingnow tooffers improve a their Control EVM expertise Account and distinguish Integ rated Baseline Training Earned Value Management Techniques public EVMS Review (IBR) EVMS and Managerthemselves with (CAM) a professional Certification certification. Program – ideal for Implementation workshops. Preparation Project Management anyone looking to improve their EVM expertise and distinguish Training Interested in learning more? H&A nowthemselves offers with a a professionalControl certification.Account Visit our web site at www.humphreys-assoc.com or call us today at (714)EV M685S a-n1730d Manager (CAM) Certification Program – ideal for Project Management anyone looking to improve their EVM expertiseInterested and distinguish in learning more? Training themselves with aVisit professional our web site certification. at www.humphreys -assoc.com or call us today at (714) 685-1730

Interested in learning more? Visit our web site at www.humphreys-assoc.com or call us today at (714) 685-1730 Everyone is Looking for Skilled EVMS Resources…We Have Them

Let The CBT Workshop Assist You!

Staff Augmentation EVMS Services  Skilled EVMS Planners, Schedulers, Program Control  EVMS Gap Analysis Analysts, Estimators, and EVMS Consultants  System Description Design and Development  Short or Long Term Assignments  Desktop Operating Procedures  Contract-to-hire personnel  IBR Preparation and Dispositioning  CAM Training and Mock Interviews

Tools Implementation and Training Web Based EVMS Training We have EVMS tools experts who can implement, train,  EVMS Fundamentals and EVMS Advanced training, integrate, and migrate EVMS data in the following tools: testing, and certification.  Microsoft Project and Project Server  Deltek MPM, wInsight, Cobra, Open Plan  Primavera Call UsToday!

website: www.CBTWorkshop.com phone: 888•644•5613 THE MEASURABLE NEWS 2017.04

PROJECT DECISION ANALYSIS PROCESS By Lev Virine, Intaver Institute

ABSTRACT Project management is the art of making the right decisions. Project managers are faced with a huge array of choices. Should a different supplier be used to improve the quality of a product? Should an additional team member be brought in to improve the development performance? Should the work be outsourced or done in-house?

In addition to project management, decision analysis is used in strategic planning, operational management, and other areas of business. Decision analysis helps oil and gas companies determine optimal exploration and production strategies with uncertainties in cost, prices, and exploration prospects. Lawyers use decision analysis to assess complex litigations with uncertain outcomes. Decision analysis helps medical professionals make correct diagnoses and prescribe the most effective treatment.

The most important components of decision analysis can be integrated into project management processes in all knowledge areas. This includes the analysis of potential alternatives as part of each stage of the project and the assessment of uncertainties as part of project risk management. New project management software utilizing quantitative analysis helps project managers make informed decisions. Recent research shows that well-established decision analysis process integrated into an overall project management framework significantly improves organizational performance.

Among the diverse problems that impede accurate decision analysis, those inherent in human mental processes are the most important and most difficult to deal with. Psychologists have discovered a number of patterns in the way people select alternatives, assess probabilities, identify and manage risks, and make decisions. The knowledge of such patterns will help decision-makers avoid potential mental traps and ultimately improve the quality of their decisions.

“3C” PRINCIPLE OF PROJECT DECISION ANALYSIS The decision-making process is a framework that helps project managers solve a variety of decision-making problems. There are no exact recipes for how decision analysis should be structured. The process can be tailored for different companies, types of projects, and the types of decisions that must be made. Any decision analysis process is based on three main rules, which we call the 3C Principle (see figure 1):

1. Consistency: Standardized decision analysis process for similar kinds of problems and opportunities enables consistent decision making over time.

2. Comprehensiveness: Full 360⁰assessments and analyses of the business situation are vital. Missing or incomplete information can lead to incorrect decisions.

3. Continuity: The value of decision analysis will significantly diminish if preformed only in discrete situations during the course of a project. Decision analysis is a continuous process of making and refining decisions during the course of a project.

The Quarterly Magazine of the College of Performance Management | mycpm.org 15 The decision-making process is a framework that helps project managers solve a variety of decision-making problems. There are no exact recipes for how decision analysis should be structured. The process can be tailored for different companies, types of projects, and the types of decisions that must be made. Any decision analysis process is based on three main rules, which we call the 3C Principle (see figure 1): 1. Consistency: Standardized decision analysis process for similar kinds of problems and opportunities enables consistent decision making over time. 2. Comprehensiveness: Full 360⁰assessments and analyses of the business situation are vital. Missing or incomplete information can lead to incorrect decisions. 3. Continuity: The value of decision analysis will significantly diminish if preformed only in discrete situations during the course of a project. Decision analysis is a continuous process of making and refining decisions during the course of a project.

Figure 1. 3C principle of decision analysis STEPS OF DECISIONFigure ANALYSIS 1. 3C principle PROCESS of decision analysis To illustrate the process (figure 2), let us analyze a hypothetical example. A software development project is in its final stage, a deadline is looming, and there is a chance the project will not be completed on time. We will examine how the decision analysis process can identify the best solution for the problem. Steps of Decision Analysis Process To illustrate the process (figure 2), let us analyze a hypothetical example. A software development project is in its final stage, a deadline is looming, and there is a chance the project will not be completed on time. We will examine how the decision analysis process can identify the best solution for the problem.

Figure 2. Steps ofof DecisionDecision AnalysisAnalysis Process

1. DECISION FRAMING Decision1. framing Decision is Framing based chiefly on subjective expert judgment. Experts provide their own beliefs inDecision the form framing of their is based answers, chiefly which on subjective can be expertbiased. judgment. There are Experts many provide forms their of biases: cultural, ownorganizational, beliefs in the motivational,form of their answers, cognitive, which and can others. be biased. Motivational There are many and formscognitive of biases are mostbiases: common cultural, in project organizational, management. motivational, cognitive, and others. Motivational and cognitive biases are most common in project management. a. Identifying Potential Problems and Opportunities In somea. Identifying cases, it Potentialis difficult Problems to identify and Opportunities the problems and opportunities. For example, what is causingIn some the cases, different it is difficult projects to identify within the the problems organization and opportunities. to go consistently For example, over budget in relationwhat tois causingthe different the different specific projects corrective within the actionsorganization that to were go c onsistentlyundertaken? over In our software developmentbudget in relation example, to the the different project specific will be corrective delayed actions if certain that wereactions undertaken? are not taken.In our software development example, the project will be delayed if certain actions are not b. Assessingtaken. Business Situation Before attempting to make a decision, it is important to assess the business environment and b.define Assessing the Businessconstraints Situation related to the problem. The assessment may also include an analysisBefore attempting of markets, to make competition, a decision, prices, it is important or anything to assess that the is business possibly environment related to the problemand define or opportunity. the constraints In relatedour example, to the problem. it is the The availability assessment of may an also additional include anresource. analysis of markets, competition, prices, or anything that is possibly related to the problem or opportunity. In our example, it is the availability of an additional resource. 16 The Measurable News 2017.04 | mycpm.org c. Determining Success Criteria In our software development example, success is defined by the chance that the project will be completed on time. Very often project managers have to make decisions based on multiple criteria, including project duration, cost, scope, and other parameters. c. Determining Success Criteria In our software development example, success is defined by the chance that the project will be completed on time. Very often project managers have to make decisions based on multiple criteria, including project duration, cost, scope, and other parameters.

d. Identifying Uncertainties Understanding the uncertainties that can affect the project is the key to the decision analysis process. In our example, there are uncertainties in task duration, start and finish times. Potentially, there could be many different types of uncertainties including uncertainties in cost, resource allocation, and others.

e. Generating Alternatives First, we identify what cannot be changed, or project constraints for making the particular decision analysis. In our software example, it is the deadline. The project scope is a constraint as well. However, there is the possibility of bringing additional resources (software developers) to accelerate the development. As a result, we have three potential project scenarios: a. “Do nothing”. In this example, it means that additional project resources will not be added to the project team. b. Bring a developer from another team within the organization. c. Hire an external contractor.

2. MODELING THE SITUATION A mathematical model helps the analysis and estimation of future events. During the modeling stage, project managers rely on heuristics or rules of thumb to make estimations and create the model. Under many circumstances, heuristics lead to predictably faulty judgments or cognitive biases.

a. Creating Models for Each Project Alternative Project managers constantly create mathematical models of projects, in most cases this is the project schedule. Sometimes, more elaborate models are required. For example, in the analysis of a complete product lifecycle, comprehensive models will include not only product development, but also marketing and sales efforts. In our example, it is possible to create three simple slightly different project schedules associated with each scenario identified at the decision-framing stage: “do nothing”, add a resource from another project team, and hire an external contractor.

b. Quantifying the Uncertainties The uncertainties, identified through the decision framing process should be quantified. One of the ways to quantify uncertainties is defining ranges for parameters. For example, define low (optimistic), base (expected), and high (pessimistic) duration estimates for each task.

Another way to define uncertainties is to list all the potential events, that could affect the project schedule and quantify their probabilities and impact. In our software development example, there is a 50% chance that external consultant will not be familiar with the subject area for the software project, which may delay the development by 20%.

3. QUANTITATIVE ANALYSIS The analysis should give project managers enough data to make an informed decision. Even with most advanced analytical tools and techniques, interpretation of the results of the analysis is the subject of multiple mental traps.

a. Determining what is Most Important A model of a project may include a considerable number of variables: large numbers of tasks, resources, risks, and other parameters. For example, certain risks will cause a failure of the project, while others risks will have no noteworthy effect on the project. To determine which project parameter is the most important, project managers can use sensitivity analysis. In our software development example, the duration required for the training of the external contractor in one of the potential project scenarios can be very uncertain because the experience of the contractor in the particular subject area is unknown.

b. Quantifying Risks Associated with the Project Uncertainties associated with input parameters were already quantified during modeling step. Now it is important to analyze how the combination of all these uncertainties might impact the project success. We can apply a number of analytical techniques for this analysis.

The Measurable News 2017.04 | mycpm.org 17 In our example, quantitative analysis shows the following probabilities under each scenario that the project will be completed on time:

a. “Do nothing” – 32% b. “Bring resource from another team” – 95% c. “Hire external contractor” – 65%

c. Determining the Value of New Information One of the useful decision analysis techniques is to assess the value of new information. For example, the goal is to select new development tools for the software project based on performance. Tests can be done to determine performance, but could be costly and time consuming. Alternatively, it is possible to select the tools based on specifications, without performing specific tests. The analytical technique helps to establish the value of new information, which in this case would be the testing results, and to determine whether money should be spent on the test.

d. Deciding on a Course of Action In many situations, selection of alternatives is not so trivial. Sometimes, a decision must be made using multiple criteria, which complicates the selection of the most efficient alternative. In our example, it is clear that, according to our success criterion, we should select alternative (b) “Bring resource from another team”.

4. ACTUAL PROJECT PERFORMANCE TRACKING Now a decision has been made and a selected course of action is under way. However, in the middle of the project, an unforeseen event occurs that causes the selected plan to derail. For example, because of other commitments, the new software developer cannot move to the project. Luckily, there is a quantitative model of the selected project alternative and the project manager can update the model, perform a new analysis and make a decision. It is very important to constantly track project performance and analyze all potential pitfalls and opportunities.

18 The Measurable News 2017.04 | mycpm.org THE MEASURABLE NEWS 2017.04

ASSESSING EARNED VALUE MANAGEMENT AND EARNED SCHEDULE FORECASTING By Walt Lipke, PMI® Oklahoma City Chapter

ABSTRACT Recent research indicates cost and schedule forecasting from EVM data is improved when the performance factor, PF = 1, is used. This paper uses a small set of real data to examine the research finding, to either confirm or refute. As well, the application of PF = 1 is employed in statistical forecasting; results are tested and compared to the index method. Observations from the research and this study are made referencing historical studies. Further research is encouraged on these topics, but with some precaution when real data is used.

INTRODUCTION

The 2015 paper, “Empirical Evaluation of Earned Value Management Forecasting Accuracy for Time and Cost” authored by Batselier and Vanhoucke, is the inspiration for this article [Batselier et al, 2015]. Their paper is an impressively comprehensive examination of forecasting from the use of Earned Value Management (EVM) data taken from 51 projects, predominantly construction.

In the history of EVM and Earned Schedule (ES) research, covering 25 years for cost and 15 years for schedule, one type of forecasting formula, incredibly, has been ignored. Included in these past studies are several published by Christensen1, Vanhoucke2, Crumrine3, and Lipke.4 Uniquely, Batselier and Vanhoucke (B&V) examine several methods of forecasting. B&V demonstrate overwhelmingly in their analysis this ignored formula yields forecasts more often better than the ones most frequently employed by EVM and ES practitioners.

This article, using a smaller set of data than that used by B&V, attempts to corroborate their finding. The primary objective, however, is to implement the improvement shown for deterministic forecasting into statistical forecasting. The focus is to assess whether the improved nominal forecast translates to better statistical forecasts. As well, the investigation may reveal logical reason for the B&V results.

The subsections following, EVM & ES Forecasting, and Statistical Forecasting, provide background for understanding the remainder of the article.

EVM & ES FORECASTING. EVM and ES forecasting formulas are very similar. They each have the same basic construct; i.e., the forecast is equal to the current value plus the remainder yet to accomplish divided by a selected performance factor.

Before discussing the formulas, the following EVM and ES terminology is introduced in table 1. It is assumed the reader has a fundamental understanding of EVM and ES. If a more complete description is needed, please reference the following: Practice Standard for Earned Value Management [PMI, 2011], and Earned Schedule [Lipke, 2009-2].

1) Christensen et al, 2002(-1,-2), 1995, 1993(-1,-2)). 2) Vanhoucke et al, 2007, 2006. 3) Crumrine et al, 2013. 4) Lipke, 2017, 2016, 2015, 2014, 2012, 2011, 2009-1, 2006.

The Quarterly Magazine of the College of Performance Management | mycpm.org 19

The subsections following, EVM & ES Forecasting, and Statistical Forecasting, provide background for understanding the remainder of the article.

EVM & ES Forecasting. EVM and ES forecasting formulas are very similar. They each have the same basic construct; i.e., the forecast is equal to the current value plus the remainder yet to accomplish divided by a selected performance factor. Before discussing the formulas, the following EVM and ES terminology is introduced in table 1. It is assumed the reader has a fundamental understanding of EVM and ES. If a more complete description is needed, please reference the following: Practice Standard for Earned Value Management [PMI, 2011], and Earned Schedule [Lipke, 2009-2].

TableTable 1.1. EVM andand ESMESM Terminology Terminology and and Formulas Formulas

In the B&V paperIn the several B&V paper performance several performance factors (PF) factors are examined(PF) are examined for EVM for and EVM ES and forecasting. For thisES paper, forecasting. only four For are this used. paper, As only depicted four are inused. table As 1, depicted cost applies in table to 1, 1 costand appliesCPI, while to schedule1 anduses CPI, 1 and while SPI(t). schedule The reasonuses 1 and only SPI(t). these The are reason studied only is theseto corroborate are studied theis to B&V finding that use of PF = 1 provides, in most instances, a better forecast of the actual outcome corroborate the B&V finding that use of PF = 1 provides, in most instances, a better than does the most often used cumulative value for the performance indexes. forecast of the actual outcome than does the most often used cumulative value for the For manyperformance years, it has indexes. generally been conceded that, overall, the performance indexes provide the mostFor manyreliable years, of theit has possible generally EVM been and conceded ES forecasting that, overall methods., the performance Some rationale for relianceindexes on theprovide cost the index most comes reliable from of the Dr. possible Chistensen’s EVM and conclusion ES forecasting that methods. CPI tends to worsen Someas the rationale project forprogresses reliance on toward the cost completion index comes [Christensen, from Dr. Chistensen’s 1993]. As conclusion well, Christensen determinedthat CPI that tends forecasts to worsen using as CPIthe pareroject optimistic, progresses which toward he completion termed the [Christensen “low bound”, [Christensen1993] . etAs al, well, 2002-1). Christensen His research determined indicates that forecasts that the using forecast CPI are using optimistic, CPI will which be hebetter than PFtermed = 1; i.e., the the “low CPI bound” forecast [Christensen will be optimistic, et al, 2002 -but1). His PF research= 1 will be indicates even more that the so. This comparativeforecast deduction using CPI may will be be better the reason than PF PF = 1;= 1i.e., had the not CPI been forecast seriously will be examinedoptimistic, butprior to the B&V paper.

STATISTICAL FORECASTING. The use of statistical methods for inferring outcomes is a long-standing proven mathematical2 approach. The statistical forecasting method for duration has been demonstrated to perform reasonably well [Lipke et al, 2009-3].

The current statistical method of duration forecasting is derived from the ES equation, IEAC(t) = PD / SPI(t), where using the cumulative value of SPI(t) yields the nominal deterministic forecast. The associated high and low Confidence Limits5 (CL) are computed from the variation of ln SPI(t)P, i.e., the logarithm of the periodic index values. As well, the statistical forecasting method for duration is equally applicable to cost by using the appropriate indexes.

Because B&V have shown PF = 1 to provide better forecasts, curiosity is raised concerning its use in statistical forecasting. Thus, It is desired to adapt PF = 1 forecasting such that comparison can be made to the present method. The adaptation is not difficult, but does need some explanation.

Let’s begin with the PFS = 1 duration forecasting expression:

IEAC(t) = AT + PD – ES

First, multiply and divide the ES term by AT. Then by arranging terms, the formula is transformed to:

5) Information about Confidence IEAC(t) = AT + PD – AT • (ES/AT) Limits may be found in [Crowe et al, 1960]. Confidence Limits are = AT + PD – AT • SPI(t) sometimes misunderstood to be thresholds for management action. The limits, instead, describe the region containing the “true” value of the parameter at the prescribed probability, i.e. Confidence Level. 6) For a more complete description of Confidence Limit calculations using EVM and ES data consult the following reference [Lipke, 2016].

20 The Measurable News 2017.04 | mycpm.org This expression facilitates the statistical use of the cumulative and periodic index values; the identical values used in the current statistical method. Thus, the forecast confidence limits are computed using the index limits in the current method.6 That is, for example, the high forecast limit becomes:

IEAC(t)H = AT + PD – AT • e^(CLL)

where e = the base number for natural logarithms subscript H denotes the high confidence limit for the forecast subscript L denotes the low confidence limit for the logarithm of the index

Analogously, the PFC = 1 formula for IEAC is transformed for statistical forecasting of cost:

IEAC = AC + BAC – AC · CPI

METHODOLOGY EVM data from 16 projects are included in the study. The project data comes from three sources and is highly varied: two projects are information technology; twelve come from high technology product development; two are construction type projects. The projects range in duration from a few months to several years. There is no indication in the data of reserves for cost or duration. Although it cannot be verified with certainty, it is believed the projects have not undergone re-planning. The use of projects void of re-planning and other anomalies such as stop work and planned down time, enables a cleaner, less encumbered evaluation of the study results. Disturbances such as these impact the computations and the subsequent analysis.

Utilizing the PF = 1 formulas derived for IEAC(t) and IEAC, the nominal and confidence limit forecasts are computed for each project. The forecasts are then analyzed utilizing four hypothesis tests, two each for schedule and cost forecasts. The hypothesis test applied is the Sign Test [NIST, 2017]. The test is made for the null hypothesis, identified as Ho. When there is insufficient statistical evidence to support Ho, the test result is the alternate hypothesis, Ha.

The four hypothesis tests for evaluating the forecast confidence limits, expressed in the form of the alternate hypothesis, are defined below: 1. H1: Final Cost is less than IEACH 2. H2: Final Cost is greater than IEACL 3. H3: Final Duration is less than IEAC(t)H 4. H4: Final Duration is greater than IEAC(t)L

It should be clear from the test definitions that the testing determines the likelihood that the outcome value (final cost or final duration, as appropriate) resides between the computed forecast confidence limits. Should the testing indicate the final value is likely outside of the confidence limits, the statistical forecast is not considered reliable.

For each of the four tests, the test statistic is computed and compared to a significance level (α) equal to 0.05.7 When the test statistic value is less than or equal to 0.05, there is enough evidence to reject the null hypothesis. The test statistic for the Sign Test is computed using the binomial distribution with each trial having a success probability of 0.5.

RESULTS/ANALYSIS To verify that duration forecasting formula PFS = 1 produces, generally, better results than PFS = SPI(t), Mean Absolute Percentage Error (MAPE)8 calculations were made. As observed in table 2, of the 16 projects, the deterministic forecasts using formula PFS = 1 had lower error for 12.

Recognizing that the PFS = 1 forecast is not always better, a limited investigation was made to see if a combination of the two methods would yield results having less error. For this, attention was shifted to cost forecasting.

7) Complete descriptions of the terms “test statistic” and “significance level” are available in mathematics books of statistics [Crowe, et al, 1960]. 8) MAPE = 1/n – (∑ |AD – Forecast(i)|/AD), where Forecast(i) is one of the n forecasts.

The Measurable News 2017.04 | mycpm.org 21

Sign Test is computed using the binomial distribution with each trial having a success probability of 0.5.

Results/Analysis To verify that duration forecasting formula PFS = 1 produces, generally, better results 8 than PFS = SPI(t), Mean Absolute Percentage Error (MAPE) calculations were made. As observed in table 2, of the 16 projects, the deterministic forecasts using formula PFS = 1 had lower error for 12. Recognizing that the PFS = 1 forecast is not always better, a limited investigation was made to see if a combination of the two methods would yield results having less error. For this, attention was shifted to cost forecasting. To begin, recall Dr. Chistensen’s observation, cited earlier, that CPI generally To begin, recall Dr. Chistensen’s observation, cited earlier, that CPI generally decreases as decreases as the project progresses. If this is true then it follows that, at some point in the project progresses. If this is true then it follows that, at some point in project completion, project completion, index forecasting should converge to the final outcome faster than index forecasting should converge to the final outcome faster than PF = 1. As well, when CPI or SPI(t)PF = 1. forecasting As well, when is used,CPI or it SPI(t) is commonly forecasting observed is used, it that is commonly computed observed results that are volatile early in computedproject execution. results are Involatile fact, earlymany in analysts project execution. discount In the fact, first many 15-20 analysts percent discount of the executionthe becausefirst 15-20 they percent believe of the the execution EVM and because ES indicators they believe are the not EVM reliable and ESenough indicators for making managementare not decisions.reliable enough Thus, for it making is reasonable management to believe decisions. PF = Thus, 1 forecasting it is reasonable should to be superior in the earlybelieve stages PF = of1 forecasting the project. should be superior in the early stages of the project.

TableTable 2. Comparison ofof Forecasting AccuracyAccuracy

With these twoWith thoughts, these two consideration thoughts, consideration was given was to creatinggiven to creating a composite a composite forecast using both forecastingforecast using formulas: both forecasting PF = 1 for formulas: the initial PF two-thirds = 1 for the ofinitial project two-thirds performance, of project with PF = indexperformance for the final, with third.PF = indexAfter for several the final trials, third. the After composite several trials, approach the composite did not produce improvedapproach forecasts. did notAlthough produce not improved nearly forecasts. as comprehensive Although not as nearly the B&V as comprehensive study, the investigation as corroborated their finding of PF = 1 performing well in every partition of project completion. the B&V study, the investigation corroborated their finding of PF = 1 performing well in

every partition of project completion. Having established for the 16 projects that PFS = 1 generally provides the superior forecast, it was thoughtHaving that statisticalestablished forecasting for the 16 projects may, likewise, that PFS show= 1 generally improvement provides in the comparison to the indexsuperiorresults method. for forecast CLs computedThe, it was tabulation thought at 90 percent thatof the statistical confidencehypothesis forecasting level test are results may, prese likewise, fornted CLs in showtable computed 3. To at 90 percentimprovementassist confidence with interpreting in level comparison are the presented results, to the indexrecall in table method.the general 3. To The assistmeaning tabulation with of interpretingofHo the and hypothesis Ha: the testresults, recall the general meaning of Ho and Ha: 1) Ha indicates the confidence limit encapsulates the final outcome 1. 8 Ha indicates the confidence limit encapsulates the final outcome MAPE2) =Ho 1/n indicates • ( |AD – tForecast(i)|/AD)he outcome lies, where outside Forecast(i) of the is confidence one of the n forecasts.limit 2. Ho indicates the outcome lies outside of the confidence limit 5 Examining theExamining table, it isthe readily table, itseen: is readily the lowseen: CL the encapsulates low CL encapsulates the final the final outcomeoutcome for both for costboth andcost schedule,and schedule, whereas whereas high high CL CL generally generally does does not. TheThe low low CL CL for cost hadfor the cost test had result the test Ha result for all Ha 16 for projects, all 16 projects, while forwhile schedule for schedule the thelow low CL CL was was observed for 14 projects.observed For for 14the projects. cost high For CL, the 15cost of highthe CL,16 projects 15 of the indicate 16 projects the indicate test result the test Ho. For schedule,result 9 of Ho. 16 Forhave schedule, Ho results 9 of 16 have Ho results

TableTable 3. Hypothesis Test ResultsResults

These results areThese tabulated results are as tabulatedprobabilities as probabilities and shown and in showntable 4.in tableThe numbers4. The numbers in the table indicatein the the probabilitytable indicate that the theprobability confidence that the limit confidence encapsulates limit encapsulates the final thevalue. final Thevalue. results shown forThe PF results = CPI shown and SPI(t)for PF come= CPI andfrom SPI(t) a previous come from study a previous [Lipke studyet al, [Lipke2009-3]. et al, As readily seen, statistical2009-3]. Asforecasting readily seen, using statistical the indexes forecasting produces using theconsiderably indexes produces more reliableconsiderably CLs. more reliable CLs.

22 Table 4. Comparison Theof Confidence Measurable Limit News Probability 2017.04 | mycpm.org

results for CLs computed at 90 percent confidence level are presented in table 3. To assist with interpreting the results, recall the general meaning of Ho and Ha:

1) Ha indicates the confidence limit encapsulates the final outcome 2) Ho indicates the outcome lies outside of the confidence limit

Examining the table, it is readily seen: the low CL encapsulates the final outcome for both cost and schedule, whereas high CL generally does not. The low CL for cost had the test result Ha for all 16 projects, while for schedule the low CL was observed for 14 projects. For the cost high CL, 15 of the 16 projects indicate the test result Ho. For schedule, 9 of 16 have Ho results

Table 3. Hypothesis Test Results

These results are tabulated as probabilities and shown in table 4. The numbers in the table indicate the probability that the confidence limit encapsulates the final value. The results shown for PF = CPI and SPI(t) come from a previous study [Lipke et al, 2009-3]. As readily seen, statistical forecasting using the indexes produces considerably more reliable CLs.

In the previousTable study 4.4. Comparison(PFComparison = index) of of, Confidence98Confidence percent LimitLimitconfidence ProbabilityProbability level was examined with the following resulting probabilities: Cost High CL = 0.927, Low CL = 1.000; In theSchedule previous High study CL (PF= 1.000, = index), Low 98CL percent = 0.997 confidence [Lipke et al, level 2009 -was3]. Theexamined consistency with the of the following resulting probabilities: Cost High CL = 0.927, Low CL = 1.000; Schedule High CL = 1.000,probability Low CL values = 0.997 indicates [Lipke theet al, CLs 2009-3]. are very The reliable. consistency For the of present the probability study (PF values = 1)6 , indicatesincreasing the CLsconfidence are very l evelreliable. did notFor causethe present appreciable study (PFincrease = 1), increasing in probability. confidence Thus, it islevel didreasoned not cause the appreciable PF = 1 statistica increasel forecasting in probability. is unreliable. Thus, it is reasoned the PF = 1 statistical forecasting is unreliable.

Figure 1. Cost Forecast, PFC = CPI

Figure 2. Cost Forecast, PFC = 1

From theseFrom results these it appears results it the appears CLs from the PFCLs = from1 forecasting PF = 1 forecasting are optimistically are optimistically biased. Visually,biased. this Visually, can be thisdeduced can be from deduced graphs from for costgraphs and for schedule, cost and comparing schedule, thecomparing statistical the forecasts from each computation method. Figures 1 and 2 clearly illustrate the optimistic bias of forecasting using PFC = 1, as well as showing PFC = CPI forecasting yields more reliable CLs. 7

The Measurable News 2017.04 | mycpm.org 23

statistical forecasts from each computation method. Figures 1 and 2 clearly illustrate the optimistic bias of forecasting using PFC = 1, as well as showing PFC = CPI forecasting yields more reliable CLs.

Figure 3. Schedule Forecast, PFS = SPI(t)

Figure 4. Schedule Forecast, PFS = 1 Similar results are obtained for duration forecasting, using the same EVM data as was used for the costSimilar graphs, results Figure are 3 obtained illustrates for PF durationS = SPI(t) forecasting, statistical forecasting,using the same while EVM figure data 4 as showswas PFusedS = for1. In the general, cost graphs, both graphs Figure have 3 illustrates plots portraying PFS = SPI(t) optimistic statistical forecasts. forecasting, However, the high CL for the PFS = SPI(t) does satisfy the hypothesis test and graphically shows a while figure 4 shows PFS = 1. In general, both graphs have plots portraying optimistic feature seen consistently. There is a graphical component to interpreting the statistical forecastsforecasts. produced However, from the PF high = index. CL for It theis observed PFS = SPI(t) in both does the satisfy cost andthe hypothesisschedule graphs; test the mostand horizontal graphically plot shows is generally a feature a veryseen good consistentl forecasty. Thereof the isactual a graphical outcome. component For the index to graphs,interpreting figures the 1 andstatistical 3, the forecastsmost horizontal produced plots from are PFthe = nominalindex. It forecast is observed for cost in both and thehigh CL for schedule. 8 OBSERVATIONS Some exploration was made with notional data. The objective was to see if there is something generally true about the two forecasting methods, PF = 1 and PF = index. If a characteristic could be discovered, then possibly project managers would have information as to when a particular forecasting formula should be applied.

The exploration was not very structured. Nevertheless, it did show that when the index is constant, this method of forecasting is superior to PF = 1. However, as the variation in performance increased, PF = 1 became the more accurate. Possibly, this is an area for future study. There may be a variation value which demarcates regions for which each forecasting PF produces its best forecasts.

This observation about variation led to reflection on how organizations handle EVM data. Non- recognition of re-plans, stop work, and down time can inflate index variation, thereby causing index forecasting to appear worse than it should.

24 The Measurable News 2017.04 | mycpm.org

cost and schedule graphs; the most horizontal plot is generally a very good forecast of the actual outcome. For the index graphs, figures 1 and 3, the most horizontal plots are the nominal forecast for cost and high CL for schedule.

Observations Some exploration was made with notional data. The objective was to see if there is something generally true about the two forecasting methods, PF = 1 and PF = index. If a characteristic could be discovered, then possibly project managers would have information as to when a particular forecasting formula should be applied. The exploration was not very structured. Nevertheless, it did show that when the index is constant, this method of forecasting is superior to PF = 1. However, as the variation in performance increased, PF = 1 became the more accurate. Possibly, this is an area for future study. There may be a variation value which demarcates regions for which each forecasting PF produces its best forecasts. This observation about variation led to reflection on how organizations handle EVM data. Non recognition of re-plans, stop work, and down time can inflate index variation, thereby causing index forecasting to appear worse than it should.

To illustrate this problem, aTable notional 5.5. NotionalNotional set of DatadataData was created. It is shown in table To illustrate5. The PV, this EV, problem, and ES a datanotional have set five of highlighted data was created. entries. ItEach is shown of these in table entries 5. The is a PV, EV, andrepeat ES data of havethe e ntryfive just highlighted prior. If all entries. of the highlightedEach of these entries entries were is a removed,repeat of the plannedentry9 just prior.duration If all of would the highlighted be 10 periods entries with were project removed, completion the planned occurring duration in period would 20. Itbe is 10 fairly periods witheasy project to deduce completion with the occurring yellow entriesin period removed 20. It is that fairly SPI(t) easy =to 0.5 deduce and has with no the variation. yellow In entries removed that SPI(t) = 0.5 and has no variation. In this circumstance the index forecast of finalthis circumstance duration is better the index than fortheecast PF = of1 forecast. final duration is better than the PF = 1 forecast. Now, let’s consider what these entries might be. Possibly each is a re-plan. Or, it Now,could let’s be consider that each what of thethese yellow entries PV might entries be. is Possiblyplanned eachdown is time. a re-plan. Then, Or, when it could the down be thattime each occurred of the yellow, conditions PV entries were issuch planned that it down was nottime. possible Then, whento accomplish the down work time and occurred,, conditions were such that it was not possible to accomplish work and, thus, EV did not thus, EV did not progress. When EV does not increase, neither does ES. For the progress. When EV does not increase, neither does ES. For the remainder of the discussion, let’sremainder assume the of theentries discussion, describe l et’sdown assume time and the stopentries work. describe down time and stop work.

Figure 5. Notional Data Forecasting Comparison

In the table,In therethe table, are threethere deterministicare three deterministic duration forecasts: duration forecasts:PD/SPI(t) CPD/SPI(t), AT + (PDC, –AT ES), + and(PD IEAC(t)sp. – ES), and As eachIEAC(t)sp. method As is each discussed method it mayis discussed be helpful it mayto view be helpfulfigure to5. viewThe figurefigure graphically portrays the performance of the three methods. 5. The figure graphically portrays the performance of the three methods. The PD/SPI(t)C forecast is made by simply using the data strings of PV and EV without regard to seeing a need for further review of the highlighted entries. The The Measurable News 2017.04 | mycpm.org 25

The PD/SPI(t)C forecast is made by simply using the data strings of PV and EV without regard to seeing a need for further review of the highlighted entries. The consequence is the forecast values are erratic, yet the calculation converges to the actual duration.

As well, the forecast method, AT + (PD – ES), does not examine the highlighted entries and uses the ES calculated values to make forecasts. It, too, converges to the actual duration. One observation is these forecasts are consistently optimistic.

Lastly, the IEAC(t)sp forecasts, a modified form of the index method, perfectly align with the final duration. These forecasts are made using the ES Calculator (Special Cases)9. This calculator takes into account down time and stop work. It filters through the interruptions to make a better forecast. For this example, the special cases calculator provided forecasting perfection; in general, improvement is expected when the conditions of down time and stop work exist, but not perfection.

The take-away from this exercise is that real EVM data used in testing forecasting methods needs close examination. If at all possible, data having re-plans should be avoided. For projects having down time and stop work, the places in the data where they occur need identification so that they can be handled appropriately.

SUMMARY/CONCLUSION Forecasts of project duration were made using real data from 16 projects. The forecasts using performance factors, SPI(t) and PFS = 1, were compared using MAPE values; 12 of the 16 forecasts made with PFS = 1 were observed to have less error with respect to the final duration. This result is in agreement with the finding stated by B&V [Bastelier et al, 2015]; i.e., forecasts using PFS = 1 are generally better. As well, a very limited examination confirmed the B&V finding that PF = 1 performs well throughout the project.

With confirmation that PF = 1 forecasts generally produce more accurate results, its use in statistical forecasting was explored. The examination revealed that the associated confidence limits are unreliable for both cost and schedule. The CLs are optimistically skewed. Thus, statistical forecasting with PF = 1 is not recommended.

Duration forecasting comparison was made using notional data which included down time and stop work. Three methods were compared; two ignoring the conditions and one recognizing them. The index method, PD/SPI(t), provided highly volatile pessimistic forecasts. The PFS = 1 method was less volatile and consistently optimistic. The method recognizing the conditions, IEAC(t)sp, yielded an accurate forecast. The significant point derived from the exercise is real data needs to be closely examined and used appropriately when performing forecasting studies. Otherwise, the study results are suspect.

SUGGESTED RESEARCH In the limited investigations of this study it was observed, when the index is reasonably constant, the deterministic forecasts were better than those made with PFS = 1. Thus, there may be a demarcation value for the variation of ln SPI(t)P identifying which forecasting method should be applied; i.e., below a specific value of variation the index method is used and above it, PFS = 1 is preferred. It is suggested to researchers that this area be investigated.

At present, the application of Earned Schedule-Longest Path (ES-LP) forecasting has not been sufficiently tested. Possibly, the various forecasting formulas could be used with ES-LP to explore further improvements to forecasting.

9) The Earned Schedule Calculator (Special Cases) is available from the Earned Schedule website (www.earnedschedule.com).

26 The Measurable News 2017.04 | mycpm.org REFERENCES: Batselier, J., M. Vanhoucke (2015). “Empirical Evaluation of Earned Value Management Forecasting Accuracy for Time and Cost,” Journal of Construction Management, 141(11): 05015010, 1-13

Christensen, D. S., D. A. Rees (2002-1). “Is the CPI-Based EAC A Lower Bound to the Final Cost of Post A-12 Contracts?,” Journal of Cost Analysis and Management, Winter: 55-65.

Christensen, D. S., C. Templin (2002-2). “EAC Evaluation Methods: Do They Still Work?,” Acquisition Review Quarterly, Spring: 105-116

Christensen, D. S., R. C. Antolini, J. W. McKinney (1995). “A Review of Estimate At Completion Research,” Journal of Cost Analysis and Management, Spring: 41-62.

Christensen, David S (1993-1). “The Estimate At Completion Problem: A Review of Three Studies.” Project Management Journal, Vol 24 March: 37-42.

Christensen, D. S., S. R. Heise (1993-2). “Cost Performance Index Stability,” National Contract Management Journal, Vol 25: 7-15.

Crowe, E., F. Davis, M. Maxfield (1960). Statistics Manual, New York, NY: Dover Publications

Crumrine, K., J. Ritschel (2013). “A Comparison of Earned Value Management as Schedule Predictors on DOD ACAT 1 Programs,” The Measurable News, Issue 2: 37-44.

Lipke, W (2017). “Forecasting Schedule Variance using Earned Schedule,” PM World Journal (online), February, Vol VI, Issue 2

Lipke, W. (2016). “Earned Value Management and Earned Schedule Performance Indexes,” Wiley StatsRef: Statistics Reference Online, August, DOI:10.1002/9781118445112.stat07891

Lipke, W. (2015). “Applying Statistical Forecasting of Project Duration to Earned Schedule-Longest Path,” The Measurable News, Issue 2: 31-38

Lipke, W. (2014). “Testing Earned Schedule Forecasting Reliability,” PM World Journal (online), July, Vol III, Issue 7

Lipke, W. (2012). “Speculations on Project Duration Forecasting,” The Measurable News, Issue 3:1, 4-7

Lipke, W. (2011). “Earned Schedule Application to Small Projects,” The Measurable News, Issue 2: 25-31

Lipke, W. (2009-1). “Project Duration Forecasting …a comparison of Earned Value Management methods to Earned Schedule,” The Measurable News, Issue 2: 24-31.

Lipke, W. (2009-2). Earned Schedule, Raleigh, NC: Lulu Publishing

Lipke, W., O. Zwikael, K. Henderson, F. Anbari (2009-3). “Prediction of Project Outcome The Application of Statistical Methods to Earned Value Management and Earned Schedule Performance Indexes,” International Journal of Project Management, May Vol 27: 400-407

Lipke, W. (2006). “Applying Earned Schedule to Critical Path Analysis and More,” The Measurable News, Fall: 26-30.

National Institute of Standards and Technology. Sign Test. (2017) http://www.itl.nist.gov/div898/software/dataplot/refman1/auxillar/signtest.htm

Project Management Institute. Practice Standard for Earned Value Management, Newtown Square, PA.: PMI 2011

Vanhoucke, M., S. Vandevoorde (2007). “A Simulation and Evaluation of Earned Value Metrics to Forecast Project Duration,” Journal of Operational Research Society, Issue 10 Vol 58: 1361-1374.

Vanhoucke, M., S. Vandevoorde (2006). “A Comparison of Different Project Duration Forecasting Methods Using Earned Value Metrics,” International Journal of Project Management, Issue 4 Vol 24: 289-302.

About the Author: Walt Lipke retired in 2005 as deputy chief of the Software Division at Tinker Air Force Base, where he led the organization to the 1999 SEI/IEEE award for Software Process Achievement. He is the creator of the Earned Schedule technique, which extracts schedule information from earned value data.

Credentials & Honors: • Master of Science Physics • EVM Europe Award (2013) • Licensed Professional Engineer • CPM Driessnack Award (2014) • Graduate of DOD Program Management Course • Australian Project Governance and Control • Physics honor society - Sigma Pi Sigma (SPS) Symposium established the annual Walt Lipke • Academic honors - Phi Kappa Phi (FKF) Project Governance and Control Excellence • PMI Metrics SIG Scholar Award (2007) Award (2017) • PMI Eric Jenett Award (2007)

The Measurable News 2017.04 | mycpm.org 27 The absolute go-Thtoe c aobmspoaluntey f goor -effectto comivpe apnryog foramr effect planniveing pr og& cramont rpolla snnoliungtion & sc onfort rtohle s molousttion cosm foprlex th e most complex industry programsindustry programs  Program Planning – PPrroopograsma l,P Slanntartiupng, –SuPrrgopoe, Rsaepl, lSantasrtup, Surge, Replans  Program Control SysPteromg rSeamtup Con & tOropel Sryastitonems Setup & Operations  Program Cost ManagPeromgeranmt Cost Management  Efficient Earned ValueEf fMicanaient gEaemrneendt (VEValueM) MCanaompglieamnecent (EVM) Compliance  Program Control TooPlsr o&g rTaemchno Conlotgroyl Tools & Technology

ClearPlan has buCillte athrPe lsatnro hnagse bstu iltet amthe osft rexonpgeertsts itne amthe o inf dexupsterryts in the industry  Our experts in prograOmu rc eonxpetrorltss mina pkreo ganra mimm conedtriaotles mimapakec tan to immyoured teaiatme impact to your team  We bring ensured coWmep libarnincge enwhensured you co needmplia int cmeo wsthen you need it most Nationwide Team oNf aEtixpoenrwtside Team of Experts  Tell the clear, winn ingTe sllt othery icnl eayour, rw pinnropoings asltory in your proposal Cost Eﬀective SolutiCoonsst Eﬀective Solutions Contact Us: Contact Us: www.clearplanconwwsulwti.cngle.caorpmlanconsulting.com info@[email protected] nconsulting.com 888.249.7990 888.249.7990

EVMLibrary.org: The CPM Online Library

A comprehensive, online library of resources about Earned Value Management (EVM) and related project management topics.

The CPM library now contains well over 2,100 documents, such as articles and presentations, and is growing everyday.

Each document has been catalogued by subject area, author, publication date, keywords, etc.

How You Can Contribute? Do you have topical articles, speeches, or presentations?

Contact Don Kaiser at [email protected] to have your contributions included in this online resource. THE MEASURABLE NEWS 2017.04

TRUST, BUT VERIFY: AN IMPROVED ESTIMATING TECHNIQUE USING THE INTEGRATED MASTER SCHEDULE (IMS) By Eric M. Lofgren — Technomics, Inc.

ABSTRACT “Trust, but verify is a form of advice given which recommends that while a source of information might be considered reliable, one should perform additional research to verify that such information is accurate, or trustworthy. The original Russian proverb is a short rhyme which states, Доверяй, но проверяй (doveryai, no proveryai).”

It has long been the wonder of management why the Integrated Master Schedule (IMS) fails to give clear and advanced warning of impending schedule delays. An estimator may follow authoritative guidance in an analysis of schedule health using key metrics, supposing that such checks authenticate schedule realism. Why, then, do practitioners find themselves caught off guard by slips when their IMS appears to be in good health? Answers to the question follow from observing the evolution of the IMS over the course of its submissions. By independently tracing activities across IMS submission, this article will show how an analysis of performance to the original baseline can improve predictions of project end dates.1

BACKGROUND

The Integrated Master Schedule is more than a tool for managing the time-phasing and resource allocation of project activities. Decision makers find value in an IMS for its ability to evaluate the impacts of schedule risk and provide early warning of schedule slip. For many Major Defense Acquisition Program (MDAP) contracts, the IMS has a questionable history of accurately reflecting schedule risk. In an analysis of eight MDAP contracts,2 schedule slips were not apparent until late in the project. It can be seen in the figure below that IMS data often fails to indicate any delay until after half-way through the original schedule duration. In fact, as a project approaches its expected end date, further delays develop resulting in a tail chase.3 The situation gives decision makers minimal leeway for managing trade-offs and implementing well-informed strategies.

Predicted schedule slip reported by IMS submission Predicted schedule slip reported by IMS submission

Observations from actual IMS data reveal the importance of comparing the most recent IMS The Quarterly Magazine of the Collegesubmission of Performance to its initial. Management For example, |poor mycpm.org schedule performance can be masked by modifying29 the baseline. Within the sample IMS data, more than a quarter of all activities had changes to their baseline end dates within the first year of execution. Many more were dropped or were otherwise unidentifiable. Those with baseline changes within the first year experienced an average of three months slip. Such “rubber” baselines hinder the schedule’s predictive power. Another way poor schedule performance can be masked is by injecting optimism into future activities. For example, even though actual task completion may have been poor, the schedule can appear on-time if a surge in task completions is forecasted. Sample data suggests this form of optimism is unfounded. The proportion of tasks finishing late generally continues to increase until the project finishes. The sample IMS data shows task performance degrades over the course of a project. A linear approximation indicates that an additional 5% of near-term tasks will finish late to the current baseline for every 10% of project duration that passes. Another way optimism can be injected is using hard constraints. The sample IMS data shows that many schedules constrain more than 5%, and up to 20%, of open activities. Of those activities with constraints, generally between half and all of them are binding. Binding means that the forecast start or end date equals the constraint date. When a constraint binds an activity, it is not driven by its predecessors’ performance and may no longer cause slip to the project schedule.

Observations from actual IMS data reveal the importance of comparing the most recent IMS submission to its initial. For example, poor schedule performance can be masked by modifying the baseline. Within the sample IMS data, more than a quarter of all activities had changes to their baseline end dates within the first year of execution. Many more were dropped or were otherwise unidentifiable. Those with baseline changes within the first year experienced an average of three months slip. Such “rubber” baselines hinder the schedule’s predictive power. Another way poor schedule performance can be masked is by injecting optimism into future activities. For example, even though actual task completion may have been poor, the schedule can appear on-time if a surge in task completions is forecasted. Sample data suggests this form of optimism is unfounded. The proportion of tasks finishing late generally continues to increase until the project finishes. The sample IMS data shows task performance degrades over the course of a project. A linear approximation indicates that an additional 5% of near-term tasks will finish late to the current baseline for every 10% of project duration that passes. Another way optimism can be injected is using hard constraints. The sample IMS data shows that many schedules constrain more than 5%, and up to 20%, of open activities. Of those activities with constraints, generally between half and all of them are binding. Binding means that the forecast start or end date equals the constraint date. When a constraint binds an activity, it is not driven by its predecessors’ performance and may no longer cause slip to the project schedule.

Maybe the most important concept in schedule construction and maintenance is logic: the idea that every activity must have at least one predecessor and one successor activity. Logic drives the sequencing of activities and interconnects the schedule so that a change to any one given activity ripples across all successors. Not one of the schedules from the sample dataset saw less than 5% of near-term tasks missing logic. Generally, 10% or more of near- term tasks were missing both predecessor and successor logic links. The rate was between 10% and 20% early in the project; half way through many projects were missing upwards of 60% of their logic links, some climbing to 80% by project’s end.

Observations of sample IMS data show that, in general, projects rely on forecasts where expectations of task performance become increasingly detached from actuals. Not only are forecasts biased toward undue optimism, but schedule quality tends to decrease significantly over time leading to schedules with fewer useful interrelationships between activities. Guidance from most authoritative sources does not speak to the trends and patterns in IMS submissions over time.4 Guidance has focused on gauging the quality of the most recent submission because only the latest IMS incorporates up-to-date information, making the IMS a living document. The inadequacy of status quo metrics stems from the perceived irrelevancy of prior IMS submissions. The metrics are based on a trust that schedules have rigorously maintained a baseline, but little has been said on how to verify that rigor through a cross-IMS analysis.

Without a point of reference to ensure logical evolution, the current IMS can only say so much. It is important to understand, for example, what baseline changes have occurred over time and how actual performance has been measured to near-term plans. While schedules are living documents, the original baseline from the initial IMS stands as the best available point of reference. The original baseline is valid for three major reasons. First, planners tend to know the major activities involved in the execution of a project. All systems can be said to have historical analogies, even those considered revolutionary. Second, contractors generally have well-defined processes for developing these systems. Third, the IMS undergoes an Integrated Baseline Review (IBR) after which both the contractor and client agree to the plan and its efficacy. Thus, the IMS from its outset may be viewed by estimators primarily as a tool for measuring the scheduler’s ability to plan in the near-term.

This article starts from an acceptable assumption—that an activity’s baseline from the first IMS is relevant through subsequent submissions. Planners generally do a good job of laying out major activities, and so early performance on near-term activities should be a good indicator of total schedule realism. Practitioners have often heard anecdotally that early schedule slip cannot be made up despite managerial tactics. This article will explore that notion and whether it finds support in the data. It will be shown that by tracing near-term activities through subsequent IMS submissions and comparing them to their original baseline, as opposed to the current baseline, one may extrapolate a more realistic contract end date far earlier in the project.

30 The Measurable News 2017.04 | mycpm.org The Technique

THE TECHNIQUEThe preceding section has shown that, in general, schedule quality and performance tend to The precedingdecrease sectionover the course has shown of a contract. that, A in high general, number schedule of binding constraintsquality and and performanceincreasing number tend to decreaseof late over tasks the result course in scheduleof a contract. optimism. A high Numerous number ba selineof binding changes constraints can conceal and poor increasingperformance. number Fewerof late logic tasks links result indicate in aschedule poorly maintained optimism. schedule. Numerous These generalizations baseline changes have can conceal poor performance. Fewer logic links indicate a poorly maintained schedule. These a negative impact on schedule realism. Even a high quality rating for any given IMS does not mean generalizations have a negative impact on schedule realism. Even a high quality rating for any giventhat IMS it has does also evolvednot mean in a consistentthat it has manner. also evolvedThe relevant in aquestion consistent becomes manner., “Can the The IMS relevant data questionon becomes, hand be better “Can utilized the IMSto measure data onschedule hand risk be andbetter extrapolate utilized a realisticto measure end da schedulete far earlier risk and extrapolatein the project?” a realistic end date far earlier in the project?”

The lynchpinThe of lynchpinthe proposed of the proposed schedule schedule estimating estimating technique technique is is the the baseline in in the the original original IMS submission.IMS submission The first. The firstavailable available IMS, IMS, preferably preferably thethe one one immediately immediately following following IBR, will IBR, be will be usedused to independently to independently tracktrack activities activities over over subsequent subsequent submissions. submissions. Activity end Activity dates are end traced dates are tracedover over time usingtime theirusing designated their designated activity names activity and/or uniquenames IDs. and/or The updated unique end IDs. dates The will updated then end datesbe comparedwill then to be their compared original baseline to their end dateoriginal and total baseline float (slack end days date).5 Theand process total floatdisregards: (slack days).5 The process disregards: new activities which get built into the schedule over time; new activities which get built into the schedule over time; activities which change identifiers; and activities which change identifiers; and modifications to activity leads, lags, constraints, and modifications to activity leads, lags, constraints, and sequencing. While the omissions appear sequencing. While the omissions appear concerning, remember, the primary purpose is to trace performanceconcerning, remember of the relatively, the primary near-term purpose is baseline to trace performance plan. Both of contractual the relatively parties near-term agreed to the baselinebaseline plan.IMS Bwhichoth contractual often takes parties months agreed to thedevelop. baseline That IMS whichsome oftenactivities takes monthscannot to be traced (fordevelop example,. That so ame task activities broken cannot up intobe traced three (for new, example smaller, a task tasks) broken is simply up into athree reality new, insofar as theresmal is noler recordtasks) is ofsimply the achanges. reality insofar as there is no record of the changes.

To determineTo schedule determine scheduleslip, the slip, analyst the analyst looks looks across across all alltasks tasks still still identifiable identifiable in insubsequent subsequent IMS submissionsIMS submissions and compares and compares them them to to their their originaloriginal baseline.baseline. The The activity activity which which has slipped has slipped most tomost that to baseline that baseline (in (interms terms of of business business days) days),, with with the originalthe original days of days float factoredof float in, factored drives in, drives the independently predicted end date. That number of days slip is added to the the independently predicted end date. That number of days slip is added to the baseline end date baseline end date for the project’s close-out activity. for the project’s close-out activity.

When the schedule has evolved to the extent that the current activities no longer reflect the original Whenbaseline, the schedule the predicted has evolved end to datethe extent stabilizes that the and current the activities estimate no islonger determined. reflect the This occurs becauseoriginal baseline, the tasks the predictedin the baseline end date IMS stabilizes have andtheir the end estimate dates is realizeddetermined and. This the occurs planning packagesbecause open the up tasks into in newthe baseline activities. IMS haveThe their rule end implemented dates realized byand thethe planning author packagesis to stop open the analysisupschedule. when into new more The activities than final .95%prediction The ruleof theimplemented for activities project byend from the is author thenthe originalis set. to stop An examplethebaseline analysis wi are llwhen illustrateeither more finished than this or no longer95%methodology. appear of the activities in the currentfrom the originalschedule. baseline The are final either prediction finished or nofor longer project appear end in theis thencurrent set. An example will illustrate this methodology. The first IMS sets the baseline for future activities. All activity end dates will be evaluated The first IMS sets the baseline for future activities. All activity end dates will be evaluated for each subsequentfor each subsequent IMS and IMS compared and compared to their to their baseline. baseline. In In the the case of of IMS IMS #2, #2, several several activities activities4 have experienced have experienced slips slipsto their to their forecast forecast end end dates. dates. ForFor exampleexample,, “Frame “Frame first first floor floor walls” haswalls” has slipped slipped4 business 4 business days days while while it only it only had had 3 3days days of float available. available. This This means means that the that task the will task will affect theaffect start the startof all of its all itssuccessor successor tasks tasks andand push push out outthe milestonethe milestone “Framing “Framing complete” complete” by 1 day, by 1 day, all elseall else equal. equal .In In IMS IMS #3, #3, “Install roof roof decking” decking” has slippedhas slipped 7 days 7to daysbaseline, to baseline,and factoring and in itsfactoring in its originaloriginal 3 3 business business days days of offloat, float, should should now affect now “Framing affect complete”“Framing by complete” 4 days. by 4 days.

AA simplified simplified exampleexample ofof an an evolving evolving IMS IMS

How has “Framing complete” been able to stay on track? In the example above, the forecasted tasks have had their durations squeezed to make up time. In the sample from actual IMS data, forecasted discrete tasks do not regularly show shorter durations than past tasks. A plausible reason is that the long-term future is represented by planning packages which incorporate many tasks. When opened, the resulting tasks are given realistic durations and distribute the implicit duration The Measurable News 2017.04 | mycpm.orgsqueeze onto planning packages still further out. Although evidence cannot be concretely pulled 31 from the data, it has been shown that forecasted performance is optimistic and there are numerous binding constraints. Such behavior is sufficient to maintain a fixed near-term schedule, even if it is unrealistic in the long-term. The total schedule might be saved because of the future’s vagueness. As time progresses, the pool of undetailed tasks dwindles and increasingly focuses implicit duration compression. By the time the schedule’s unrealism is noted—possibly by pure reckoning—it would be relatively late in the project.

Results

The proposed technique disregards over-optimism by relying on early task performance to the original baseline. Because major activities are reasonably phased from the start, the total schedule receives a one-for-one slip with the worst performer to the original baseline. The idea becomes increasingly attractive when considering the fact that schedules tend to lose an alarming number

How has “Framing complete” been able to stay on track? In the example above, the forecasted tasks have had their durations squeezed to make up time. In the sample from actual IMS data, forecasted discrete tasks do not regularly show shorter durations than past tasks. A plausible reason is that the long-term future is represented by planning packages which incorporate many tasks. When opened, the resulting tasks are given realistic durations and distribute the implicit duration squeeze onto planning packages still further out. Although evidence cannot be concretely pulled from the data, it has been shown that forecasted performance is optimistic and there are numerous binding constraints. Such behavior is sufficient to maintain a fixed near-term schedule, even if it is unrealistic in the long-term. The total schedule might be saved because of the future’s vagueness. As time progresses, the pool of undetailed tasks dwindles and increasingly focuses implicit duration compression. By the time the schedule’s unrealism is noted—possibly by pure reckoning—it would be relatively late in the project.

RESULTS The proposed technique disregards over-optimism by relying on early task performance to the original baseline. Because major activities are reasonably phased from the start, the total schedule receives a one-for-one slip with the worst performer to the original baseline. The idea becomes increasingly attractive when considering the fact that schedules tend to lose anof alarming logic links number over time of, perhapslogic links implying over higher time, reliabilityperhaps of implying the networking higher imparted reliability on theof the networkingoriginal imparted baseline. Theon theestimation original technique baseline. is not The intended estimation to forecast technique exact dates, is notbut sintendedhould be to forecastviewed exact as dates, a rough but order should of magnitude be viewed for would as -abe rough schedule order slips .of Below, magnitude one may for see would-bethat from schedulethe slips.contractor’s Below, point one of mayview (blue)see that, the IMSfrom predict the contractor’sed less than a 10% point schedule of view slip (blue),up until nearthe IMS predicted less than a 10% schedule slip up until near 70% of the total duration had passed. 70% of the total duration had passed. Then significant delays were belatedly realized; the project Then significant delays were belatedly realized; the project eventually slipped by almost eventually slipped by almost 45% to baseline. The independent point of view, representing the 45% to baseline. The independent point of view, representing the technique proposed in this article,technique quickly proposed registered in this article schedule, quickly risk register to withined schedule about risk 10% to withinof actual about slip. 10% Note of actual that the (red) independentslip. Note that linethe (red)plateaus independent shortly line after plateau 50%s shortly of latest after schedule. 50% of latest This schedule. is where This the is current schedulewhere largely the currentreflects schedule new largelyactivities; reflects in other new activities;words, activities in other words, from activitiesthe original from IMS the have for the originalmost part IMS beenhave for completed the most part or been dropped. completed or dropped.

Results fromfrom anan actualactual contract contract The figure below illustrates the new metric’s predictive power across all eight projects. By 50% of Thethe figurelatest below schedule illustrates duration, the new the metric’s independent predictive metricpower across registered all eight a projects majority. By of realized50% schedule of the latestslip. The schedule mean duration, absolute the independenterror for the metric 41-50% register ofed schedule a majority bin of is realized only seven months;schedule compare slip. that The tomean the absolute contractor’s error for 25 the months 41-50% ofof error.schedule It equatesbin is only to se aven year-and-a-half months; of schedulecompare slip that on to average the contractor’s not yet 25 monthsrealized of error.by the It equatescontractor’s to a year IMS.-and- aIn-half fact, of schedulethe contractor slip IMS didon not average reach not a yetmean realized absolute by the errorcontractor’s of seven IMS. months In fact, the until contractor close toIMS 80% did ofnot the reach total a schedulemean had absolute elapsed. error With of seven more months than until a year close of to lead 80% time,of the thetotal improved schedule had predictions elapsed. With provides management an early warning indicator that allows them to do advanced planning for cost- more than a year of lead time, the improved predictions provides management an early warning schedule-technical tradeoffs. Early recognition of major difficulties also leads to greater flexibilityindicator in project that allows termination them to do because advanced ofplanning fewer forsunk cost costs.-schedule Schedule-technical qualitytradeoffs. as Earlytraditionally calculatedrecognition largely of ignores major difficulties the quality also leadsof the to schedule’sgreater flexibility evolution. in project termination because of fewer sunk costs. Schedule quality as traditionally calculated largely ignores the quality of the schedule’s evolution.

32 The Measurable News 2017.04 | mycpm.org

MeanMean absolute absolute error error of contractorof contractor and and independent independent schedule schedule estimates

Using the technique presented above, an analyst may catch a potential schedule slip earlier in the project with relative accuracy. While schedules are a living document and develop over time, plannersUsing the dotechnique a good presented job at above the outset, an analyst of phasingmay catch majora potential milestones schedule slip and earlier near- in term tasks.the project Though with projects relative accuracy. change, While the deliverableschedules are doesn’ta living document often transform and develop completely. over time, However,planners cumulative do a good changes job at theto outseta schedule of phasing tend major to deteriorate milestones and its nearrealism.-term tasksAs described,. Though scheduleprojects performance change, the is deliverable best registered doesn’t often early transform for a variety completely. of reasons However, including cumulative increasinglychanges optimisticto a forecasts schedule tend and to decreasingdeteriorate its quality.realism. As It descmayribed be ,concluded schedule performance that contractors is best registered quickly reveal theirearly pacefor a variety of work of reasons and “settle” including into increasingly a performance; optimistic orforecasts management and decreasing strategies quality. suchIt as work-aroundsmay be canconcluded do little that to contractors alleviate ailingquickly projects. reveal their Until pace a culture of work of and improved “settle” scheduleinto a maintenance takes root, early performance to the original baseline early may serve as the performance; or management strategies such as work-arounds can do little to alleviate ailing best indicator of realized schedule slip. projects. Until a culture of improved schedule maintenance takes root, early performance to the The independentoriginal baseline metric early finds may serveuseful as theapplication best indicator for of estimating realized schedule costs slip. as well. If the metric predicts a one year slip over that reported in the IMS, the analyst may extend the average project costThe burn independent rate out metricfor the finds same useful duration. application The for figure estimating below costs does as well. just If that the metricusing one of the real predictsprojects a one analyzed year slip inover this that article. reported The in the red IMS, line the is analystthe cost may of extend the traditional the average projectIndependent Estimatecost At burn Completion rate out for (IEAC) the same plus duration. the mean The figure actual below burn does rate just multiplied that using byone the of thenumber real of months projectsslip the analyzed independent in this article metric. The predicts red line is over the cost the of current the traditional IMS. IndependentThe new independent Estimate At cost estimateCompletion behaves (IEAC) similar plus the to mean the scheduleactual burn estimatorrate multiplie andd by gives the number a good of monthsearly indication slip the of realizedindependent costs. metric predicts over the current IMS. The new independent cost estimate behaves similar to the schedule estimator and gives a good early indication of realized costs.

Independent metric applied to cost estimation for an actual project Independent metric applied to cost estimation for an actual project Discussion

A year after the development of the estimating technique presented in this article, Shedrick Bridgeforth independently tested it along with various other schedule estimating techniques.6 He found that for a set of 12 completed defense satellite development efforts, the technique presented in this article, which he called the Independent Duration Estimate (IDE), outperformed all other EVMS-based approaches in predicting the realized schedule. Bridgeforth wrote that “These results suggest Lofgren’s approach (IDE) is the most accurate technique.” In particular, the so-called IDE approach provided more realistic estimates far earlier in the development cycle than other techniques. Bridgeforth recommends using the IDE technique for space development contracts when the data are available.

While Bridgeforth’s results support the findings in this article, he cautions that “Lofgren’s The Measurable News 2017.04 | mycpm.orgIDE framework does not consider the critical path; it considers all tasks as equally important. The 33 IDE may struggle to become a best practice because it ignores the CPM [Critical Path Method] and is relatively new.” While the technique is both new and not well recognized in the cost and schedule analysis communities, it is not completely true that the technique ignores the CPM. As explained above, the total float from the original schedule is taken into account, factoring in the logic behind the critical path put into the baseline of the first IMS. Activities in successive schedules that slip more days than they had total float in the first IMS then compete to become schedule drivers on an unofficial critical path. The technique presented here is independent precisely because it ignores changes to tasks and networking relative to the schedule’s original baseline—changes which alter the disposition of what would have otherwise been the critical path.

DISCUSSION A year after the development of the estimating technique presented in this article, Shedrick Bridgeforth independently tested it along with various other schedule estimating techniques.6 He found that for a set of 12 completed defense satellite development efforts, the technique presented in this article, which he called the Independent Duration Estimate (IDE), outperformed all other EVMS-based approaches in predicting the realized schedule. Bridgeforth wrote that “These results suggest Lofgren’s approach (IDE) is the most accurate technique.” In particular, the so-called IDE approach provided more realistic estimates far earlier in the development cycle than other techniques. Bridgeforth recommends using the IDE technique for space development contracts when the data are available.

While Bridgeforth’s results support the findings in this article, he cautions that “Lofgren’s IDE framework does not consider the critical path; it considers all tasks as equally important. The IDE may struggle to become a best practice because it ignores the CPM [Critical Path Method] and is relatively new.” While the technique is both new and not well recognized in the cost and schedule analysis communities, it is not completely true that the technique ignores the CPM. As explained above, the total float from the original schedule is taken into account, factoring in the logic behind the critical path put into the baseline of the first IMS. Activities in successive schedules that slip more days than they had total float in the first IMS then compete to become schedule drivers on an unofficial critical path. The technique presented here is independent precisely because it ignores changes to tasks and networking relative to the schedule’s original baseline—changes which alter the disposition of what would have otherwise been the critical path.

If the activities and relationships networked in the baseline schedule remained constant over time, the activities driving the IDE technique should be exactly those activities found on the contractor’s critical path in the IMS. What distorts the signal of schedule progress is the cumulative changes to the content and networking of activities. By stripping away such “noise,” which in many cases results from efforts to mitigate realized risks, the technique proposed here focuses on actual performance to baseline plan for near-term tasks. It rejects updated expectations to forecasts of task efficiency, which often has an optimistic bias. While Bridgeforth would be correct to say the technique ignores the current critical path, it does not ignore the critical path as networked into the baseline schedule. In this sense, it creates a counterfactual scenario where the baseline tasks played out as originally planned. The independent estimate is derived from the variance between the current critical path and its counterfactual.

When Bridgeforth wrote that the technique considers all tasks to be equally important, he was completely correct. In some sense, a successful project needs all tasks to come together according to some sequencing—this is the basis of the systems approach. However, actual schedules that are detailed enough for use as a management tool will contain tasks that are in no way critical to the deliverable. Numerous examples of such tasks can be imagined in detailed schedules. A casual use of the technique might result in an inconsequential task, or set of tasks, driving the independent estimate. The point brings up an important issue. The technique presented here is not intended to be a precise. For complex schedules, it is a rather blunt instrument for measuring variance to the counterfactual from baseline assumptions. The analyst must be discerning when evaluating the tasks driving the independent estimate. For example, a best practice is to line the tasks up in rank order of how much they are driving the estimate in the analysis. What is content of each task? Did they experience baseline changes to their start or end dates? What activities preceded and succeeded them, and has that network been stable or logical? However, if the IMS is volatile, having dropped and replaced many activities, then seemingly inconsequential tasks may provide some residual signal as to the realized schedule slip of important unseen tasks. The power of the proposed technique more generally is to facilitate project analysis. It is not solely to provide more accurate predictions faster; if that were the goal, then the best rule might be to immediately assume a slip of 40 percent or so. Unlike the presented technique, such a rule does not take individual project circumstances into account and provides no basis for targeting management, or for understanding the impact of changes to project assumptions.

34 The Measurable News 2017.04 | mycpm.org In some cases, there are good reasons for baseline changes, and the fact of the matter is that schedules should continue to be allowed rapid and frequent changes to baseline assumptions, particularly in large or uncertain projects. A recent and important exception to the focus on the current IMS submission is the GAO’s “Best Practices for Project Schedules,” which advocates a continuing analysis of the baseline schedule relative to the most recent instantiation of the schedule. It states in the 10th and final best practice: “The baseline schedule is not the same as the current schedule. The current schedule is updated from actual performance data… The baseline schedule represents the program’s commitments to all stakeholders, while the current schedule represents the actual plan to date.”7 Rigorous maintenance of the original baseline may be too constraining for the schedule to be useful day-to-day and may end up creating unintended consequences. However, referencing the baseline and pointing out the biggest deviations can provide a fruitful starting point from which to ask questions about whether the IMS is providing a realistic assessment of progress. In most cases, variation between the contractor’s end date and the end date from a discerning use of the presented technique should be tolerated within a certain range, reflecting a degree of trust between parties that is crucial to project success. Baseline assumptions can be wrong or non-optimal, especially at the task level, and the contractor must be allowed flexibility to exercise real options that change direction based on updated information, particularly in research and development work.8 When the production sequence is well-known and repeatable, deviations from an agreed upon baseline should receive less tolerance.

CONCLUSION As the single source of comprehensive schedule information on most projects, the IMS’s predictive ability rests on its quality. It is preferred that the IMS exhibits high quality such that the analyst can develop actionable plans using its forecast. Currently, analysts spend a great deal of effort trying to tease out the biases in schedules to generate more “realistic” forecasts. The technique presented in this article is no different. It searches for bias by independently tracking baseline activities through subsequent schedules. All such manipulation of native schedule data to support independent predictions is highly speculative. Attention to schedule maintenance is a superior use of scarce resources relative to guessing its inherent biases. Longitudinal checks of the sort described in this article would expose the need to eliminate—or to better understand—the biases that analysts attempt to exploit for the purpose of prediction.9 Oversight of this sort can go a long way to ensuring high schedule quality from the outset.

Though improving schedule quality is the first-best solution, in today’s scheduling environment there is much to be done using the author’s schedule estimating technique. First and foremost, it is important to scrutinize the tasks that drive the independent technique’s estimate. Put the drivers in rank order and pick out the outliers. The technique is one way to approach the more urgent need of longitudinal schedule analysis. Second is the need for replication studies to test the metric’s robustness. Third, to assess the value of this technique for Indefinite Delivery, Indefinite Quantity (IDIQ) contracts, a more detailed approach is required, such as analysis by task-order. Further assessment is also needed for augmentation to cost estimation. One method (extending mean burn rates) has been presented in this article, but many other approaches are possible. Finally, a large collection of IMSs used for analysis can also provide data-driven generalizations of schedule quality and realism over time. Additional insights into IMS evolution can help practitioners understand how to better manage their schedule, and provides a benchmark for evaluating relative schedule quality.

The Measurable News 2017.04 | mycpm.org 35 Endnotes 1) This article is based on a 2014 paper and presentation for the International Cost Estimating and Analysis Association (ICEAA) workshop. See Lofgren, Eric. “Trust, but Verify: An Improved Estimating Technique Using the Integrated Master Schedule (IMS),” ICEAA 2014.

2) MDAP contract data accessed through the Earned Value Management Central Repository (EVM-CR). This article reflects an analysis of all contract schedules available to the researcher. Data not included: all contracts without data spanning contract start to end and IMSs in PDF or picture format. Eight contract projects are reported. Uniform data extraction methodology used across contract schedules: extracted 19 standard fields for non-summary activities; converted into Microsoft Excel flat-files and used standard template for metrics gathering and analysis. Additional contract insight gained from Defense Acquisition Management Information Retrieval (DAMIR).

Note that the observational findings of the Background section included an additional four Indefinite Delivery, Indefinite Quantity (IDIQ) contracts. The IDIQ contracts remained in the observational findings because conclusions drawn from task hit/miss, logic, constraints, and so forth, are not affected by the IDIQ nature of the contracts. The author did not have detailed data as to which activities went to which task orders to allow for the technique to be applied to IDIQ contracts.

3) Cf. “Ending the EAC Tail Chase: An Unbiased EAC Predictor using Progress Metrics,” Eric R. Druker, et al., SCEA/ISPA, 2007.

4) Authoritative guidance includes “Earned Value Management System (EVMS) Program Analysis Pamphlet (PAP)” by Defense Contract Management Agency (DCMA), the “GAO Schedule Assessment Guide” by U.S. Government Accountability Office (GAO), and the “Joint Cost and Schedule Risk and Uncertainty Handbook” posted by the Naval Center for Cost Analysis. See additional EVMS guidance from Performance Assessment and Root Cause Analyses (PARCA) including the IMS Data Item Description (DID) and the “IPMR Implementation Guide.” The “Integrated Master Plan and Integrated Master Schedule Preparation and Use Guide” is another useful resource.

5) Total float (also called total slack) is the number of days an activity is allowed to move to the right before any further slips affect the project end date. When an activity has zero days of float, it might find itself on the “critical path,” or a sequence of activities representing the shortest duration between the schedule’s ‘as of’ date and the end of the project.

6) Shedrick Bridgeforth, Jonathan Ritschel, Edward White, and Grant Keaton. “Using Earned Value Data to Forecast the Duration of Department of Defense Space Acquisition Programs.” 11 Sep. 2015. Journal of Cost Analysis and Parametrics, Vol. 8, Issue 2, pp. 92-107. For more details, see Bridgeforth, Shedrick. “Using Earned Value Data to Forecast the Duration of Department of Defense Space Acquisition Programs.” Thesis for Air Force Institute of Technology (AFIT). http://www.dtic.mil/docs/citations/ADA615411. Accessed July 2017.

7) “Best Practices for Project Schedules,” Government Accountability Office (GAO), Dec. 2015, GAO-16-89G.

8) Flexibility in project scheduling is especially important in research and development (R&D) projects where the definition and outcome of tasks cannot be known in advance, leading to reflexive relationships between activities. Rigorous baseline maintenance may not in all cases be optimal when adaptation is required for success. For example, RAND researchers W. Meckling, B. Klein, and E. Mesthene wrote that “Any attempt to schedule an entire R&D program at one time is likely to lead to inefficiency, either because plans for the later stages may have to be scrapped and remade on the basis of information yielded by early tests, or because, in pursuing premature plans, a development program may fail to profit from new information gained along the way. Either case will cause delays, or raise costs, or both.” See “Military Research and Development Policies.” RAND Corp., 4 December 1958, pp. 3.

9) For more information on schedule quality and longitudinal checks, see Lofgren, Eric M. “Putting Schedule Checks to the Test,” ICEAA 2016.

36 The Measurable News 2017.04 | mycpm.org THE MEASURABLE NEWS 2017.04

AGILE'S EARNED SCHEDULE BASELINE By Robert Van De Velde, PhD

ABSTRACT For managing schedule performance on an Agile project, canonical Agile techniques fall short. Estimates are essential for meeting project time constraints, but Agile’s relative estimates are too “fuzzy” to meet the needs of Sprint planning. Typical Agile Burndown charts focus on Release Point counts that only suggest how well or poorly the schedule is performing. The addition of Earned Value helps, but EVM’s traditional schedule metrics are inadequate. Earned Schedule fills the gaps. It provides a robust baseline to measure past schedule performance and to estimate future impact on delivery date and performance level.

For plan-driven projects, schedule baselines are a given. For Agile projects, baselines are suspect. The difference is evident in the charters of the two approaches.

On one hand, the Project Management Institute states that the schedule baseline is the time- phased plan against which project execution is measured and managed (“PMB”, 2013, p. 549). Without the baseline, there is no basis for measuring and managing schedule performance.

On the other hand, both the Agile Manifesto (Beck et al., 2001a) and Principles (Beck et al., 2001b) omit any reference to baselines. The Manifesto takes the omission a step further by stating that change is valued over following a plan. As the schedule baseline is, by definition, a plan, it is clearly not a high priority for Agile.

The different perspectives are rooted in disparate objectives. Canonical Agile projects seek the early and continuous delivery of the Product Vision. Orthodox plan-driven projects seek the on-time and on-budget delivery of the project objectives.

Schedule and cost constraints make a crucial difference. To deliver on-time and on-budget, you need credible targets for timeline and funding. Baselines embody the targets.

Whether or not Agile projects are ever free of such constraints is debatable. That debate is not the focus here. Instead, the focus is on those projects that use the Agile framework but are also bound by constraints, specifically the time constraint.

DEFINING THE SCHEDULE BASELINE

In Agile projects, a schedule baseline can be derived from velocity. Velocity is the number of Release Points (aka, Story Points) for each Sprint.

How is the velocity set? Again, there are challenges from the canonical Agile view. As the Agile Alliance puts it: “phrases such as ‘setting the velocity’ reveal a basic misunderstanding” (“Velocity”, n.d.). In that view, velocity is a retroactive measurement, something done after the fact. It is not a forward-looking estimate of the future completion rate.

The objection is another consequence of omitting time as a constraint. Without that constraint, you simply measure the number of Release Points completed in each Sprint. At most, that number can be kept in mind as Product Backlog Items are selected for the next Sprint.

In a time-constrained Agile project, the “next Sprint” is not enough. Velocity measures must reach beyond the next Sprint to cover the whole project timeline. Without that scope, you cannot reasonably commit to a delivery date. With that scope, you need an estimate, specifically an estimate that encompasses the whole timeline.

The Quarterly Magazine of the College of Performance Management | mycpm.org 37 SCRUM ESTIMATES There are some variants of Agile that advocate forward-looking velocity estimates. For instance, the Scrum framework uses techniques such as T-shirt sizing, Planning Poker, and Fibonacci bucketing to produce estimates (Sil, 2013).

Scrum estimates are for the relative size of Product Backlog Items. The size is determined by comparingAgile ’sItems Earned in Schedule the Backlog: Baseline Item A is larger than Item B which is larger than Item C. But, what dimensionRobert Van Deis beingVelde, PhD sized? That is not so clear. Some say it is the amount of complexity; others say it is cost; still others say it is uncertainty.

Scrum estimates are for the relative size of Product Backlog Items. The size is determined by comparing In the end,Items Scrum’s in the Backlog: relative Item Aestimates is larger than are Item viewed B which isas larger replacements than Item C. But, for what absolute dimension Work is Hour estimatesbeing sized? (“Relative That is not soEstimation”, clear. Some say n.d.). it is the So, amount complexity, of complexity; cost, others and say uncertainty it is cost; still othersare just considerationssay it is uncertainty. used to size work effort.

In the end, Scrum’s relative estimates are viewed as replacements for absolute Work Hour estimates Relative(“Relative estimates Estimation”, often n.d.)differentiate. So, complexity, sizes cost, byand assigning uncertainty arenumbers just considerations from a geometricused to size series or from thework Fibonacci effort. sequence. That gives the appearance of quantifying the differences. For instance, an Item assigned an 8 is larger than one assigned a 2. Relative estimates often differentiate sizes by assigning numbers from a geometric series or from the Fibonacci sequence. That gives the appearance of quantifying the differences. For instance, an Item Many proponentsassigned an 8 ofis larger Agile than do one not assigned stop there.a 2. They say that relative estimates also indicate how much larger one Item is than another (Singh, 2016). For instance, an Item assigned an 8 is four-timesMany proponents larger than of Agile one do assignednot stop there. a 2.They say that relative estimates also indicate how much larger one Item is than another (Singh, 2016). For instance, an Item assigned an 8 is four-times larger than one assigned a 2. We have found that Agile teams often agree on the order between Items. But, we have observedWe frequent have found disagreementthat Agile teams often over agree how on themuch order difference between Items. there But, we is havebetween observed Items. frequent Is an Item labeled disagreementwith a Fibonacci over how numbermuch difference of “21” there really is between 7 times Items. larger Is an Item than labeled one with labeled a Fibonacci as “3”? Is an Item labelednumber with of “21” a really geometric 7 times larger series than number one labeled “3” as really“3”? Is anone-third Item labeled the with effort a geometric of aseries “9”? number “3” really one-third the effort of a “9”?

LIMITSLimits OF ofRELATIVE Relative Estimates ESTIMATES The disagreements reflect divergent beliefs about size. The situation is similar to one that The disagreements reflect divergent beliefs about size. The situation is similar to one that occurs in social occurs inscience social and science marketing and research. marketing There, Lik research.ert scales measure There, psychological Likert scales states measure such as levels psychological of states suchsatisfaction as levels (e.g., withof satisfaction a product or service). (e.g., Thewith comparative a product levels or are service). often associated The comparative with numbers levels are oftenfrom associated 1 to 5, as depicted with innumbers Figure 1.1 from 1 to 5, as depicted in Figure 1.1

FigureFigure 1 1

Likert scalesLikert scales have have been been studied studied extensivelyextensively (Stevens, (Stevens, 1946; Michell, 1946; 1986;Michell, Sauro, 1986; 2011 )Sauro,. The studies 2011). The have raised questions such as: do the numbers have an objective numerical basis, and are the intervals studies betweenhave raised levels equal questions? The upshot such is as:that dothe scalethe numbers represents subjectivehave an states objective and that numerical such states basis, and are the cannotintervals be objectively between measured. levels So,equal? we cannot The be upshot sure that is the that difference the scale in assigned represents numbers issubjective the states andsame that as the such attribute states they represent.cannot be objectively measured. So, we cannot be sure that the difference in assigned numbers is the same as the attribute they represent.

Similarly,1 Technically, Agile team there members is a difference use between numbers a Likert to scale express and a Likertbeliefs item about (Vanek, the 2012 relative; Uebersax, size of work effort. But, what one person believes is twice as much effort may differ from what another 2006). The item is what a survey respondent is asked to evaluate, e.g., level of satisfaction with a recently purchased product. In well-formed surveys, there are multiple items intended to reveal the respondent’s 1) Technically, there is a difference person believes is twice the effort. So, although 4 is twice the size of 2, we cannot be sure that everyoneunderlying on psychological the team state. means The Likertthe same scale is thing the sum when of all thethey items. assign So, in “4”a survey to anwith Item. 5 items The most between a Likert scale and a Likert and responses ranging from 1=Very Dissatisfied to 5=Very Satisfied, the Likert scale is 5 to 25. Speaking item (Vanek, 2012; Uebersax, that weprecisely, can be therefore,sure of Figureis that 1 representsthe Item the is formatlarger of anda Likert more-or-less-type item. twice the size of a “2”. 2006). The item is what a survey respondent is asked to evaluate, WHY “FUZZINESS” MATTERS 2 e.g., level of satisfaction with a Does the “fuzziness” matter? Yes, it matters. The estimates are used in Sprint planning. Sprint recently purchased product. In well-formed surveys, there are planning, in turn, is important for meeting time constraints. Backlog Items are selected to fit into multiple items intended to reveal a Sprint based in part on the estimated velocity of the team. Without that guide, the Sprint goal the respondent’s underlying might be set too high or too low. Either way, project time commitments would be undermined. psychological state. The Likert scale is the sum of all the items. For selecting Items of the right size, relative estimates fall short. Using fuzzy estimates is So, in a survey with 5 items and like driving through a town that has posted its speed limit as “35-ish”. If you are not worried responses ranging from 1=Very about time—just drive at 15, and you should be OK. Or, if you do not care about safety, drive Dissatisfied to 5=Very Satisfied, at 55 but recognize that you might crash. the Likert scale is 5 to 25. Speaking precisely, therefore, Figure 1 represents the format of a Likert- If time matters, you need to know how fast you can go without breaking the limit but still type item. getting through as quickly and safely as possible. That is what a cardinal estimate tells you. It

38 The Measurable News 2017.04 | mycpm.org goes beyond the subjective state to something that can be objectively measured. It provides a clear baseline for assessing performance on past Sprints and planning future ones. That is why we cannot stop the estimating process with relative estimates. Instead, we need to develop cardinal estimates.2

LIMITS OF RELEASE POINT BASELINES Armed with cardinal velocity estimates, the Release Point baseline can be set. It is runs from the Release Point total to zero. The end of each Sprint marks completion of an increment of estimated velocity. For comparison, the number of Release Points actually completed at the end of each Sprint is used to decrement the Release Point total. A burn chart is generally used to Agileillustrate’s Earned the Schedule estimated Baseline and actual velocity. Figure 2 illustrates the chart, specifically the BurndownRobert Van Chart, De Velde, from PhD a recent project.

FigureFigure 22

Schedule performance is inferred from the relationship between the Actual Burn line and the Planned ScheduleBurn performance line. When the Actualis inferred Burn line from is above the the relationship Planned Burn betweenline, it suggests the that Actual schedule Burn line and the Plannedperformance Burn line. is When lagging . theWhen Actual the Actual Burn Burn line line is belowabove the thePlan nedPlanned Burn line Burn, it suggests line, it that suggests that scheduleschedule performance performance is islagging. better than When expected the. Finally, Actual the Burnsize of linethe gap is betweenbelow the two Planned lines suggests Burn line, the level of schedule performance efficiency. it suggests that schedule performance is better than expected. Finally, the size of the gap betweenIt isthe important two lines to note suggests that the chart the only level suggests of schedule how well the performance schedule is performing. efficiency. The data points in the chart are not units of time but are, instead, Release Point counts. There is no quantification of It is importantschedule to performance note that efficiency the chart. Finally, only there suggests is no estimate how well of the the performance schedule impact is performing. on delivery date. The The Release Point Baseline and its pictorialization, the burn chart, are good tools for a quick reading on data pointsthe situation, in the chartbut they are omit not key units pieces of of timeinformation. but are, instead, Release Point counts. There is no quantification of schedule performance efficiency. Finally, there is no estimate of the performance impact onOne delivery reason they date. fall shortThe isRelease that Release Point Points Baseline do not measureand its value.pictorialization, On time-constrained the burn projects, chart, we are good toolsneed for to knowa quick not onlyreading the number on the of Pointssituation, but also but the they value omitof those key points. pieces That of knowledge information. enables us to see beyond the completion of low value items that make burn charts look good but are not really moving the project forward. One reason they fall short is that Release Points do not measure value. On time-constrained projects,Note we that need the termto know “value” nothere onlyreflect sthe the efficiencynumber of of execution Points rather but thanalso its the effectiveness. value of Thatthose is, itpoints. That knowledgeis Earned Value enables rather usthan to Business see beyond Value (Alleman, the completion 2011; Alleman, of 2009 low). valueEarned itemsValue Management that make burn (EVM) tells us the budgeted cost of the planned velocity and the amount of that budget earned by the charts lookactual good velocity but. The are traditional not really EVM measure moving of schedulethe project performance, forward. the Schedule Performance Index (SPI), is the ratio between the cumulative earned value and total planned value. Note that the term “value” here reflects the efficiency of execution rather than its effectiveness.Unfortunately, That traditional is, it is EVMEarned measures Value of schedulerather thanperformance Business are inadequate Value (Alleman,. After a project 2011; is about Alleman, two-thirds complete, the SPI rises inexorably to a perfect 1.0. Even if the project finishes late, the SPI 2009). Earnedends at 1.0 Value—a counterintuitive Management indication (EVM) of performance tells us the. Furthermore, budgeted traditional cost ofEVM the measures planned do not velocity and the includeamount an estimate of that at budget completion earned for time (EACt)by the or actualan estimate velocity. of the schedule The traditional performance EVM level measure of schedule performance, the Schedule Performance Index (SPI), is the ratio between the 2) Discussion of techniques for cumulative earned value and total planned value. producing cardinal estimates in Agile projects is outside the scope of this paper. Glen Alleman has Unfortunately, traditional EVM measures of schedule performance are inadequate. After a 4project many instructive posts on the topic is about two-thirds complete, the SPI rises inexorably to a perfect 1.0. Even if the project finishes in his blog (see Alleman, 2015a; late, the SPI ends at 1.0—a counterintuitive indication of performance. Furthermore, traditional Alleman, 2015b; and Alleman, 2017). EVM measures do not include an estimate at completion for time (EACt) or an estimate of the 3) There are extensions to schedule performance level required to complete on time (TSPI).3 Finally, traditional EVM schedule traditional EVM that include performance measures are framed in terms of dollars, rather than units of time. formulas for EACt (e.g., Anbari, 2001). The extensions are Fortunately, Earned Schedule has closed the gaps. undermined by reliance on SPI.

The Measurable News 2017.04 | mycpm.org 39 EARNED SCHEDULE The amount of time earned on a project is defined as “the time at which the value currently earned should have been earned” (Lipke, 2009, p. 14). The definition neatly ties planned value and earned value into time, framing ES metrics in units of time rather than cost.

The calculation of Earned Schedule is simple. Count the number of Sprints in which the current total of Earned Value is greater than or equal to the cumulative Planned Value.4 Usually, some Earned Value remains after the last full Sprint is counted. The fractional time earned equals the ratio between the left-over Earned Value and the Planned Value for the next Sprint beyond the last full one.

Schedule performance efficiency for time (SPIt) is calculated as the ratio between the amount of schedule earned and the actual time. Based on this efficiency, the EACt is estimated as the ratio between the Planned Duration and SPIt. Finally, the TSPI is the ratio between the Planned Duration less the Earned Schedule and the Planned Duration less the Actual Time.

Studies have repeatedly demonstrated that Earned Schedule metrics are superior to other EVM schedule performance measures (Vanhoucke and Vandevoorde, 2007; Lipke, 2008; Crumrine and Ritschel, 2013). In our own practice, we have used the metrics successfully on many projects—some of which used the Agile framework. We have found ES metrics to be especially useful in managing hybrid project portfolios comprising both plan-driven and Agile projects.

EARNED SCHEDULE BASELINE The Earned Schedule baseline is framed by the pace of Release Point delivery. Cardinal estimates for the number of planned Release Points and the mean velocity determine the number of Sprints. Given the project start date and number of Sprints, the planned finish date is set. The Earned Schedule baseline fits into that frame as follows: for each elapsed Sprint, one Sprint should be earned.

If all Release Points carry the same value, the mean velocity ensures that the periodic Planned Value is the same for each planned Sprint. It follows that the periodic amount of planned Earned Schedule is the same for each planned Sprint. Expressed as a burn chart, the baseline runs straight from the end of the first Sprint to the end of the last planned Sprint.

It is possible for Release Points to carry different values. For instance, if Release Points express Work Hours, their value might vary depending on the resource responsible for delivering them. In such cases, the baseline is still one earned Sprint for each elapsed Sprint, but there’s a catch.

If Release Points carry different values, a mean Planned Value should be set for all Sprints and used to guide the selection of Items for the next Sprint. A constant Planned Value ensures that the periodic Earned Schedule is the same across all planned Sprints. The ES baseline, therefore, will be isomorphic with the Release Point baseline, providing common ground for comparison.

By contrast, the actual ES burn will usually be non-linear. The amount of schedule earned generally varies from Sprint to Sprint. At the end of each Sprint, the total number of Sprints is decremented by the total amount of schedule earned. So, in a burn chart, the ES burn line typically weaves around the ES baseline.

Figure 3 illustrates the ES Burndown from the same project used for Figure 2. The ES baseline runs straight from the beginning to the planned finish. The ES Burn line runs on or above the baseline. The chart is enhanced with a table displaying additional ES metrics.

4) A Sprint is defined as a time- boxed unit of delivery. All Sprints in a project have the same duration, normally 1 week to 1 month. Calculating ES in terms of Sprints is, therefore, equivalent to calculating ES in units of time.

40 The Measurable News 2017.04 | mycpm.org Agile’s Earned Schedule Baseline Robert Van De Velde, PhD

total amount of schedule earned. So, in a burn chart, the ES burn line typically weaves around the ES baseline.

FigureFigure 33

Interpreting the ES Burndown Chart INTERPRETING THE ES BURNDOWN CHART The ES BurndownThe ES Burndown Chart Chart is iseasy easy to interpret interpret.. If the If ES the Burn ES line Burn is above line the is ESabove baseline the, schedule ES baseline, performance lags the estimate. If the ES Burn line is below the baseline, performance is ahead of the scheduleestimate performance. If it is on the lags line, theschedule estimate. performance If the is exactlyES Burn on the line estimate is below. Because the thebaseline, data points performancerepresent is units ahead of time, of the chart estimate. explicitly Ifshows it is howon timethe isline, being schedule used on the performance project. is exactly on the estimate. Because the data points represent units of time, the chart explicitly shows how time is beingIn Figure used 3, the onchart the makes project. it clear that schedule performance is lagging in most Sprints. The performance deficit is clearest when a large gap opens at Sprints 2 through 5. By contrast, Figure 2 suggests that the project is generally running on or slightly behind schedule. (In Sprint 8, it even appears In Figurethat 3, the the project chart is slightly makes ahead it clear of schedule. that schedule) Given that theperformance project actually is fini laggingshed after in the most Baseline Sprints. The performanceFinish, the deficit ES Burndown is clearest is a more when accurate a large representation gap opens of schedule at Sprints performance 2 through. 5. By contrast, Figure 2 suggests that the project is generally running on or slightly behind schedule. (In The project’s backstory helps explain the chart. To build momentum, the project team decided to spend a Sprint 8,couple it even of Sprints appears working that on quickthe projectdeliverables, is slightlyeven though ahead they were of schedule.)low value and Givendid not alignthat with the project actuallythe finished mean Planned after Value the. TheBaseline quick deliverables Finish, the preserved ES Burndown Release Point is productiona more accurateat the estimated representation of schedulerate but performance. delivered less Earned Value than planned. Hence, the Release Point chart tracked close to plan, and the ES chart did not.

The project’sThe table backstory included with helps the chart explain quantif iethed how chart. time wasTo buildbeing used momentum, on the project. the When project the SPIt team was decidedbelow to spend 1.0, schedule a couple performance of Sprints was lagging working. If the on SPIt quick had been deliverables, over 1.0, schedule even performance though they were low valuewould and have did been not better align than with expected. the meanAs commonly Planned happens, Value. the SPItThe was quick never deliverables exactly equal to preserved 1.0. Release Point production at the estimated rate but delivered less Earned Value than planned. Hence, the Release Point chart tracked close to plan, and the ES chart did not. 6

The table included with the chart quantified how time was being used on the project. When the SPIt was below 1.0, schedule performance was lagging. If the SPIt had been over 1.0, schedule performance would have been better than expected. As commonly happens, the SPIt was never exactly equal to 1.0.

Early in the project, as the team built momentum, the SPIt suffered. Eventually, the team returned to the original plan, and the SPIt improved. The improvement did not, however, recover all of the lost time. The SPIt accurately reflected performance throughout the project lifecycle. When the project exceeded the Baseline Finish, the SPIt ended below 1.0.

5) The Baseline Finish includes Contingency but excludes Reserve. The EACt and TSPI threw additional light on schedule performance. The Baseline Finish had 5 The Committed Finish includes been set at Sprint 9 with a Committed Finish at Sprint 10. The EACt consistently showed both (see Van De Velde, 2017). that, given actual performance levels, the Baseline Finish would not be met.6 Often, Contingency and Reserve are limited to allocations of money. The TSPI re-enforced that view. In several Sprints, the TSPI exceeded the commonly They should also cover allocations accepted threshold for recoverability (i.e., a value of 1.1 as per Lipke, 2016; Lipke, 2009, pp. of time. 90-91). Unsurprisingly, it was beyond the threshold at Sprint 8, just before the TSPI became 6) It is also possible to subject the undefined.7 EACt to statistical analysis. The likely range of a project’s EACt values can be calculated based on The most positive signal in the metrics surfaced after Sprint 5. The EACt began to show that its historical SPIt metrics (see Van the Committed Finish would be met. De Velde, 2015a-2015c). 7) By definition, the TSPI becomes In the example, although schedule performance was not uniformly bad, it was never good undefined once the Actual enough to meet the Baseline Finish. The ES Burndown Chart signalled the potential for delay, Duration reaches the Planned and the associated metrics quantified the implications. Duration.

The Measurable News 2017.04 | mycpm.org 41 CONCLUSION On time-constrained Agile projects, the Earned Schedule Baseline provides a robust yardstick for measuring schedule performance. Using the ES Baseline entails divergence from both canonical Agile practice and plan-driven methodology. The discrepancies must be acknowledged, but in the end, the value that Earned Schedule brings to Agile projects makes the departures worthwhile.

REFERENCES Alleman, G.B. (2009, January 10). Some Serious Misunderstandings About Earned Value [Blog post]. Retrieved from http://herdingcats.typepad.com/my_weblog/2009/01/some-serious- misunderstandings-about-earned-value.html

Alleman, G.B. (2011, August 02). Measuring Business Value [Blog post]. Retrieved from http://herdingcats.typepad.com/my_weblog/2011/08/measuring-business-value.html

Alleman, G.B. (2015a, March 10). Criteria for a “Good” Estimate [Blog post]. Retrieved from http://herdingcats.typepad.com/my_weblog/2015/03/criteria-for-a-good-estimate.html

Alleman, G.B. (2015b, December 26). The Story Point Problem [Blog post]. Retrieved from http://herdingcats.typepad.com/my_weblog/2015/12/the-story-point-problem.html/

Alleman, G.B. (2017, April 6). Velocity versus Speed (Update) [Blog post]. Retrieved from http://herdingcats.typepad.com/my_weblog/2017/04/velocity-versus-speed.html

Anbari, F.T. (2001, November). Applications and Extensions of the Earned Value Analysis Method. In Proceedings of the Project Management Institute 2001 Seminars & Symposium, 1-10. Nashville, TN, USA. Newtown Square, PA: Project Management Institute. Retrieved from http://www.pmi.org/learning/library/earned-value-analysis-forecast-outcomes-7898?id=7898

Beck, K., Beedle, M., van Bennekum, A., Cockburn, A., Cunningham, W., Fowler, M., Grenning, J., Highsmith, J., Hunt, A., Jeffries, R., Kern, J., Marick, B., Martin, R.C., Mellor, S., Schwaber K., Sutherland, J., and Thomas, D.. (2001a). Agile Manifesto. Retrieved from http://agilemanifesto.org/

Beck, K., Beedle, M., van Bennekum, A., Cockburn, A., Cunningham, W., Fowler, M., Grenning, J., Highsmith, J., Hunt, A., Jeffries, R., Kern, J., Marick, B., Martin, R.C., Mellor, S., Schwaber K., Sutherland, J., and Thomas, D.. (2001b). Principles Behind the Agile Manifesto. Retrieved from http://agilemanifesto.org/principles.html

Crumrine, K. and Ritschel, J.D. (2013). A Comparison of Earned Value Management and Earned Schedule as Schedule Predictors on DoD ACAT I Programs. The Measurable News, 2013(2), 37-44. Retrieved from http://www.earnedschedule.com/Docs/Crumrine%20Article%20-%20 Measurable%20News%20(May%202013).pdf

Lipke, W. (2008). Project Duration Forecasting: Comparing Earned Value Management Methods to Earned Schedule. CrossTalk. Retrieved from http://www.earnedschedule.com/Docs/Project%20 Duration%20Forecasting%20%20-%20Walt%20Lipke.pdf

Lipke, W. (2009). Earned Schedule. Raleigh, North Carolina: Lulu Publishing.

Lipke, W. (2016). Examination of the Threshold for the To Complete Indexes. PM World Journal, V (III). Retrieved from http://www.earnedschedule.com/Docs/PMWJ-Examination%20of%20 Thresholds%20for%20To%20Complete%20Indexes.pdf

Michell, J. (1986). Measurement Scales and Statistics: A Clash of Paradigms. Psychological Bulletin, 100 (3), 398-407.

PMB (i.e., Performance Measurement Baseline). (2013). A Guide to the Project Management Body of Knowledge (PMBOK Guide). Newtown Square, Pa: Project Management Institute. Note: The phrase “time-phased plan” is from Baseline. (2004). A Guide to the Project Management Body of Knowledge (PMBOK Guide). Newtown Square, Pa: Project Management Institute, p. 352.

Relative Estimation. (n.d.) Retrieved from https://www.agilealliance.org/glossary/relative-estimation/

Sauro, J. (2011, June 01). Should You Care If Your Rating Scale Data Is Interval Or Ordinal? Retrieved from https://measuringu.com/interval-ordinal/

Sil, K. (2013, November 13). How to Estimate Quickly and Efficiently. Retrieved from https://www. scrumalliance.org/community/articles/2013/november/success-story-how-to-estimate-quickly- and-efficien

Singh, V. (2016, January). Agile Estimation Techniques. Retrieved from https://www.scrumalliance.org/ community/articles/2016/january/agile-estimation-techniques/ Stevens, S.S. (1946). On the Theory of Scales of Measurement. Science, 103 (2684), 677-680.

42 The Measurable News 2017.04 | mycpm.org Uebersax, J.S. (2006). Likert Scales: Dispelling the Confusion [Blog post]. Retrieved from http://www.john-uebersax.com/stat/likert.htm

Van De Velde, R. (2015a, October 31). ES Statistical Analysis [Blog post]. Retrieved from http://www.projectflightdeck.com/cESExchange000.php?blog_archive=2015-10

Van De Velde, R. (2015b, November 11). ES Statistical Analysis in Action [Blog posts]. Retrieved from http://www.projectflightdeck.com/cESExchange000.php?blog_archive=2015-11

Van De Velde, R. (2015c, December 8). ES Statistical Analysis Pro and Con [Blog post]. Retrieved from http://www.projectflightdeck.com/cESExchange000.php?blog_archive=2015-12

Van De Velde, R. (2017, February 27). ES for Agile Projects: Step 2 Baseline the Schedule [Blog post]. Retrieved from http://www.projectflightdeck.com/cESExchange000.php?blog_archive=2017-02

Vanhoucke, M. and Vandevoorde, S. (2007, October). A Simulation and Evaluation of Earned Value Metrics to Forecast the Project Duration. Journal of the Operational Research Society, 58, 1361-1374.

Vanek, C. (2012, April 24). What is a Likert Scale? Retrieved from https://www.surveygizmo.com/survey- blog/likert-scale-what-is-it-how-to-analyze-it-and-when-to-use-it/

Velocity. (n.d.) Retrieved from https://www.agilealliance.org/glossary/velocity/

ABOUT THE AUTHOR Robert Van De Velde owns and operates ProjectFlightDeck.com, a company focused on Earned Schedule products and services. As a project manager, Rob has a 30-year track record of delivering IT programs and projects in a variety of domains. He has successfully used ES on both plan-driven and Agile projects. Rob holds a PhD, a PMP, and a Black Belt in MS Project. In 2014, he became a Certified Scrum Master. Rob posts regularly on LinkedIn’s ES group and on his blog, EarnedScheduleExchange.com.

CONTACT INFORMATION Robert Van De Velde Owner/Operator ProjectFlightDeck.com [email protected] 3416 Sawmill Valley Drive Mississauga, Ontario Canada L5L 3A4 (905) 828-0508

The Measurable News 2017.04 | mycpm.org 43 THE MEASURABLE NEWS 2017.04

INTEGRATED PROGRAM PERFORMANCE MANAGEMENT READING LIST By Glen B. Alleman

Building Information Modeling (BIM) is a process that creates and manages information on a construction project across the project lifecycle. A key output of this process is the digital description of every aspect of the built asset. The BIM draws on information assembled collaboratively and updated at key stages of a project. Creating a digital BIM enables those who interact with the building to optimize their actions, resulting in a greater whole life value for the asset.

The question for the IPPM community is. “How can BIM and the lessons learned from its evolution be used in the ’built‘ environment of aircraft, ships, ground vehicles, and other assets in the Federal Government beyond buildings and facilities.?”

Here’s a sample of papers and books on BIM and its connection to DOD acquisition processes. Google will find all these documents.

• BIM Handbook: A Guide to Building Information Modeling for Owners, Managers, Designers, Engineers, and Contractors, Second Edition, Chuck Eastman, Paul Teicholz, Rafael Sacks, and Kathleen Liston, John Wiley & Sons, 2001 – this is a good starting point for BIM. Goggle will find you an electronic downloadable version of this 611 page book. • “BIM and Cost Estimating,” REVIT® Building Information Modeling, Autodesk – Autodesk has many white papers on the BIM topic. • “3D Engineered Models: Schedule, Cost, and Post-Construction, Program Case Study – 4D and 5D modeling: NYSDOT’s Approach to Optimizing Resources,” U. S. Department of Transportation, Federal Highway Administration, FHWA-HIF-16-024. • “An Alternative Approach to Collaboration 4D Construction Planning,” K. Ruikar and S. Emmitt, Editors, Journal of Information Technology in Construction, Volume 14, pg. 30, 2009. • “BIM-based Integrated Framework for Detailed Cost Estimation and Schedule Planning of Construction Projects," Hxu Liu, Ming Lu and Mohammed Al-Hussein, 31st International Symposium on Automation and Robotics in Construction and Mining, 2014. • “Building Information Management (BIM) Implementation in naval construction,” Raymond Rohena, LSU Master’s Theses, 2011. • “Comparing BIM in construction with 3D modeling in shipbuilding industries: Is the grass greener on the other side?” Ran Luming and Vishal Singh, IFIP International Conference on Product Lifecycle Management, 2015, pp. 193-202. • “Flexible Work Breakdown Structure for Integrated Cost and Schedule Control,” Youngsoo Jung and Sungkwon Woo, Journal of Construction Engineering and Management, September/October 2004, pp. 616-625. • “HESTIA Model Based System Engineering with SysML,” Lui Wang, Spacecraft Software Engineering Branch, JSC, May 6, 2016. • “IFC Standard – A Review of History, Development, and Standardization," www.icon. org/2012/9 • “Importance of Integrating Cost Management with Building Information Modeling (BIM),” K. Sunil, C. Pathirage, and J. Underwood, International Postgraduate Research Conference (IPGRC 2015), 10-12 June 2015, Salford Quays. http://usir.salford.ac.uk/35630/ • “Integration of Cost and Schedule Using BIM,” Su-Ling Fan, Chen-Hua Wu and Chien- Chun Hun, Journal of Applied Science and Engineering, Vol. 18, No. 3, pp. 223-232, 2015.

44 The Quarterly Magazine of the College of Performance Management | mycpm.org • “Intelligent BIM-Based Construction Scheduling Using Discrete Event Simulation,” Markus König, Ilka Habenicht, Christian Koch, and Sven Spieckermann, Proceedings of the IEEE 2012 Winter Simulation Conference. • “A dependency network description of building information models,” K. Al Sayed, M. Bew , and A. Penn, Proceedings of the International Conference on Smart Infrastructure and Construction, Cambridge Centre for Smart Infrastructure & Construction, 2016. • “XML-based Modeling of Risk Estimation Criteria to Support Safety Management in Shipbuilding,” Youhee Choi, Jeong-Ho Park, and Byungtae Jang, Proceedings of the World Congress on Engineering and Computer Science, 2016 Vol I October 19-21, 2016, San Francisco, USA. • "Traditional Design versus BIM Based Design,” Ireneusz Czmocha and Adam PĊkala, XXIII R-S-P seminar, Theoretical Foundation of Civil Engineering (23RSP) (TFoCE 2014), Procedia Engineering, 91, 2014, pp. 210 – 215 • GSA BIM Guides, https://www.gsa.gov/real-estate/design-construction/3d4d-building- information-modeling/bim-guides • “Linking Knowledge-Based Systems to CAD Design Data with an Object-Oriented Building Product Model,” Kenji Ito, Yasumasa Ueno, Raymond Levitt, and Adnan Darwiche, Center of Integrated Facility Engineering, Stanford University, technical report Number 17, August 1989.

The Measurable News 2017.04 | mycpm.org 45 THE MEASURABLE NEWS 2017.04

VENDORS & SERVICES

SNA Software P.O. Box 20996 see our ad on p. 14 Sarasota, FL 34276 www.sna-software.com Phone: (888) 644-5613 www.humphreys-assoc.com THE MEASURABLE NEWS www.CBTWorkshop.com see our ad on p. 14 see our ad on p. 04 The Quarterly Magazine of the College of Performance Management

Planning every step — Every step as planned™

JOSH GARVEY PRESIDENT * 2017.04 Got your ? Cell: 727.543.2116 | Office: 888.249.7990 Get it [email protected] now! EVPrep™ Exam Study Guide www.clearplanconsulting.com EVPrep™ Exam Prep Workshops www.mgmt-technologies.com see our ad on p. 28 *Earned Value Professional

11902-ClearPlan_BC_0115_V1-Versions.indd 25 2/11/15 10:39 AM

Get Involved In CPM: Become A Volunteer

Most people agree that the value of membership in any organization is greatly impacted by one’s involvement in that organization. CPM has many opportunities for you to become engaged in your organization.

You can help with our certificate program, become a leader in our two conferences each year or other volunteer opportunities as they appear on the website. Take the time to read each of the job descriptions and ask questions of the point of contact for each if you are not sure what is expected. PDU’s are available for volunteering in CPM. mycpm.org These opportunities are listed on our website: www.mycpm.org./ABOUT US/VOLUNTEERS THE MEASURABLE NEWS

The Quarterly Magazine of the College of Performance Management 2017.04

mycpm.org