Earned Value Management (EVM) Related and Advanced Topics

Presented at the 2018 ICEAA Professional Development & Training Workshop - www.iceaaonline.com

v1.2

Unit V - Module 15 1

Welcome to Module 15 Earned Value Management (EVM), in which we discuss the application of an integrated toolset to manage cost, schedule, and earned value in project execution.

EVM, properly implemented, should help the program manager establish and execute to a reasonable baseline, thus avoiding the “Dumb and Dumber” situation of being so far behind schedule and over budget that the only prospects for recovery are “So you’re telling me there’s a chance!”

v1.2 Acknowledgments

• ICEAA is indebted to TASC, Inc., for the development and maintenance of the Cost Estimating Body of Knowledge (CEBoK®) – ICEAA is also indebted to Technomics, Inc., for the independent review and maintenance of CEBoK® • ICEAA is also indebted to the following individuals who have made significant contributions to the development, review, and maintenance of CostPROF and CEBoK ® • Module 15 Earned Value Management (EVM) – Lead authors: Jennifer M. Rose, Maureen L. Tedford – Senior reviewers: Richard L. Coleman, Brian L. Octeau, Fred K. Blackburn, Colleen M. Craig – Managing editor: Peter J. Braxton

Unit V - Module 15 2

This slide lists the organizational and individual contributors to this module of CEBoK® and its predecessor, the erstwhile Cost Programmed Review Of Fundamentals (CostPROF), over the course of more than a decade.

v1.2 Outline

• Earned Value History • Integrated Baseline Review • Baseline Analysis • DCMA Tripwires • EV, ES, and the IMS • EVM/RM Integration • Advanced EACs • Rules of Thumb

Unit V - Module 15 3

This section contains some advanced topics including a discussion on Baseline Analysis and Rebaselining, Advanced discussion on calculating Estimates at Complete, analysis of Earned Value data in conjunction with the Integrated Master Schedule, and some well-known “Rules of Thumb.”

v1.2 Earned Value History • Cost/Schedule Control System Criteria (C/SCSC) – Established in 1967 by Government – Defined minimal standards for contractor management control systems – Based on studies of Industry and Gov’t practices – Did not impose a specific system • Industry Standard Guidelines for EVMS – Industry-defined standards and practices for managing acquisitions – Recognized by DoD in 1996 as a reliable basis for earned value management

http://www.acq.osd.mil/evm/resources/policies-standards.shtml

Unit V - Module 15 4

There is a lot of DoD guidance and effort that went into the development and employment of EVMS. Here we just present a high-level history of how EVM came about.

EVM was first known as Cost/Schedule Control System Criteria, or C/SCSC (C/SC2 for those mathematicians who like using exponents). Established in 1967, C/SCSC defined a set of criteria for contractors to follow to provide objective, credible project performance data for reporting to the government. C/SCSC was a good start, but was more cumbersome than the EVM used today. Some of the criteria were viewed as too detailed and restrictive, and it hampered the contractor’s ability to manage the program. Some programs ended up maintaining and reconciling two sets of data – one to manage the effort and one to report C/SCSC data to the government.

In 1989, DoD shifted the organization responsibility for C/SCSC from the Undersecretary of Defense (Comptroller) (USD(C)) to the Undersecretary of Defense (Acquisition) (USD(A)). Shortly thereafter, C/SCSC policy moved from DoDI 7000.2 to DoDI 5000.2. These were important shifts, because they represented a change in focus from financial management (“bean counting”) to program management.

In an effort to keep the parts of C/SCSC that were good and revamp the parts that were cumbersome, Industry took the initiative to define a new set of standards and practices for managing acquisitions. The new set was similar to the C/SCSC criteria but allowed more flexibility and freedom in the implementation of EVM. DoD recognized the industry standards in 1996 as a reliable basis for EVM.

The cited Wayne Abba article on the Web provides a good overview of the history of C/SCSC and EVM.

v1.2 Integrated Baseline Review (IBR)

• An Integrated Baseline Review (IBR) is conducted after contract award – Government PM and technical staff work jointly with contractor counterparts – Verify the technical content of the PMB – Verify sufficiency of the related budgets and schedules • IBR also supports understanding of risks inherent in the PMB • Typically, follow-on IBRs occur after a major program event (e.g. rolling wave, re-plan, rebaseline, single-point-adjustment or OTB)

Unit V - Module 15 5

An integrated baseline review, or IBR, is an opportunity for the Government Program Office to verify the technical content of the PMB and sufficiency of the related budgets and schedules. The Government program management and technical staff work jointly with contractor counterparts to review the technical scope in PMB in conjunction with time-phased budget (BAC) and schedule activities (IMS). Assessing the adequacy of related budgets, schedules, and resources and assuring the management control processes are implemented, increases the Government’s program manager’s confidence in the PMB and the Contractor’s anticipated execution of the work. An IBR also provides insight into underlying management processes that enable steady-state mutual understanding of program management and risks. It solidifies areas requiring extra oversight/attention by establishing and maintaining a mutual understanding of the technical content and risks inherent in the PMB and underlying management control systems. An IBR supports the following objectives: • Ensure the technical content of control accounts is consistent with the contract scope/SOW and WBS • Review contractor's interpretation of the SOW and breakdown of work • Ensure PMB is complete – all work scope captured • Ensure that there is a logical sequence of planned effort consistent with the contract schedule • Assess resource availability for assigned tasks • Proper mix of resources have been assigned to accomplish all requirements • Assess the validity of control account budgets – look at basis of estimate (BOE) and assess risk • Assess validity of earned value methods and understand how contractor plans to measure real accomplishments • Gain a mutual understanding of the cost/schedule management process and the management control processes that are implemented • Understand what is behind the numbers on the contractor’s performance reports An IBR should be viewed as a “continuing process,” not a program “event.” The initial IBR occurs following contract award. Follow-on IBRs occur after a major program event (e.g. rolling wave, re-plan, re-baseline, single- point-adjustment or OTB). As the project matures, on-going PMB assessments determine need for subsequent IBRs.

v1.2 What is Reviewed in an IBR?

Product Delivery Planning (Business Case, Program Execution Plan, Requirements Set)

(Design, Develop, Test, Production, Deploy) Program Phases and Control Gates SRR PDR TRR IOC FOC

Technical / Prog Mgmt Process Framework and • SOW • CWBS/OBS Artifacts enables Schedule Feeds • CWBS Dictionary Standard Work, • Requirements • Technical Framework Templates, • Planning Templates • Product to CWBS mapping • Integrated Master Schedule Resources Planning • BOM • Summary, Intermediate, Detail • Program Management Plan schedules • Control Account Plans- • Risk Management Plan • Vertical and Horizontal Integration Timephased budget and • Program Execution Plan • Schedule Margin / Reserve resources • Technical Planning Assumptions • Critical Path • Weekly name runs Demonstrate and Risks • Schedule Assumptions and Risks • Staffing plans • RAM Relationships in • Dependencies: Internal, External, • Time-phased forecast • Work Authorizations GFx, Legacy / Heritage • Reports (CPR, CFSR) Wall-walks, • Task Descriptions Show • Entry and Exit Criteria Storyboards • Budget Change Request • Resource Loaded Schedule With threaded • BOE traces Risks and Opportunities (Quantified / Qualified) Unit V - Module 15 6

An IBR is conducted by focusing on the end-state but breaking the phases into “manageable pieces.” Delivery artifacts are mapped to technical, schedule, and resource plans and the linkages between the artifacts and the technical, schedule, and cost plans should be reviewed and understood. Attention should be given to the deliverables that precede the Control Gate. Throughout the IBR process (and before and after it is conducted) the team must identify, quantify, and qualify risks and opportunities. These can be risks within the technical content, schedule plan, expected cost, ability to resource tasks when needed, and the underlying management processes that help The most successful IBRs are those in which all participants work to form a collaborative environment and understand the importance of IBR (and EVM in general), as emphasized by management on both Government and contractor sides.

v1.2 Baseline Analysis • Common Baseline Problems include: – Rubber Baseline • Excessive changes • “Kick the can down the road,” i.e., moving work toward end of contract - More painful to fix later – Front-Loaded Baselines • Too much budget and insufficient work planned in early stage off project • Represents poor initial planning – Out of Date baselines • Contract changes not incorporated into PMB in a timely manner • Baseline lacks sufficient detail to achieve adequate performance measurement

Unit V - Module 15 7

There are a few well known baseline problems that one should watch for. “Rubber Baseline” is a term to describe a baseline that changes too frequently and often changes by taking work planned for a near term month and pushing it out further in the future. As a result, schedule and cost variances are masked early on and not fully realized until well into the project. The further a project has progressed, the more expensive it is to either fix a variance problem or to cancel an effort. “Front-Loaded Baseline” describes a baseline that has too much budget and insufficient work planned in the early months. In other words, the budget allocated for the work performed in the later months is not sufficient to complete that work. Again the result is to mask cost variances until later in the program when it is more expensive to either fix or cancel. Finally, an “Out of Date Baseline” describes a situation where legitimate baseline changes have not been made in a timely fashion. As a result, the performance is being measured against a baseline that does not represent reality. The result is either overstated or understated variances, which leads to an inability to use the earned value data to project accurately the future performance and requirements.

v1.2 Baseline Analysis - Rebaselining • PMB changes frequently throughout the life of the contract – Typical to change at least quarterly due to added work or unforeseen changes requiring use of MR – Referred to as Baseline Change • PMB is frequently Rebaselined at least once during project – Rebaseline represents major revamping of PMB – Acceptable reasons to Rebaseline include: • Major new work awarded/added to contract • Customer driven changes to end product • Approval of new approach to solving customer problem – Avoid rebaselining solely to eliminate variances

Unit V - Module 15 8

Every baseline will need change at some point in time. Usually baseline changes occur at least once a quarter and are due to addition of new work to contract or use of Management Reserve. Frequently a baseline will be significantly revamped, or rebaselined, at least once during a project’s life. Usually rebaselining occurs as the result of a major ECP or project change. Valid reasons to rebaseline include addition of major new work, project scope changes directed by the customer, or a “technical breakthrough” or new approach to solving a problem. Rebaselining to eliminate large variance should be avoided. Sometimes, however, an Over the Target Baseline (OTB) is negotiated to remove variances to achieve a PMB against which relevant performance measurement can occur. More information on OTB is on the next slide.

Note that most contractors will not rebaseline based on a technical breakthrough until close to contract completion. They will continue to report favorable variances until effort nears completion just in case unanticipated cost overruns occur in the future or in case the “technical breakthrough” ends up not working. If a contractor rebaselines to a lower figure, the expected profit on the effort is reduced. If cost overruns happen at a later time and the contractor needs to request more funds to cover the overrun, the government usually does not give fee on overrun dollars. Sometimes, the government imposes a cost-share clause on all overrun dollars. Most projects will not remove fee-earning dollars due to an underrun in one WBS and add non-fee- earning dollars to another WBS that has overrun. Usually, the bottom line is the only line used when evaluating need to make contract changes, especially those that change the fee. So, it is to the contractor’s benefits to maintain the favorable variances until the company is sure future overruns are unlikely.

v1.2 Baseline Analysis – OTB • Over Target Baseline (OTB) developed when: – Work performed is no longer consistent with the PMB and CBB – Contractor needs OTB to effect proper management control • OTB typically used when remaining work is significantly greater than available budget

Unit V - Module 15 9

Sometimes a rebaseline occurs to heal cost and schedule variances. This is called an over target baseline, or OTB (which has nothing to do with off-track betting!). This can happen for more than one reason, including when the work performed is no longer consistent with the PMB and CBB. For example, at the request of the government, the contractor agrees to perform small amounts of additional work or change the work effort related to but not within the funded scope of the contract. Often this may occur several times over the course of many months. Sometimes this additional work will be covered using MR is enough MR exists. If MR is not available, the contractor may begin reporting cost and schedule variances due to the addition of this work. The variances become large so an OTB is negotiated to eliminate those variances. In this case, the variances are not really due to overrun, rather to the accomplishment of additional work that never had accompanying contract paperwork and funding. Purist would disagree with this happening, but often, if an effort is small, the preparation of ECP paperwork is inefficient and could have significant schedule impact. If a contractor and government team have a good working relationship, this slow build up of overrun due to new, related scope can occur.

An OTB can also be processed if the overrun and/or delay becomes so extreme, it clouds the predictive ability of the earned value data. If a contractor is overrunning a project seriously, the cost reports will always show unfavorable variances. Once a variance becomes too high, it is difficult to see trends regarding “new” variances that are still young enough to be controlled versus new portions of “old” variances which are only an incremental overrunning due to the same problems that created the overrun in the first place. By processing an OTB and eliminating the “old” variances and fixing the “old” problems with a new BCWS plan, “new” variances and “new” problems are more readily identified and can be addressed in sufficient time.

v1.2 Baseline Analysis – OTB • OTB, as with any major replan, should be used with caution – Masks contract performance – Should be done early and only once – Best to use OTB from current position and not erase past variances

Unit V - Module 15 10

So, while an OTB should not be readily done, on occasion, the government and contractor will settled on an agreed to amount of cost/schedule overrun, establish an OTB amount, and reset the cost and/or schedule variances. Sometimes, in lieu of an OTB, a cost overrun ECP will be processed and the contract value changed. This would occur more often if there is a cost-share clause invoked which limits the billing the company can make to the government for the overrun dollars.

Although it is best to process an OTB earlier in the program, in reality, OTBs tend to happen in the middle due to the optimistic human nature – We always think we can fix something and are reluctant to accept that a variance is there to stay. As a result, OTBs often reduce or erase past variances as well as heal future variance yet to be incurred.

v1.2 Baseline Analysis – Post Rebaseline

• If rebaseline is major, it presents a challenge for analyzing performance data • A rebaseline usually eliminates all variances – Often sets BCWS=BCWP=ACWP so CV=SV=0 – Updates BAC and LRE so VAC=0 – Cumulative Performance Indices hover near 1.0 • Cumulative data shows smaller % variances since the base is larger • Difficult to see if contractor is performing any better than before rebaseline

Unit V - Module 15 11

Analyzing data after a rebaseline can be tricky since a rebaseline usually eliminates all variances. Since EV data is cumulative, all future data reports variances with a larger base. As a result, the magnitude of % and index variances are minimized. It is difficult to assess how the contractor is performing since the baseline when cumulative data is used.

Baseline Analysis – v1.2 Post Rebaseline Tips • Better analysis conducted by considering performance since the rebaseline – Subtract cumulative BCWS, BCWP & ACWP at the rebaseline point from cumulative BCWS, BCWP & ACWP reported in current month’s reports – Compute CV, SV, CPI, etc. using the modified values – Provide measure of performance since the re-baseline

Unit V - Module 15 12

We can assess the contractor’s true performance better by looking at data only since the rebaseline. TO accomplish this analysis, we need to subtract the cumulative BCWS, BCWP, and ACWP from that reported at the time of rebaseline. Using the new set of data, variances and indices can be computed to measure performance since the rebaseline.

v1.2 Baseline Analysis - Example • Example: – January • BCWS = BCWP = ACWP = $100 –July • BCWS = $150; BCWP = $145; ACWP = $150 • Cumulative CPI, SPI: – CPI = BCWP / ACWP = $145 / 150 = 0.97 – SPI = BCWP / BCWS = $145 / 150 = 0.97 – Looks good!

Unit V - Module 15 13

For example, suppose a program rebaselined after having completed $100 of work. Six months later, the BCWS = $150, BCWP = $145, and ACWP = $150. The cumulative indices look good – only a 3% variance, which would not even trigger a yellow “caution” response. But how has the contractor’s performance been since the rebaseline?

v1.2 Baseline Analysis - Example • Performance Data since Rebaseline – Jan to Jul BCWS = $150 – $100 = $50 – Jan to Jul BCWP = $145 – $100 = $45 – Jan to Jul ACWP = $150 – $100 = $50 • CPI, SPI since Rebaseline – CPI = BCWP / ACWP = $45 / 50 = 0.9 – SPI = BCWP / BCWS = $45 / 50 = 0.9 – Not so good!

Unit V - Module 15 14

By calculating the BCWS, BCWP, ACWP since the rebaseline, we can assess the contractor’s recent performance. We see that the variances are actually about –10% which is not that good.

v1.2 Primary Trip Wires

• EVM Center Contractor EVMS confidence rating – Corrective Action Requests (CAR Levels 1-4) – EVM System maturity levels

• Integrated Baseline Review (IBR) – Initial review conducted within 180 days – Ongoing reviews triggered by Mods and/or OTBs – Outcome of the review •Cost •Schedule •Technical Baseline Risk •Resources •Processes

Earned Value and the Acquisition Program, Roberta Tomasini, DAU, 2008, http://www.dau.mil/conferences/presentations/2006_peo_syscom/tue/a6-kester.pdf.

Unit V - Module 15 15

The Defense Contract Management Agency initiated a “trip wires” initiative in 2006. This slide and the one that follows provide Primary and Secondary Trip Wire. More information on the trip wires initiative can be found in the cited online presentation.

Corrective Action Requests (CAR) are requests made for corrective action of a contractual non-compliance or deficiency by addressing the root cause of the problem. The four CAR levels identify varying levels of escalation and response for the corrective action.

v1.2 Secondary Trip Wire Metrics

• Baseline Execution Index (BEI) –IMS – Cum index less than 0.95 • Schedule Performance Index (SPI) –CPR – Cum index less than 0.95 • Critical Path Length Index (CPLI) –IMS – Cum index less than 0.95 • Cost Performance Index (CPI) –CPR – Cum index less than 0.95 • To Complete Performance Index (TCPI) –CPR – CPI to TCPI delta of 10% • Contract Modifications – Contract Mods to Original Base value of 10% • PMB Revisions – CPR, CFSR – Changes to Monthly Time-Phased PMB value of 5%

Earned Value and the Acquisition Program, Roberta Tomasini, DAU, 2008, http://www.dau.mil/conferences/presentations/2006_peo_syscom/tue/a6-kester.pdf.

Unit V - Module 15 16

The Defense Contract Management Agency initiated a “trip wires” initiative in 2006. More information on the trip wires initiative can be found in the cited online presentation.

Additional definitions for newly introduced terms include: Baseline Execution Index (BEI) shows the efficiency of actual work measured against the baseline. Critical Path Length Index (CPLI) measures the relative efficiency required to complete a milestone on- time.

Note that some of these trip wires, such as the CPI vs. TCPI Trip Wire threshold, are suggested thresholds that differ from the Rules of Thumb and other statements in the Core Knowledge section of this module. EVM analysts should set thresholds based on their comfort level considering the level of risk in the program.

v1.2 Earned Value and the IMS • EVMS should be tied to the Integrated Master

Schedule (IMS) in some fashion AKA Master – Same WBS Schedule – BCWS developed/updated using task information captured in IMS – Sometimes BCWP % complete derived from IMS • EVMS SV in terms of $ and %, not time – Treat EVMS SV as an indicator/warning – True schedule analysis on IMS is best way to assess schedule status – Earned Schedule analysis used to provide additional understanding of IMS issues

Unit V - Module 15 17

Earned Value should be closely linked to the Integrated Master Schedule, or IMS. Both should adhere to the same WBS. When developing the PMB, the time-phasing of the budget should mirror the time phasing of the related tasks in the IMS. Sometimes, BCWP is derived from the IMS percent complete data. However, while its important for the EVMS and IMS to be somewhat in sync, the best method to determine true schedule health is to conduct classical schedule analysis. EVMS SV should be treated only as a warning or indicator of potential problems.

v1.2 IMS/EVM Integration via WBS

Master Program Schedule WBS 1.0 Level 1 1996-98 1999-01 2002-04

WBS 1.1 --- WBS 1.2 --- WBS WBS 1.3 --- 1.1 1.2 1.8 Level 2 WBS 1.4 ---

WBS 1.8 ---

Intermediate Schedule WBS 2002 2003 2004 1.8.1 1.8.2 1.8.3 1.8.4 Level 3 WBS 1.8.3

WBS 1.8.3.1 WBS 1.8.3.2 WBS 1.8.3.3 WBS WBS 1.8.3.4 1.8.3.1 1.8.3.2 1.8.3.3 Level 4 ------

Control Account #2 Schedule 1.8.3.1.2 JMAMJF J AS OND Control Work Packages CA #1 Accounts x.2 a x.2 b x.2 c CA #2 x.2 d Planning Pkgs x.2 pp x. 2pp CA #3 Schedules and work plans are developed based on summary level constraints and contractual milestones Unit V - Module 15 NEW! 18 © 2002-2013 ICEAA. All rights reserved.

This graphic depicts the breakdown of the schedule from high level tasks, associated with high level WBS elements, to lower level tasks, associated with the lower level WBS items. The lowest reporting level task is at the Control Account level. This is then broken into detailed work packages and planning packages. The schedule view on the left equates to the WBS view on the right.

v1.2 Earned Value and the IMS • Compare SV to IMS task Schedule Delay – If task on critical path, SV corresponds to time delay – If task not on critical path, SV may not correspond to time delay, unless SV causes task to become critical – Calculate Slip using EV data: Slip(mths) = Cum SV / Avg 3-mth Cur BCWS • Translate into weeks (*4.3), workdays (*22), or calendar days (*30) • Variations on BCWS (ITD Avg, 6-mth Avg, Current) • As average BCWS approaches 0 (near end of project), Slip rises exponentially – does not reflect reality

Unit V - Module 15 19

If an SV reported in EVMS relates to a critical path task, then the SV corresponds to a time delay, unless SV is due to a fixable error or situation. Even is WBS task is not on the critical path, a serious SV could indicate the task has now become critical. One quick and dirty way to relate the EVMS SV $ to time is to calculate the Slip. Slip is the ratio of cum SV over the average current period scheduled work for the past three months (variations include using a six-month average, average from project inception, current BCWS only, etc.). The idea is if we can determine, In other words, by taking our schedule variance to date and dividing it by approximately how much scheduled $ we expect to accomplish in a month, we can translate approximately how much time would be required to make up the schedule variance. Using a variation on BCWS may be appropriate depending on how variable BCWS is throughout the life of the project. Using a three-month Average is good because it provides answer assuming expected resources that are on board at the current time.

This calculation is somewhat flawed in that it assumes similar performance, makeup-time equal to original plan time, and that SV is due to time-related delay such as labor (sometimes SV is due to late arriving material which will automatically fix itself more quickly than a labor-based SV). Plus, sometimes the Slip calculation means nothing in terms of project completion because the task is not on the critical path and can be accomplished concurrently with critical path tasks. And finally, as a word of caution, the Slip calculation increases exponentially as the BCWS draws closer to 0, which occurs more commonly towards the end of a program. As project completion draws near, determining real Slip using only a current or three-month average BCWS may result in over-inflated Slip estimates. But Slip does at least provide an idea of what the time delay impact COULD be for the project. In other words, its better than nothing.

v1.2 Earned Value and the IMS • EV and IMS are never completely in sync – Differences could be due to different analytics – Don’t get hung up on numbers, but focus on qualitative inconsistencies – Sort out & explain inconsistencies – Recommend policy/procedure changes if needed

Unit V - Module 15 20

When comparing the IMS and EVMS, it is important to realize that the two will never be completely in sync. Some of the differences are due to different analytics and tracking tools employed. The key is not to focus on the exact numbers and how they are different, but rather to focus on qualitative inconsistencies. For example, is EVMS reporting favorable SV while IMS is reporting project delay? While explaining the inconsistencies, sometimes errors in the procedures are found which are causing the discrepancies. If needed, recommend changes to the procedures to keep the EVMS and IMS more in sync.

v1.2 Critical Path (CP) vs. Non-CP SVs

Time now March April May June July

4/1 4/14 Schedule delay 9 4/15 6/2 Critical Path Pushes out this task 6/3 6/23 SV causes task to Pushes out this task extend beyond projected end date 6/24 7/14

SV on CP =Delay on CP SV Schedule delay at complete

Time now March April May June July

4/1 4/14 Schedule delay

Non Critical Path 4/15 6/2 Pushes out this task Tasks 1, 2, 3, 4 6/3 6/23 SV causes task to Pushes out this task extend beyond projected end date 6/24 7/14

Schedule delay at complete for this task, Critical Path but still before CP end date, so no schedule SV not on CP <> Delay SVnot on CP <> Tasks 5, 6, 7 delay for program Unit V - Module 15 21

It was stated earlier that a SV equates to a schedule delay if the task is on the critical path (or become s part of the critical path due to the delay). Drawing from the schedule graphic used to defined BCW S and BCWP in the main module, this simple example illustrated in visual form what his statement means. In the top example, the tasks are on the CP. The SV seen in task 2 causes that task to extend its completion date. This in turn pushes out tasks 3 an 4, therefore causing a delay in the schedule at complete. In the bottom example, the tasks 1, 2, 3, and 4 are no on the critical path. The same SV o ccurs, still pushing out the completion of tasks 2, 3, and 4. However, the amount those tasks were pu shed out do not exceed the tasks on the critical path. Tasks 5, 6, and 7 are still completed later and ar e unaffected by the activities in Tasks 1, 2, 3, and 4. Therefore this SV does not equate to a schedule delay.

Analysis of Past Performance - v1.2 Earned Schedule (ES) • Analysis of schedule performance using EVM is limited – Measures schedule performance in cost, not in units of time – SV always returns to zero, even if the project is behind schedule in terms of time • Earned Schedule (ES) - new analytical technique – Stems from EVM and uses same data – Bridge between EVM and integrated schedule analysis – Provides a schedule metric in terms of time, not cost • The PMI EVM Practice Standard includes information on ES, but notes it is an emerging practice – Use with caution • www.earnedschedule.com provides a consolidated forum for ES information and resources

Unit V - Module 15 22

EVM’s focus is around cost, not schedule. EVM measures schedule performance in terms of cost. “What is the cost value of the work I had planned to complete (BCWS), but have not yet completed (BCWP)?” SV is not measure in terms of time. If the task(s) with negative SV are on the critical path, the negative SV will result in a schedule delay. However if the task(s) with negative SV are not on the critical path, a schedule delay may not occur. An example illustrating this is included in upcoming slides. But, regardless on if the task is on the critical path, EVM does not easily provide insight into the time a schedule will be delayed. Also, because the SV metric always returns to zero upon completion of the project, there is no historical information about how the project performed in terms of schedule time. A project could end with significant delay but the SV metric implies perfect performance.

Earned Schedule (ES) is a relatively new analytical technique that addresses the issues of the EVM SV. It is an extension to EVM and uses the same data provided for EVM. The ES schedule performance indicators are time-based versus cost and the ES indicators provide a status and predictive ability for schedule. Because the Earned Schedule metrics use time based measures, they augment traditional EVM and integrated schedule analysis. Earned Schedule is the bridge between EVM and schedule analysis. It has been shown that ES can be used for detailed schedule analysis and that it has the potential to improve both cost and schedule prediction.

The ES website, at the given URL, aggregates the available information concerning Earned Schedule (published papers, conference presentations and terminology) and provides the availability to download the ES calculator spreadsheet. News concerning ES is posted for keeping everyone as informed as possible. Also, for convenience, we have listed in Contacts those available to provide assistance with your application. Sites of interest are identified which may help in your search for an answer to your question or problem.

Analysis of Past Performance – v1.2 SV(t) and SPI(t) • Note: ES metrics developed under PMI and therefore use the PMI earned value terms of PV (aka BCWS) and EV (aka BCWP). • ES calculated by comparing the PV to the EV for completed work and adding the fraction of the incomplete PV work

–EScum = C + I where: C = number of whole time increments of the PMB for which EV ≥ PV I = (EV – PVC) / (PVC+1 – PVC) • As work progresses, ES provides a SV(t) and SPI (t) metric to measure schedule performance in terms of time – SV(t) = ES – Actual Time (AT) – SPI(t) = ES / AT

Unit V - Module 15 23

Earned Schedule (ES) is calculated as the Number of completed PV (BCWS) time increments EV (BCWP) exceeds + the fraction of the incomplete PV increment.

The ES calculation will indicate when the project is late performing. (Recall, the EVM schedule indicators erroneously show perfect performance at project completion.)

SV in terms of time (SV(t)) is calculated by comparing the ES to the actual time required to complete the work. SPI(t) provides the same information in an index form.

ES could be determined graphically, but the calculation using the formula allows for more robust analysis.

Analysis of Past Performance – v1.2 SV($) vs. SV(t) • Graphical Comparison of EVM’s SV and ES’s SV:

SV(t) vs SV($) 250

Time Now 200

150 SV($) PVcum Behind schedule

$ in monetary terms EVcum 100

0 ES AT Time

PV EV SV(t) Behind schedule in time terms

Unit V - Module 15 24

This graph depicts the visual difference between the SV calculated within EVM and the SV calculated within ES.

v1.2 EVM/RM Integration • EVM is focused on integration of technical, schedule, and cost • Risk management is focused on planning, tracking, handling, and reporting risks

“Integrating EVM with RM A Statistical Analysis of Survey Results,” Alissa Kumley, Peter Braxton, Richard Coleman, Wayne Abba, Gay Infanti, John Driessnack, SCEA/ISPA, 2005.

Unit V - Module 15 25

The cited paper presents survey results summarizing emerging efforts to better integration EVM and Risk Management.

v1.2 EVM/RM Integration Key Findings

•Training – Training will help to overcome • Barriers associated with process immaturity and lack of knowledge/skills needed to integrate • Improper EVM usage (perception that it is a financial tool) – Training is needed to improve understanding of benefits of integrating EVM and RM – Should reside in PM curriculum – Should include techniques and tools for achieving process integration • Policy – Policy changes & implementation guidance will help remove integration barriers • Lack of management commitment • Internal/external culture • Emotional • Process – Major barrier to process integration is organizational – Overcoming these boundaries is necessary for integrations success – NDIA PMSC should partner with other communities to create demand for process integration through cross-functional cooperation

“Integrating EVM with RM A Statistical Analysis of Survey Results,” Alissa Kumley, Peter Braxton, Richard Coleman, Wayne Abba, Gay Infanti, John Driessnack, SCEA/ISPA, 2005

Unit V - Module 15 26

This is a brief summary of the findings. See the paper and the survey itself, sponsored by the National Defense Industrial Association (NDIA) Program Management Systems Committee (PMSC), for more details.

v1.2 Advanced EACs

• Other EAC calculations – ACWP + BCWR • May be acceptable for LOE only – ACWP + BCWR/(0.8CPI + 0.2SPI) • Use towards end of contract • Variation – use X% CPI and Y% SPI provided X+Y=100% and can justify reason behind X, Y selection – Variations on CPI Forecast (BAC/CPI): • 3 month CPI • 6 month CPI • Current month CPI (very volatile) 8 – Perform linear regression using ACWP vs. BCWP

Unit V - Module 15 27

The most commonly-used EAC calculations, CPI and CPI * SPI, were discussed in the main training module. However there are many other EAC calculations used. Other possibilities include:

ACWP + BCWR – only as good as the integrity of the BCWR

ACWP + BCWR/(0.8CPI + 0.2SPI) – this is a weighted form of the CPI*SPI forecast that adjusts the emphasis placed on cost versus schedule depending on the individual circumstances of the program. It is often used towards end of contract

There are also variations on CPI Forecast (BAC/CPI), such as using any of the following in the denominator of the calculation: • 3 month CPI • 6 month CPI • Current month CPI (very volatile)

And of course, one could always plot the ACWP and BCWP data and perform a linear regression analysis. However, a regression line does not assume a change in rates for performance indices. (Refer to Module 8 Regression Analysis for technical details on regression.)

v1.2 Progress-Based EACs • EACs are notoriously difficult to predict • Significant improvement in EAC prediction can be achieved if there is a reliable unbiased indicator of progress – If the progress indicator is sufficiently granular, unbiased EACs with standard errors of as little as 10% can be achieved by the 20% progress point and better SEEs are achievable at greater progress

“Progress-based EAC calculations: Ending the EAC Tail Chase: An Unbiased EAC Predictor using Progress Metrics,” Eric R. Druker, Richard L. Coleman, Elizabeth L. Boyadjis, Jeffrey R. Jaekle, SCEA, 2007

Unit V - Module 15 28

A promising approach is to predict EAC as a function of ACWP using a family of regressions at various percent complete interval. Percent complete can be determined using the traditional BCWP/BAC, but there are known issues with this, not the least of which is the propensity of BAC to change over time, via contract mods or the like. The cited paper has greater success using a consistent measure of physical progress on a shipbuilding program. Even though the progress measure itself is biased, an unbiased EAC can be developed based on it.

v1.2 Rules of Thumb – Cost

• Cum CPI usually stable once project is 15- 20% complete • Cum CPI will not change more than 10% from the value at the 20% complete point in time • Percent overrun at completion will not be less than percent overrun at 15% complete • Statistically, subcontracts will overrun more, in percent, than the prime contract

These are generally believed to be true but may conflict with real- world examples Unit V - Module 15 29

Here are some commonly used EVM rules of thumb. However, as with any rules of thumb, use them appropriately and with caution. There are always exceptions to the rule!

First, some Cost rules of thumb….

v1.2 Rules of Thumb – Schedule

• Cum SPI usually stable once project is 15-20% complete • Early SVs typically lead to CVs later – Managers tend to respond to schedule delays by applying more resources and overtime • SV is self-healing: – As % complete approaches 100%, SV approaches 0 – Once % complete > 80%, place less emphasis on SV • Programs relying heavily on software are typically further behind than non-software-intensive programs These are generally believed to be true but may conflict with real- world examples

Unit V - Module 15 30

Finally, some Schedule rules of thumb.

v1.2

Earned Value Tools • Tools Overview • EVMS Tools • EV Analysis Tools

Unit V - Module 15 31

This section provides a brief discussion on some of the more commonly used tools both for Earned Value Management Systems and for Earned Value Analysis. Company information and a webpage are provided for each tool discussed so the student may conduct further research if desired.

v1.2 Tools Overview • Two types of tools: – Tools for implementing EVMS and tracking/reporting earned value – Tools for analyzing earned value data • For more information on tools not discussed in this presentation, see https://acc.dau.mil/CommunityBrowser.aspx?id=17609

Unit V - Module 15 32

There are two types of earned value tools:

EVMS tools are designed for contractors and/or government program offices to use to implement EVM to manage a project. These tools facilitate the definition of WBS, OBS, control accounts, time-phased budgets, etc., The tools also facilitate tracking of performance taken and actual costs. These tools do have some EV analysis ability, however usually the license cost for these tools prohibits their use solely as analysis tools.

EV analysis tools are designed to take in a contractor’s earned value report and generate the current and predictive statistics and graphs for an analyst to review. These tools are often used by those who use EVMS tools because it helps the earned value managers understand what their earned value will look like to the government. It also helps to highlight problem areas that are not always obvious when looking at the greater details.

More information on EVMS and EV analysis tools can be found via the OSD performance measurement website at the address noted here.

v1.2 EVMS Tools • Tools for implementing EVMS – Artemis Views Artemis Management Systems http://www.aisc.com – Dekker TRAKKER Dekker Ltd. http://www.dekkerltd.com/ –Deltek • Deltek Cobra – http://www.deltek.com/products/ipm/cobra.aspx • Deltek MicroFrame Project Manager (MPM) http://www.deltek.com/products/ipm/mpm.aspx

Unit V - Module 15 33

We will mention four of the most commonly-used EVMS tools: Artemis Views, Dekker TRAKKER, Cobra, and MicroFrame Project Manager (MPM). MPM is probably the most well known, although probably because it has been around longest.

v1.2 EVMS Tools – Artemis Views

• “Artemis Views the industry leading enterprise project and resource management solution delivers enhanced project management capability through a single, synchronized product suite. With Earned Value Management, organizations can manage and control all cost related project and program information on an enterprise level by using familiar cost planning tools from a single, graphical Windows-based application.” – More information available on the Web

Unit V - Module 15 34

The Artemis description is taken directly from the company’s website.

v1.2 EVMS Tools – Dekker TRAKKER® • “Dekker TRAKKER® is the enterprise project management (EPM) engine that drives the Dekker PMIS™. TRAKKER comes equipped to manage cost, schedule, resource, performance and financial information, and can integrate with any existing software and servers you might already use, such as Microsoft® Project, Office or SQL Server. An earned value component is available for users who control their efforts via earned value management (EVM).” – More information available on the Web

Unit V - Module 15 35

The Dekker TRAKKER description is taken directly from the company’s website.

v1.2 EVMS Tools – Deltek CobraTM

• Deltek Cobra is an easy-to-use, powerful system for managing project costs, measuring earned value and analyzing budgets, actuals and forecasts. Complete with an intuitive interface, helpful wizards and government compliance reporting, Deltek Cobra helps businesses comply with rigorous government standards such as: – American National Standards Institute / Electronics Industry Alliance 748 for Earned Value Management Systems (ANSI 748) – Office of Management and Budget (OMB) Circular No. A-11 Exhibit 300 and 53 – Cost/Scheduling Status Report/Defense Federal Acquisition Regulation Supplement (C/SSR DFARS) clauses – Earned Value Management Implementation Guide (EVMIG) of the Defense Contract Management Agency (DCMA) – NASA Interim Directive to NPR 7120.5C, NASA Program and Project Management Processes and Requirements – Australian Department of Defense 5655 • More information available on the Web

Unit V - Module 15 36

The Cobra description is taken directly from the company’s website.

v1.2 EVMS Tools – Deltek MPMTM

• Deltek MPM is a program-based earned value measurement solution that is both easy to install and use. Used by hundreds of organizations—including 8 of the top 10 aerospace and defense contractors—Deltek's EVM software is compliant with ANSI/EIA-748 and supports critical programs for Department of Defense, Department of Energy, NASA and Department of Transportation. With MPM, you can: – Identify trends early and adjust tactics to meet diverse program challenges – Plan, track and analyze projects effectively throughout the entire program lifecycle – Enable cross-project analysis with MPM's comprehensive data warehouse – Enhance pricing, planning and "what if" analysis with powerful utilities – Improve managers' visibility with real-time project analysis – Take advantage of more than 80 standard, government and compliance reporting templates – Support better decision making and increase customer satisfaction • More information available on the Web Unit V - Module 15 37

The MPM description is taken directly from the company’s website.

v1.2 EV Analysis Tools • Tools for analyzing Earned Value – wInsight Deltek, Inc. http://www.deltek.com/products/ipm/winsi ght.aspx – Dekker iPursuit Dekker Ltd. http://www.dekkerltd.com/ipursuit.aspx

Unit V - Module 15 38

For EV analysis tools, we mention both the “industry standard” wInsight and an alternate tool from Dekker.

v1.2 EV Analysis Tools – wInsight • “Deltek wInsight™ is the de facto standard for analyzing, sharing, and reporting earned value management data. wInsight's standard set of reports provide early warning indicators to quickly identify variances in project performance, enabling organizations to proactively identify potential problems before they occur. wInsight is designed to give an easy to understand, program-level view to technical, schedule and financial professionals.” – More information available on the Web

Unit V - Module 15 39

The wInsight description is taken directly from the company’s website.

v1.2 wInsight: Stoplight Table

WBS Description % Cmp BCWS BCWP ACWP SV SV CV CV BAC LRE VAC VAC CPI SPI TCPI-LRE CPI Fcst CPI*SPI Fcst 1 0 Project Level 76.8 32,548 31,969 34,381 -579  -2,411  41,639 41,639 0  0.930 0.982 1.332 44780 44968 2 75.3 28,362 28,014 30,429 -348  -2,415  37,215 37,489 -274  0.921 0.988 1.303 40422 40546 3 A PROGRAM MANAGEM 91.9 1,760 1,760 1,742 0  19  1,915 1,748 168  1.011 1.000 24.763 1895 1895 4 ACSUBCONTRACTORS 57.5 10,986 10,986 10,984 0  2  19,111 16,073 3,038  1.000 1.000 1.596 19107 19107 5 B System Engineer 91.5 1,507 1,493 1,196 -14  297  1,631 1,226 405  1.248 0.991 4.677 1307 1308 6 C Software Engine 98.9 7,959 7,872 8,677 -87  -805  7,959 9,895 -1,936  0.907 0.989 0.072 8773 8774 7 D HARDWARE ENGINE 100.0 367 393 313 26  80  393 313 80  1.255 1.072 0.000 313 313 8 E Integration & T 88.1 3,232 3,028 4,943 -204  -1,916  3,438 5,638 -2,200  0.612 0.937 0.591 5613 5658 9 F ILS IPT 84.4 443 374 493 -69  -120  443 507 -65  0.757 0.844 4.921 584 601 10 G Hardware Fabric 90.7 2,108 2,108 2,081 0  28  2,325 2,089 236  1.013 1.000 26.530 2294 2294 11 [PMB] PERF MEASURE BL 76.8 32,548 31,969 34,381 -579  -2,411  41,639 41,639 0  0.930 0.982 1.332 44780 44968

Unit V - Module 15 40

The Stoplight Table above is a popular extract from the wInsight software. The user can rearrange columns and choose different data fields as needed. The table provides a powerful synopsis of the underlying data that really enables the analysis process.

The color codes are mostly intuitive (green = good, yellow = caution, red = bad), the blue color reserved for very good often connotes “too good to be true” and merits investigation. The arrows indicate trends (two- headed horizontal = holding steady, up = improving, down = worsening).

v1.2 wInsight: Future Baseline Tracking

BCWS Comparison

4,000.0

3,500.0

3,000.0

2,500.0

2,000.0

1,500.0 Future Plan (BCWS) Plan Future 1,000.0

500.0

0.0 Dec-07 Jan-08 Feb-08 Mar-08 Apr-08 May-08 Jun-08 Jul-08 Aug-08 Sep-08 Oct-08 Nov-08 Dec-08 Jan-09

Nov -07 Dec-07 Jan-08 Feb-08 Mar- 08

Unit V - Module 15 41

wInsight has two modules – User and Administrator. In the administrator module, the data for future months is captured. That data can be exported to Excel and used to graph the baseline trends from month to month. From graphs similar to the one above, its easy to see when future baseline changes are made.

v1.2 wInsight: ACWP Chart

WBS Description % Cmp ACWP BAC LRE ACWP-BAC ACWP-LRE BCWP CV VAC CPI Fcst CPI*SPI Fcst 1 CLIN002ND 96.24 4,511.1 3,447.4 4,066.0 1063.7 445.1 3,317.8  4,687.3 4,687.3 2 K 96.84 7,959.9 7,775.5 7,893.7 184.4 66.2 7,529.9   8,219.5 8,219.1 3 CLIN001AH 98.23 13,952.0 13,909.4 13,909.9 42.6 42.1 13,663.8  14,202.8 14,202.6 4 UM 99.27 1,953.8 1,916.0 1,916.0 37.8 37.8 1,902.0  1,968.2 1,968.2 5 F 99.80 1,929.6 1,927.0 1,927.0 2.6 2.6 1,923.2  1,933.4 1,933.4 6 C 99.88 24,577.9 24,181.9 24,181.9 396.0 396.0 24,153.8  24,606.5 24,606.5

Unit V - Module 15 42

Here is the standard wInsight ACWP Chart, highlighting the status (color fill) and trend (arrows) for CV and VAC.

v1.2 wInsight: TCPI Chart

WBS DESCRIPTION %Cmp CPI TCPI-LRE CPI - TCPI CV CV BAC LRE VAC VAC SV SV CPI Fcst CPI*SPI Fcst 1 AC SUBCONTRACTORS 57.49 1.000 1.596 -0.596 2.3  19,111.1 16,073.2 3,037.9  0.0  19,107.1 19,107.1 2 PMB PERF MEASURE BL 76.78 0.930 1.332 -0.402 -2411.4  41,638.8 41,638.8 0.0  -578.5  44,779.6 44,967.7 3 F ILS IPT 84.43 0.757 4.921 -4.164 -119.7  442.6 507.4 -64.8  -68.9  584.4 601.1 4 COM COST OF MONEY 89.38 1.231 3.684 -2.453 42.1  250.9 189.4 61.5  -18.3  203.8 205.6 5 G&A GEN & ADMIN 89.40 0.990 2.319 -1.329 -39.0  4,173.3 3,960.6 212.7  -212.1  4,217.0 4,242.4 6 G Hardware Fabric 90.70 1.013 26.530 -25.517 27.8  2,324.5 2,088.7 235.8  0.0  2,293.9 2,293.9 7 B System Engineer 91.54 1.248 4.677 -3.429 296.8  1,631.2 1,225.9 405.3  -13.8  1,307.0 1,308.0 8 A PROGRAM MANAGEM 91.91 1.011 24.763 -23.752 18.8  1,915.4 1,747.8 167.6  0.0  1,894.9 1,894.9

Unit V - Module 15 43

Here is the standard wInsight TCPI Chart, highlighting the status (color fill) and trend (arrows) for CV, VAC, and SV.

v1.2 EV Analysis Tools – iPursuit™ • “Dekker iPursuit® is a robust portfolio project management (PPM) analysis tool that can be used on a single project to monitor performance and ensure success, or that can be used to analyze an entire enterprise by linking to different, operational databases to establish the guidelines used for effective PPM. An optional earned value management (EVM) component is also available and can be used to easily extrapolate earned value information directly from Microsoft® Project.” – More information available on the Web

Unit V - Module 15 44

The iPursuit™ description is taken directly from the company’s website.