Year Nine Report
Total Page:16
File Type:pdf, Size:1020Kb
DEPARTMENT OF COMMERCE PERSONNEL MANAGEMENT DEMONSTRATION PROJECT EVALUATION YEAR NINE REPORT Washington, DC April 15, 2008 FINAL REPORT DoC Personnel Management Demonstration Project FINAL REPORT TABLE OF CONTENTS EXECUTIVE SUMMARY .............................................................................................. ES-1 ES.1. The Department of Commerce has completed nine years of the Personnel Management Demonstration Project, designed to test and evaluate a series of alternative personnel practices and to determine the generalizability of these interventions elsewhere .............................................................................. ES-1 ES.2. At the conclusion of nine years, evidence exists that a number of the interventions are having the desired effects......................................................... ES-4 ES.3. Recommendations are offered to help focus the Demonstration Project as it moves forward ................................................................................................... ES-10 1. INTRODUCTION..........................................................................................................1-1 1.1. The Department of Commerce has completed nine years of the Personnel Management Demonstration Project, designed to test and evaluate a series of alternative personnel practices and to determine the generalizability of these interventions elsewhere .................................................................................1-1 1.2. This report provides an assessment of Year Nine of the DoC Personnel Management Demonstration Project ......................................................................1-2 1.3. The structure of this report parallels the previous reports; it evaluates each personnel intervention and recommends actions for continued operation .............1-2 2. DoC DEMONSTRATION PROJECT AND ITS EVALUATION............................2-1 2.1. The Demonstration Project is being conducted to test the effects of innovative human resources practices in different organizations with a variety of occupational groups................................................................................2-1 2.2. The general objectives of this Demonstration Project emphasize the development of a higher performing workforce, as well as greater efficiency and flexibility of personnel processes.....................................................................2-2 2.3. The Demonstration Project includes DoC organizations with a wide range of missions and occupations........................................................................................2-3 2.4. The Demonstration Project encompasses 12,929 employees in both the Demonstration and Comparison Groups ..............................................................2-10 2.5. A broad range of interventions has been implemented under the Demonstration Project ..........................................................................................2-16 2.6. A valid evaluation of the Demonstration Project is critical in determining whether to continue the tested interventions and whether to make them a part of other government organizations................................................................2-25 Year Nine Final Report i FINAL REPORT DoC Personnel Management Demonstration Project 3. DATA COLLECTION AND ANALYSES ..................................................................3-1 3.1. Booz Allen conducted 28 interviews with staff in the DoC organizations participating in the Demonstration Group to determine their perceptions of the project ...............................................................................................................3-1 3.2. A total of 20 focus groups were conducted with employees from the Demonstration and Comparison Groups to help assess the Demonstration Project’s impact ......................................................................................................3-2 3.3. A survey of Demonstration and Comparison Group participants provided a key data source for our assessment.........................................................................3-3 3.4. Booz Allen used objective personnel data to measure the impact of the Demonstration Project's interventions ....................................................................3-7 3.5. Booz Allen collected HR summary data from the participating organizations as an additional means of tracking and analyzing data on the use of the Demonstration Project interventions ....................................................................3-10 3.6. Booz Allen collected site historian logs, which provide context for the experiences and perceptions of Demonstration Project participants ....................3-10 3.7. Booz Allen collected cost data to determine the extent of salary costs in the Demonstration Project ..........................................................................................3-11 4. FINDINGS AND CONCLUSIONS..............................................................................4-1 4.1. Employee satisfaction with the work environment and jobs remained strong and favorability toward the Demonstration Project continued to increase.............4-1 4.2. Demonstration Group participants continued to view greater potential for career progression than do the Comparison Group participants.............................4-8 4.3. Some Demonstration Group supervisors are experiencing the intended benefits of delegated classification authority and the automated classification system, such as time savings ..........................................................4-11 4.4. Understanding of the Demonstration Project’s performance appraisal system continues to improve.............................................................................................4-13 4.5. As occurred in all previous years, the pay-for-performance system continues to exhibit a positive link between pay and performance ......................................4-22 4.6. Many of the Demonstration Group scientists and engineers who had time left in their three-year probationary periods were kept on probation, which gave managers a longer timeframe to evaluate performance ...............................4-59 4.7. The Demonstration Project recruitment and staffing interventions are working well, although many of the interventions are no longer unique to the Demonstration Project ....................................................................................4-60 4.8. Many of the retention interventions are having the desired effect as employee motivators.............................................................................................4-71 ii Year Nine Final Report DoC Personnel Management Demonstration Project FINAL REPORT 4.9. Organizational performance has improved in some ways, but not others; while individual performance has not improved substantially, certain aspects of workforce quality have improved.....................................................................4-81 4.10. The Demonstration Project’s interventions have not impacted DoC’s adherence to the Merit System Principles or avoidance of the Prohibited Personnel Practices ...............................................................................................4-84 4.11. The Demonstration Project interventions continue to reflect a system in which there is no evidence of unfair treatment based on race, gender, or veteran status.........................................................................................................4-87 5. COST ANALYSIS .........................................................................................................5-1 5.1. Pay Pool Managers and Rating Officials use a specified process for determining their employees’ performance-based pay increases ...........................5-1 5.2. The salary growth rate was slower for the Demonstration Group than the Comparison Group, providing evidence that the Demonstration Project has met its goal to be budget neutral.............................................................................5-2 5.3. The size of the gap between the Demonstration Group and the Comparison Group in average per person salary costs was nearly constant across the years, suggesting that the Demonstration Group’s higher salary costs are due to compositional differences and not to increased payouts ....................................5-3 6. ANSWERS TO RESEARCH QUESTIONS ...............................................................6-1 6.1. Responses to OPM’s research questions show that the Demonstration Project has operated effectively and has demonstrable evidence of the success of key objectives ........................................................................................6-1 6.2. The Year Nine evaluation provides evidence that DoC has met many of its objectives for the Demonstration Project................................................................6-3 7. RECOMMENDATIONS...............................................................................................7-1 7.1. Strengthen the Demonstration Project’s branding and early education efforts ......7-1 7.2. Evaluate and revise, as appropriate, the Demonstration Project’s communication strategy..........................................................................................7-1 7.3. Create a dashboard to improve transparency..........................................................7-2 7.4. Develop strategies