Texas Tech University, Jon Ilseng, December 2015 Measuring Productivity And Performance In A System Of Systems Of Government Organizations, Educational Institutions And Industry To Meet The Needs Of Society by Jon Ilseng, B.S., Basic Sciences, M.S Engineering Management, M.E. Transdisciplinary Studies. A Dissertation In MECHANICAL ENGINEERING Submitted to the Graduate Faculty of Texas Tech University in Partial Fulfillment of the Requirements for the Degree of DOCTOR OF PHILOSOPHY Approved Dr. Atila Ertas Dr. Susan Mengel Dr. Burak Aksak Dr. David Wyrick Mark A. Sheridan, PhD Dean of Graduate School December, 2015 Copyright 2015, Jon Ilseng Texas Tech University, Jon Ilseng, December 2015 ACKNOWLEDGMENTS There are numerous persons I wish to acknowledge as I prepare this dissertation. First, I would like to thank my beautiful wife Kim for the support, encouragement and love during this difficult process. She was always willing to listen to the gripes, complaints I had and was a tremendous encourager. I would like to acknowledge my son J.J. Ilseng, daughter Brooke Thompson and her husband Ryan Thompson, and daughter Kaitlin Olson and her husband Jeff Olson. They were always there to encourage me from the very start of the PhD Program. I would like to thank Committee Advisor Dr. David Wyrick for the tremendous advice on research material and the excellent feedback he gave me as I prepared this dissertation. I know it was very difficult for him as he was out of the country but his responsive feedback to the numerous questions I had was very helpful. I would like to think Dr. Atila Ertas for the excellent feedback he gave on this dissertation paper. Dr. Ertas has been an outstanding resource and provided excellent guidelines on how to pursue a PhD. iii Texas Tech University, Jon Ilseng, December 2015 TABLE OF CONTENTS ABSTRACT 1 INTRODUCTION ........................................................................................................... 1 1.1 Background ...................................................................................................................... 1 1.1.1 Introduction ............................................................................................................. 1 1.2 Motivation ........................................................................................................................ 4 1.3 Goal .................................................................................................................................. 5 1.4 Significance ..................................................................................................................... 7 1.5 Research Question ........................................................................................................... 8 1.6 Specific Aims ................................................................................................................... 8 1.6.1 State the performance measures in industry, government and education (Aim 1) ...................................................................................................................................... 8 1.6.2 Identify System Relationships from a SoS perspective (Aim 2) ............................ 9 1.6.3 Propose a SoS sustainability Performance Measures (Aim 3) ............................... 9 1.7 Scope of Work ............................................................................................................... 10 1.8 Organization of the Dissertation .................................................................................... 11 2 Literature Search ............................................................................................................ 12 2.1 Sustainability Literature ................................................................................................. 12 2.2 System-of-Systems Literature........................................................................................ 15 2.3 Balanced Scorecard Framework Literature ................................................................... 16 2.4 Current performance measures collected by Government Organizations, For-profit companies, non-profit companies and educational institusions literature ........................... 16 2.5 Evidential Reasoning Literature .................................................................................... 24 2.6 Data Envelopment Analysis Literature .......................................................................... 25 2.7 Current Performance Measurement Systems Literature ................................................ 29 3 RESEARCH DESIGN AND METHODS ..................................................................... 39 3.1 Introduction .................................................................................................................... 39 3.2 Performance Measures (Aim 1) ..................................................................................... 41 iv Texas Tech University, Jon Ilseng, December 2015 3.2.1 Data Envelopment Analysis.................................................................................. 41 3.2.2 2 Evidential Reasoning ......................................................................................... 44 3.3 Identify System Relationships from a SoS perspective (Aim 2) ................................... 45 3.3.1 Introduction ........................................................................................................... 45 3.3.2 SoS Relationship ................................................................................................... 46 3.4 Propose a SoS sustainability Performance Measures (Aim 3) ...................................... 47 3.4.1 Introduction ........................................................................................................... 47 3.4.2 Approach ............................................................................................................... 47 4 IMPLEMENTATION .................................................................................................... 49 4.1 Introduction .................................................................................................................... 49 4.1.1 Case Studies for Government Agencies ............................................................... 50 4.1.2 Case Studies for For-Profit Organizations ............................................................ 54 4.1.3 Case Studies for Educational Institutions ............................................................. 58 4.1.4 Case Studies for Data Envelopment Analysis ...................................................... 61 4.1.5 Case Studies for Evidential Reasoning ................................................................. 61 5 METHODOLOGY ........................................................................................................ 64 6 SURVEY DEVELOPMENT AND DATA COLLECTION ......................................... 69 6.1 Cost Performance Index Data ........................................................................................ 69 6.2 Logistics Performance Index Data ................................................................................. 71 6.3 City of Dallas Measures ................................................................................................. 79 6.3.1 Economic Vibrancy Focus Area ........................................................................... 80 6.3.2 Office of Economic Development- Dallas Protocol and World Affairs ............... 83 6.3.3 Clean Healthy Environment Focus Area .............................................................. 85 6.3.4 Effective, Efficient and Economical Focus Area .................................................. 88 6.4 Texas Tech University Institutional Review Board Proposal ........................................ 91 7 RESULTS AND DISCUSSION .................................................................................... 92 8 CONCLUSIONS, FUTURE WORK AND CONTRIBUTIONS ................................ 107 Appendix A CPICTD DATA ............................................................................................ 137 Appendix B LPI DATA .................................................................................................. 155 v Texas Tech University, Jon Ilseng, December 2015 Appendix C - DALLAS ECONOMIC VIBRANCY FOCUS AREA DATA ................ 177 Appendix D - DALLAS CLEAN HEALTHY FOCUS AREA DATA .......................... 185 Appendix E DALLAS EFFECTIVE, EFFICIENT AND ECONOMICAL FOCUS AREA DATA ............................................................................................................................. 191 Appendix F TEXAS TECH UNIVERSITY INSTITUTIONAL REVIEW BOARD PROPOSAL .................................................................................................................... 197 APPENDIX G ONLINE SURVEY RESULTS .............................................................. 211 APPENDIX H COST LOGISTICS INDEX SURVEY RESULTS ............................... 239 APPENDIX I PERFORMANCE LOGISTICS INDEX SURVEY RESULTS .............. 243 APPENDIX J AVAILABILITY LOGISTICS INDEX SURVEY RESULTS............... 247 LIST OF TABLES Table 3-1: Data Envelopment Analysis Model ..................................................................41 Table 3-2: CSM performance dimensions and measures [112] .........................................42 Table 3.2.1-3-3: Inputs and outputs of public university teaching