Evaluation of Human Space Missions and International Space Station Utilization
Total Page:16
File Type:pdf, Size:1020Kb
Evaluation of Human Space Missions and International Space Station Utilization Period from April 2013 to March 2018 Project No. 17/18-02-01 Prepared by the Audit & Evaluation Directorate December 2019 EVALUATION OF HUMAN SPACE MISSIONS AND UTILIZATION OF THE ISS PROJECT NO. 17/18 – 02-01 Table of Contents List of figures and tables .............................................................................................................................. iii Acronyms used in the report ....................................................................................................................... iv Summary ....................................................................................................................................................... v 1 Introduction .......................................................................................................................................... 1 2 Description of the program .................................................................................................................. 1 2.1 General context of the program ................................................................................................... 1 2.2 Program logic model, activities, clients and partners ................................................................... 3 2.3 Program resources ........................................................................................................................ 8 Financial resources ................................................................................................................ 8 Human resources .................................................................................................................. 8 Management structure ......................................................................................................... 9 3 Approach and evaluation methods .................................................................................................... 10 3.1 Purpose and scope ...................................................................................................................... 10 3.2 Methods ...................................................................................................................................... 11 Comparative study and literature review ........................................................................... 11 Document review ................................................................................................................ 11 Interviews with key informers ............................................................................................ 11 Case studies......................................................................................................................... 12 Internal data ........................................................................................................................ 12 3.3 Limitations................................................................................................................................... 13 4 Findings of the evaluation .................................................................................................................. 14 4.1 Relevance .................................................................................................................................... 14 How did the program meet needs? .................................................................................... 14 How are the program objectives in keeping with federal government priorities? ............ 29 What are the roles and responsibilities of the federal government in relation to the operation of the program? ................................................................................................. 30 4.2 Results ......................................................................................................................................... 31 Data on performance .......................................................................................................... 31 How were the expected outputs achieved? ....................................................................... 32 How were the expected immediate results achieved?....................................................... 35 How were the expected intermediate results achieved? ................................................... 36 How were the expected final results achieved? ................................................................. 38 AUDIT AND EVALUATION DIRECTORATE i EVALUATION OF HUMAN SPACE MISSIONS AND UTILIZATION OF THE ISS PROJECT NO. 17/18 – 02-01 Did the implementation of the program have unexpected negative or positive impacts? 40 4.3 Efficiency and economy .............................................................................................................. 41 Leveraging of partnerships ................................................................................................. 41 ISS model ............................................................................................................................. 44 CSA model ........................................................................................................................... 45 Are there more cost-effective ways to implement the current program? ......................... 48 4.4 What are the impacts of the program on GBA+ groups? ........................................................... 48 5 Action plan .......................................................................................................................................... 49 AUDIT AND EVALUATION DIRECTORATE ii EVALUATION OF HUMAN SPACE MISSIONS AND UTILIZATION OF THE ISS PROJECT NO. 17/18 – 02-01 List of figures and tables Figure 1: Program logic model ...................................................................................................................... 3 Figure 2: Chronology of the CSA’s scientific instruments……………………........................ ............................... 5 Figure 3: Elements of the logic model demonstrating the relation of awareness activities to the final results…………………......................................................................................................................28 Figure 4: Research area for each ISS partner agency ................................................................................. 42 Table 1: Resources assigned to the program ................................................................................................ 8 Table 2: Number of full-time equivalents, expected and actual per year (2013–2014 to 2017–2018) ....... 9 Table 3: Evaluation questions for Human Space Missions and ISS Utilization ........................................... 10 Table 4: Number of interviews and respondents by category………………………………………..…..………………….12 Table 5: Human Health and Performance Risks Summary. ........................................................................ 19 Table 6: Research, scientific instruments and space sub-systems ............................................................. 33 Table 7: ISS utilization time ........................................................................................................................ 33 Table 8: Results of output indicators related to astronauts ....................................................................... 35 Table 9: Achievement of immediate targeted results………………………………………………………………………..…. 36 Table 10: The benefits of the ISS increase thanks to international collaboration ...................................... 42 AUDIT AND EVALUATION DIRECTORATE iii EVALUATION OF HUMAN SPACE MISSIONS AND UTILIZATION OF THE ISS PROJECT NO. 17/18 – 02-01 Acronyms used in the report AO Announcement of Opportunity ARC Average Relative Citation CSA Canadian Space Agency ESA European Space Agency EVAs Extra-vehicular activities FTE Full-time equivalent GBA+ Gender-based analysis plus HQP Highly Qualified Personnel IGA Intergovernmental Agreement IIRB Integrated Investment Review Board INO Institut national d’optique ISECG International Space Exploration Coordination Group ISLSWG International Space Life Science Working Group ISS International Space Station JAXA Japanese Aerospace Exploration Agency LEO Low Earth Orbit MOU Memoranda of Understanding MSS Mobile Servicing System NASA National Aeronautics and Space Administration NFB National Film Board of Canada NPO Non-profit organization OECD Organization for Economic Cooperation and Development PM Strategy Performance Measurement Strategy R&D Research and Development SP Sub-program SSP Sub-sub-program STEM Science, technology, engineering and mathematics TB Treasury Board of Canada UNO United Nations Organization AUDIT AND EVALUATION DIRECTORATE iv EVALUATION OF HUMAN SPACE MISSIONS AND UTILIZATION OF THE ISS PROJECT NO. 17/18 – 02-01 Summary This report covers the findings of the evaluation of sub-program 1.2.3. Human Space Missions and Support and sub-sub-program 1.2.1.2. International Space Station Utilization (hereinafter called the “program”), implemented by the Canadian Space Agency (CSA) for the period from April 1, 2013 to March 31, 2018. In 1985, the Canadian Space Station Program began when Prime Minister Brian Mulroney accepted the invitation by President of the United States Ronald Reagan to participate in the multilateral program to develop and build the International Space Station (ISS). The initial partners were the United