Evaluation of NRC Measurement Science and Standards
Total Page:16
File Type:pdf, Size:1020Kb
Office of Audit and Evaluation Evaluation of NRC Measurement Science and Standards June 18, 2015 Template 1 Prepared by: Office of Audit and Evaluation National Research Council Canada Approval: This report was approved by NRC’s President on June 18, 2015 Evaluation of NRC Measurement Science and Standards June 18, 2015 TABLE OF CONTENTS Table of Tables ......................................................................................................................... ii Table of Figures ....................................................................................................................... ii Acknowledgements .................................................................................................................iii Acronyms and Abbreviations ................................................................................................. v Executive Summary ................................................................................................................vii 1. Introduction ....................................................................................................................... 1 1.1 Evaluation Overview .................................................................................................... 1 2. Profile of the MSS Portfolio .............................................................................................. 4 2.1 Support to NRC Programs ........................................................................................... 4 2.2 International Bureau of Weights and Measures (BIPM) ................................................ 5 2.3 Canada’s National Measurement System .................................................................... 5 2.4 MSS Organizational Structure ...................................................................................... 6 2.5 Portfolio Resources ...................................................................................................... 7 2.6 Profile of MSS Clients .................................................................................................. 8 3. Relevance .........................................................................................................................10 3.1 The Need for Metrology and a National Metrology Institute in Canada ........................10 3.2 Meeting the Needs of Other Government Departments ..............................................15 3.3 Alignment with Federal Roles, Responsibilities and Priorities .....................................18 4. Performance .....................................................................................................................21 4.1 Positioning and Progress toward the Achievement of Expected Outcomes .................21 4.2 Impacts on Clients ......................................................................................................24 4.3 International Reputation for Metrology ........................................................................29 5. Resource Utilization ........................................................................................................33 5.1 Operational Efficiency .................................................................................................33 6. Conclusion .......................................................................................................................37 7. Management Response ...................................................................................................38 Appendix A: MSS Logic Model ........................................................................................... A-1 Appendix B: Methodology ................................................................................................... B-1 Appendix C: Selection of Documents Reviewed ................................................................. C-1 i Evaluation of NRC Measurement Science and Standards June 18, 2015 TABLE OF TABLES Table 1: Scale used when reporting on views and opinions of respondents ............................... 3 Table 2: MSS financial data ($) .................................................................................................. 7 Table 3: Canada’s contribution to the BIPM ............................................................................... 8 Table 4: OGD users of Canada’s national measurement system and of MSS products and services.....................................................................................................................................16 Table 5: Canadian participation on CIPM committees ...............................................................30 Table 6: MSS cross-program human resource utilization (data as of October 2014) .................35 Table 7: Stakeholders interviewed .......................................................................................... B-3 Table 8: Representation of case study clients within the MSS-hosted programs ..................... B-5 T A B L E O F FIGURES Figure 1: MSS sources of revenue, by client type (A) and by contract type (B) (2013-14) .......... 9 Figure 2: Revenue from sales of goods and information products (2009-10 to 2013-14) ...........13 Figure 3: Position and influence of federal players in Canada’s national measurement system 17 Figure 4: MSS revenues from OGD clients as compared to revenues from other clients ...........18 Figure 5: Percentage of revenue targets met (2013-14) ............................................................23 Figure 6: Number of publications with an affiliation to MSS .......................................................31 Figure 7: Total number of key comparisons (top 15 nations as of May 2014) ............................32 ii Evaluation of NRC Measurement Science and Standards June 18, 2015 ACKNOWLEDGEMENT S The Office of Audit and Evaluation would like to gratefully acknowledge the management and staff of the Measurement Science and Standards (MSS) Portfolio for their support and contribution to this evaluation. We would also like to acknowledge the contribution of knowledge and expertise from other branches of NRC, including Finance, Business Management Support, Communications, Knowledge Management, and Planning and Reporting Services. The evaluation team would also like to express its gratitude to members of the Evaluation Advisory Committee for their indispensable guidance and advice. Finally, the evaluation team would like to thank the MSS clients and other stakeholders who provided rich information and insights in support of this evaluation. National Research Council iii Evaluation of NRC Measurement Science and Standards June 18, 2015 National Research Council iv Evaluation of NRC Measurement Science and Standards June 18, 2015 ACRONYMS AND ABBREVI ATIONS AAFC Agriculture and Agri-Food Canada AECL Atomic Energy of Canada Limited ASPM Administrative Services and Property Management BIPM Bureau International des poids et mesures BMS Business Management Support CBSA Canada Border Services Agency CFIA Canadian Food Inspection Agency CGPM General Conference on Weights and Measures CIPM Comité international des poids et mesures CLAS Calibration Laboratory Assessment Service CMC Calibration and Measurement Capabilities CNC Cellulose nanocrystals CNSC Canadian Nuclear Safety Commission CRM Certified reference material DFATD Department of Foreign Affairs, Trade and Development DFO Department of Fisheries and Oceans DL Discipline Leader DND Department of National Defence EAC Evaluation Advisory Committee EC Environment Canada FTE Full-time equivalent GDMS Glow discharge mass spectrometry GDP Gross Domestic Product GM General Manager GoCo Government-owned contractor-operated HC Health Canada HHT Human Health Therapeutics HR Human resources IC Industry Canada IEC International Electrotechnical Commission ILAC International Laboratory Accreditation Cooperation IMB Institute for Marine Biology INMS Institute for National Measurement Standards IP Intellectual property ISO International Standards Organization ITSS Information Technology and Security Services MC Measurement Canada MIS Metrology for Industry and Society MRA Mutual Recognition Arrangement MSET Measurement Science for Emerging Technologies MSS Measurement Science and Standards NINT National Institute of Nanotechnology NIST National Institute of Standards and Technology NMI National metrology institute NMR Nuclear Magnetic Resonance NPL National Physical Laboratory NRC National Research Council NRCan Natural Resources Canada National Research Council v Evaluation of NRC Measurement Science and Standards June 18, 2015 OAE Office of Audit and Evaluation OAS Organization of American States OGD Other government department PMO Project management office PS Public Safety Canada QHR Quantum Hall Resistance RCMP Royal Canadian Mounted Police R&D Research and development RTL Resource Team Leader RTO Research and technology organization SCC Standards Council of Canada SEC Senior Executive Committee SI Système International (international system of units) SSNMS Scientific Support for the National Measurement System TC Transport Canada U.K. United Kingdom U.S. United States National Research Council vi Evaluation of NRC Measurement Science and Standards June 18, 2015 EXECUTIVE SUMM ARY This report presents the results of the 2014-15 evaluation of the National Research Council (NRC) Measurement Science and Standards (MSS) Portfolio. MSS is Canada’s national metrology institute (NMI), conducting research