Handbook on Constructing Composite Indicators
Total Page:16
File Type:pdf, Size:1020Kb
AVAILABLE ON LINE Handbook on Constructing Composite Indicators METHODOLOGY AND USER GUIDE This Handbook is a guide for constructing and using composite indicators for policy makers, academics, the media and other interested parties. While there are several types of composite indicators, this Handbook is concerned with those which compare and rank country performance in areas such as industrial competitiveness, sustainable development, globalisation and innovation. The Handbook aims to contribute to a better understanding of the complexity of composite indicators and to an improvement of the techniques currently used to build them. In particular, it contains a set of technical guidelines that can help constructors of composite indicators to improve the quality of their outputs. Handbook Handbook on Constructing Composite Indicators Composite Constructing on Handbook on Constructing Composite Indicators METHODOLOGY AND USER GUIDE METHODOLOGY AND USER GUIDE USER AND METHODOLOGY Subscribers to this printed periodical are entitled to free online access. If you do not yet have online access via your institution’s network, contact your librarian or, if you subscribe personally, send an email to [email protected]. ISBN 978-92-64-04345-9 30 2008 25 1 P ����������������������� -:HSTCQE=UYXYZ^: 001-002-999-eng.fm Page 1 Tuesday, August 19, 2008 7:41 AM Handbook on Constructing Composite Indicators METHODOLOGY AND USER GUIDE ORGANISATION FOR ECONOMIC CO-OPERATION AND DEVELOPMENT The OECD is a unique forum where the governments of 30 democracies work together to address the economic, social and environmental challenges of globalisation. The OECD is also at the forefront of efforts to understand and to help governments respond to new developments and concerns, such as corporate governance, the information economy and the challenges of an ageing population. The Organisation provides a setting where governments can compare policy experiences, seek answers to common problems, identify good practice and work to co-ordinate domestic and international policies. The OECD member countries are: Australia, Austria, Belgium, Canada, the Czech Republic, Denmark, Finland, France, Germany, Greece, Hungary, Iceland, Ireland, Italy, Japan, Korea, Luxembourg, Mexico, the Netherlands, New Zealand, Norway, Poland, Portugal, the Slovak Republic, Spain, Sweden, Switzerland, Turkey, the United Kingdom and the United States. The Commission of the European Communities takes part in the work of the OECD. OECD Publishing disseminates widely the results of the Organisation’s statistics gathering and research on economic, social and environmental issues, as well as the conventions, guidelines and standards agreed by its members. This work is published on the responsibility of the Secretary-General of the OECD. The opinions expressed and arguments employed herein do not necessarily reflect the official views of the Organisation or of the governments of its member countries or of the European Commission. Neither the European Commission nor any person acting on behalf of the Commission is responsible for the use which might be made of this publication. Corrigenda to OECD publications may be found on line at: www.oecd.org/publishing/corrigenda. © OECD 2008 You can copy, download or print OECD content for your own use, and you can include excerpts from OECD publications, databases and multimedia products in your own documents, presentations, blogs, websites and teaching materials, provided that suitable acknowledgment of OECD as source and copyright owner is given. All requests for public or commercial use and translation rights should be submitted to [email protected]. Requests for permission to photocopy portions of this material for public or commercial use shall be addressed directly to the Copyright Clearance Center (CCC) at [email protected] or the Centre français d'exploitation du droit de copie (CFC) [email protected]. FOREWORD This Handbook aims to provide a guide to the construction and use of composite indicators, for policy-makers, academics, the media and other interested parties. While there are several types of composite indicators, this Handbook is concerned with those which compare and rank country performance in areas such as industrial competitiveness, sustainable development, globalization and innovation. The Handbook aims to contribute to a better understanding of the complexity of composite indicators and to an improvement in the techniques currently used to build them. In particular, it contains a set of technical guidelines that can help constructors of composite indicators to improve the quality of their outputs. It has been jointly prepared by the OECD (the Statistics Directorate and the Directorate for Science, Technology and Industry) and the Econometrics and Applied Statistics Unit of the Joint Research Centre (JRC) of the European Commission in Ispra, Italy. Primary authors from the JRC are Michela Nardo, Michaela Saisana, Andrea Saltelli and Stefano Tarantola. Primary authors from the OECD are Anders Hoffmann and Enrico Giovannini. Editorial assistance was provided by Candice Stevens, Gunseli Baygan, Karsten Olsen and Sarah Moore. Many people contributed to improve this handbook with their valuable comments and suggestions. The authors wish to thank especially Jochen Jesinghaus from the European Commission, DG Joint Research Centre; Tanja Srebotnjak from the Yale Center for Environmental Law and Policy; Laurens Cherchye and Tom Van Puyenbroeck from the Catholic University of Leuven; Pascal Rivière from INSEE ; Tom Griffin, Senior Statistical Adviser to the UNDP Human Development Report; and Ari LATVALA from the European Commission, DG Enterprise and Industry. A special thank goes to Giuseppe Munda (Universitat Autònoma de Barcelona and European Commission, DG Joint Research Centre) who supplied all the material for the chapter on aggregation methods and the box on measurement scales and to Eurostat (Unit B5-'Methodology and research', DDG-02 ' Statistical governance, quality and evaluation' and other members of the Task Force on Composite Indicators) for the useful comments and the improvements they suggested. We are also grateful to the Statistical Offices of the OECD Committee on Statistics whose comments contributed to enhance the quality of this handbook. Further information on the topics treated in this handbook and on other issues related to composite indicators can be found in the web page: http://composite-indicators.jrc.ec.europa.eu/ The research was partly funded by the European Commission, Research Directorate, under the project KEI (Knowledge Economy Indicators), Contract FP6 No. 502529. In the OECD context, the work has benefited from a grant from the Danish Government. The views expressed are those of the authors and should not be regarded as stating an official position of either the European Commission or the OECD. HANDBOOK ON CONSTRUCTING COMPOSITE INDICATORS: METHODOLOGY AND USER GUIDE – ISBN 978-92-64-04345-9 - © OECD 2008 3 TABLE OF CONTENTS INTRODUCTION ....................................................................................................................... 13 Pros and cons of composite indicators ..................................................................................... 13 Aim of the Handbook ............................................................................................................... 14 What’s next .............................................................................................................................. 17 PART I. CONSTRUCTING A COMPOSITE INDICATOR ..................................................... 19 1. Steps for constructing a composite indicator .................................................................. 19 1.1 Developing a theoretical framework ............................................................................... 22 1.2 Selecting variables .......................................................................................................... 23 1.3 Imputation of missing data .............................................................................................. 24 1.4 Multivariate analysis ....................................................................................................... 25 1.5 Normalisation of data ...................................................................................................... 27 1.6 Weighting and aggregation ............................................................................................. 31 1.7 Robustness and sensitivity .............................................................................................. 34 1.8 Back to the details ........................................................................................................... 35 1.9 Links to other variables ................................................................................................... 39 1. 10 Presentation and dissemination ....................................................................................... 40 2. A quality framework for composite indicators ................................................................ 44 2.1 Quality profile for composite indicators .......................................................................... 44 2.2 Quality dimensions for basic data .................................................................................... 46 2.3 Quality dimensions for procedures to build and disseminate composite indicators ...................................................................