Inte grated Flood Risk Analysis and Management Methodologies Methodology for ex-post evaluation of measures and instruments in flood risk management (postEval) Au gust 2007 Report Nr: T12-07-01 Summary of Contents: Measures and Instruments Conditions Evaluation framework Criteria, indicators and methods Selection of indicators Case study results Conclusions and recommendations Co-ordinator: Paul Samuels HR Wallingford UK Project Contract No: GOCE-CT-2004-505420 Project website: www.floodsite.net FLOODsite Task 12 / Methodolgy for ex-post evaluation of measures and instruments DOCUMENT INFORMATION Methodology for ex-post evaluation of measures and instruments in Title flood risk management (postEval) Lead Author Alfred Olfert Contributors Jochen Schanze Distribution Project Team Olfert A and Schanze J (2007), Methodology for ex-post evaluation of measures and instruments in flood risk management (postEval), Document Reference Leibniz Institute for Ecological and Regional Development (IOER), FLOODsite Report T12-07-01, Dresden. DOCUMENT HISTORY Date Revision Prepared by Organisation Approved by Notes 07/01/07 1.0draft A. Olfert IOER 27/08/07 1.1draft A. Olfert IOER DISCLAIMER This report is a contribution to research generally and third parties should not rely on it in specific applications without first checking its suitability. In addition to contributions from individual members of the FLOODsite project consortium, various sections of this work may rely on data supplied by or drawn from sources external to the project consortium. Members of the FLOODsite project consortium do not accept liability for loss or damage suffered by any third party as a result of errors or inaccuracies in such data. Members of the FLOODsite project consortium will only accept responsibility for the use of material contained in this report in specific projects if they have been engaged to advise upon a specific commission and given the opportunity to express a view on the reliability of the material concerned for the particular application. © FLOODsite Consortium ii FLOODsite Task 12 / Methodolgy for ex-post evaluation of measures and instruments SUMMARY Design, implementation and operation of measures and instruments for effective and sustainable flood risk reduction require knowledge from previous practice. Systematic ex-post evaluation of interventions into the flood risk system and is an important basis for learning in support of future development of strategies and strategic options in flood risk management. Lacking methodological approaches currently prevent from comprehensive ex-post evaluation. As an effect, these remain uncommon, irregular and unsystematic. The report drafts a “Methodology for ex-post evaluation of pre-flood measures and instruments” (ex-post EFM) for the investigation of (side-)effects, effectiveness, efficiency, robustness and flexibility of physical measures and policy instruments. The methodology aims at providing a framework for the evaluation of measures and instruments after their implementation. The framework is laid out to be generically applicable with all measures and instruments at project level. By applying the methodology, information about existing measures and instruments shall be made available for the planning of future flood risk reduction. The Methodology addresses pre-flood and flood event measures and instruments at project level aimed at the reduction of flood risk respectively flood damage. Interventions in all elements of the Source-Pathway-Receptor-Consequences model are considers. Interventions of interest for ex-post evaluation are single measures and instruments or strongly connected combinations of those seen in the context of selected natural and societal conditions. The Methodology mainly consists of criteria and methods for the evaluation of physical measures and policy instruments. These aim at exploring effects (incl. side-effects), effectiveness, cost effectiveness, robustness and flexibility of existing interventions in to the flood risk system under reverting to experiences from recent flood events. The overall performance of the interventions is investigated under consideration of hydrological, ecological, social and economic aspects. Corresponding to the multiple criteria approach of the methodology a wide range of methods is used including quantitative as well as qualitative approaches. Natural and societal conditions are defined as part of the methodology and facilitate the case specific selection of criteria. The selection methodology enables a quick and systematic selection of appropriate criteria based on a partly formalised two step approach. A wide range of measures and instruments are identified and classified as basis for the methodology. These are presented in a newly developed classification system. Classification and the identified types of intervention are presented in a web-based information system. iii FLOODsite Task 12 / Methodolgy for ex-post evaluation of measures and instruments CONTENTS Document Information ii Document History ii Disclaimer ii Summary iii Contents iv Table of contents A METHODOLOGY................................................................................................................. 1 1. Introduction ....................................................................................................................1 1.1 Background.......................................................................................................... 1 1.2 Research questions for ex-post evaluation........................................................... 2 1.3 Structure of the methodology .............................................................................. 3 2. Measures and instruments............................................................................................. 6 2.1 Measures and instruments - interventions at project level................................... 6 2.2 Measures and instruments in the flood risk system ............................................. 7 2.3 Measures and instruments – a classification...................................................... 10 2.3.1 Why a new classification...................................................................... 10 2.3.2 Level 1: Physical measures and policy instruments............................. 11 2.3.3 Level 2: Functional character of measures and instruments ................ 12 2.3.4 Level 3: Types of measures and instruments ....................................... 14 2.4 Single measures and instruments and combinations of those............................ 16 3. Conditions for measures and instruments ................................................................. 18 3.1 Importance of conditions ................................................................................... 18 3.2 Type of flood ..................................................................................................... 19 3.3 Probability of flood............................................................................................ 20 3.4 Land use............................................................................................................. 20 3.5 Type of water body............................................................................................ 20 3.6 Specific conditions............................................................................................. 21 4. Evaluation framework ................................................................................................. 22 4.1 Criteria – main viewpoints of evaluation........................................................... 22 4.2 Indicators – yardsticks of evaluation ................................................................. 23 4.3 Evaluation framework ....................................................................................... 24 5. Indicators, evaluation of criteria and evaluation methods ....................................... 27 5.1 Evaluation of effects.......................................................................................... 27 5.1.1 Impacts of flooding and flood risk reduction ....................................... 27 5.1.2 Categories of effects............................................................................. 28 5.1.3 Indicators of hydrological and hydraulic effects.................................. 29 5.1.4 Indicators of socio-cultural effects....................................................... 29 5.1.5 Indicators of economic effects ............................................................. 30 5.1.6 Indicators of ecological effects ............................................................ 32 5.1.7 Determination of effects....................................................................... 37 iv FLOODsite Task 12 / Methodolgy for ex-post evaluation of measures and instruments 5.2 Evaluation of effectiveness................................................................................ 40 5.2.1 Effectiveness - the degree of goal achievement ................................... 40 5.2.2 Objectives............................................................................................. 42 5.2.3 Determination of effectiveness............................................................. 42 5.3 Evaluation of efficiency – the benefit/cost ratio................................................ 45 5.3.1 Economic and non–economic effects in cost-effectiveness ................. 45 5.3.2 Benefits and costs in cost-effectiveness