The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology Abstracts of the Fifth International Verification Methods Workshop, 1 - 7 December 2011, Melbourne, Australia CAWCR Technical Report No. 046 Keith A. Day (editor) November 2011 Abstracts of the Fifth International Verification Methods Workshop, 1 - 7 December 2011 Melbourne, Australia Keith A. Day (Editor) Centre for Australian Weather and Climate Research, GPO Box 1289, Melbourne, VIC 3001, Australia CAWCR Technical Report No. 046 November 2011 Series ISSN: 1836-019X National Library of Australia Cataloguing-in-Publication entry Author: International Verification Methods Workshop (2011 : Melbourne, Victoria) Title: Abstracts of the Fifth International Verification Methods Workshop, 1 - 7 December 2011, Melbourne, Australia / Editor Keith A. Day. ISBN: 978-0-643-10730-4 (Electronic Resource – PDF) Series: CAWCR technical report; No. 46 Notes: Includes index. Subjects: Meteorology--Research--Congresses. Enquiries should be addressed to: Keith Day Centre for Australian Weather and Climate Research: A partnership between the Bureau of Meteorology and CSIRO GPO Box 1289 Melbourne VIC 3001 Australia [email protected] Phone: 61 3 9669 8311 Fax: 61 3 9669 4660 Fifth International Verification Workshop Program Committee Beth Ebert (BOM, Australia) Laurie Wilson (CMC, Canada) Barb Brown (NCAR, USA) Barbara Casati (Ouranos, Canada) Caio Coelho (CPTEC, Brazil) Anna Ghelli (ECMWF, UK) Martin Göber (DWD, Germany) Simon Mason (IRI, USA) Marion Mittermaier (Met Office, UK) Pertti Nurmi (FMI, Finland) Joel Stein (Météo-France) Yuejian Zhu (NCEP, USA) Copyright and Disclaimer © 2011 CSIRO and the Bureau of Meteorology. To the extent permitted by law, all rights are reserved and no part of this publication covered by copyright may be reproduced or copied in any form or by any means except with the written permission of CSIRO and the Bureau of Meteorology. CSIRO and the Bureau of Meteorology advise that the information contained in this publication comprises general statements based on scientific research. The reader is advised and needs to be aware that such information may be incomplete or unable to be used in any specific situation. No reliance or actions must therefore be made on that information without seeking prior expert professional, scientific and technical advice. To the extent permitted by law, CSIRO and the Bureau of Meteorology (including each of its employees and consultants) excludes all liability to any person for any consequences, including but not limited to all losses, damages, costs, expenses and any other compensation, arising directly or indirectly from using this publication (in part or in whole) and any information or material contained in it. Contents Foreword by Beth Ebert and Laurie Wilson.......................................................................................... 1 Abstracts Andreas P. Weigel Verification of ensemble forecasts: a guided tour through a zoo of skill metrics ........................... 2 Yuejian Zhu Ensemble based probabilistic forecast verification.............................................................................. 3 Zied Ben Bouallegue, Susanne Theis and, Christoph Gebhardt From verification results to probabilistic products:Spatial techniques applied to ensemble forecasting.................................................................................................................................................. 4 Andrew B. Watkins Validation, verification and value: the brave new world of operational climate forecasting......... 5 Harvey Stern and Jonathan Pollock Verifying the accuracy of two decades of seasonal climate outlooks ............................................... 6 Oscar Alves Working Group on Seasonal to Interannual Prediction (WGSIP) ..................................................... 7 Damien Irving, Sarah Perkins, Josephine Brown, Alex Sen Gupta, Aurel Moise, Bradley Murphy, Les Muir, Robert Colman, Scott Power, Francois Delage and Jaclyn Brown Climate projection evaluation: a Pacific Climate Change Science Program perspective............... 8 Jack Katzfey and Mohar Chattopadhyay Evaluation of high resolution regional climate simulations over the islands in the South Pacific......................................................................................................................................................... 9 Aurel Moise, Damien Irving and Penny Whetton Global climate model evaluation plans at CAWCR .......................................................................... 10 Philip Gill Recent developments in aviation verification at the UK Met Office .............................................. 12 David Aihoshi PMTIP: A user based trend for aviation performance metrics........................................................ 13 Jennifer Mahoney, Missy Petty, Geary Layne, Steve Lack, Matt Wandishin, Mike Kay, Andy Loughe, Brian Pettegrew, and Joan Hart Impact-based verification: A bridge between weather forecast and operational decision .......... 14 Vivek C. Nagar A two layer scheme to verify aerodrome forecasts (TAF)................................................................ 15 Rodney Potts, Tali Boneh, Yuelong Miao and Rabi Rivett Fog forecasting at Perth Airport – application of a Bayesian Decision Network......................... 16 i Susan Joslyn Uncertainty forecasts and the end-user.................................................................................................17 Timothy Bullock, Ming Szeto and Laurence Wilson User-oriented verification: The concept of forecaster intent and its application to verification systems ......................................................................................................................................................18 Väinö Nurmi, Pertti Nurmi and Adriaan Perrels Expected impacts and value of improvements in weather forecasting on the transportation .......19 Michael Sharpe Verification of the UK Shipping Forecast using a new objective area forecast verification system........................................................................................................................................................20 John Louey and Milos Setek Verification presentation in management reporting ...........................................................................21 Simon Caine Statistical assessment of tropical convection-permitting model simulations using a cell- tracking algorithm....................................................................................................................................22 Edward Tollerud, Tara Jensen, Barbara Brown, Tressa Fowler, Wally Clark, Randy Bullock, John Halley Gotway, and Paul Oldenburg Object-based spatial verification techniques for diagnostic analyses of precipitation patterns during extreme rainfall episodes............................................................................................................23 G.T Weymouth amd N. Faggian Morphing for grid editing and verification in an operational weather forecast system.................24 T.-Y. Koh, B. C. Bhatt, K. K. W. Cheung, C. K. Teo, Y. H. Lee, Matthias Roth and Purnawirman Using the spectral scaling exponent for validation of quantitative precipitation forecasts...........26 Paul A. Kucera, Courtney Weeks, Barbara Brown, and Randy Bullock Development of new diagnostic tools to evaluate NWP cloud and precipitation products using A-Train satellite observations .....................................................................................................27 Belinda Roux, Alan Seed, Robert Dahni, and Ryan Smith Conditional rainfall verification using synoptic typing......................................................................28 Barbara G. Brown Evaluation methods for tropical cyclone forecasts .............................................................................29 Elizabeth Ebert Radius of reliability: A distance metric for verifying spatial probability forecasts .......................30 Tressa L. Fowler, Eric Gilleland and Barbara G. Brown Some new techniques for hurricane verification.................................................................................31 Gary Brassington Validation and verification methods applied to ocean forecasting...................................................32 ii Abstracts of the International Verification Methods Workshop 1 - 7 December 2011, Melbourne, Australia. Barbara G. Brown, Rita D. Roberts, Amanda R. S. Anderson, Eric Nelson and Matthew Pocernich Regional evaluation of convective nowcasts and application of contingency table performance diagrams............................................................................................................................ 33 Laurence Wilson, Andrew Giles and Erik Buhler Incorporating timeliness information into an accuracy measure for Canadian weather warnings ................................................................................................................................................... 34 Martin Göber On user, process and model oriented warning verification............................................................... 35 Tanja Weusthoff Verification of operational (high impact) weather warnings............................................................ 36 Ian Jolliffe Verification of wind direction forecasts .............................................................................................