Advances in Geosciences www.adv-geosci.net 1680-7340 1680-7359 25 Precipitation: Measurement, Climatology, Remote Sensing, and Modeling (EGU Session 2009) 2010 10.5194/adgeo-25-29-2010 http://www.adv-geosci.net/25/29/2010/ http://www.adv-geosci.net/25/29/2010/adgeo-25-29-2010.html http://www.adv-geosci.net/25/29/2010/adgeo-25-29-2010.pdf 29 36 2010-03-09 The forecaster's added value in QPF M. Turco mturco@am.ub.es M. Milelli ARPA Piemonte (Regional Environmental Protection Agency), Torino, Italy now at: GAMA (Meteorological Hazards Analysis Team), Department of Astronomy & Meteorology, Faculty of Physics, University of Barcelona, Barcelona, Spain To the authors' knowledge there are relatively few studies that try to answer this question: "Are humans able to add value to computer-generated forecasts and warnings?". Moreover, the answers are not always positive. In particular some postprocessing method is competitive or superior to human forecast. Within the alert system of ARPA Piemonte it is possible to study in an objective manner if the human forecaster is able to add value with respect to computer-generated forecasts. Every day the meteorology group of the Centro Funzionale of Regione Piemonte produces the HQPF (Human Quantitative Precipitation Forecast) in terms of an areal average and maximum value for each of the 13 warning areas, which have been created according to meteo-hydrological criteria. This allows the decision makers to produce an evaluation of the expected effects by comparing these HQPFs with predefined rainfall thresholds. Another important ingredient in this study is the very dense non-GTS (Global Telecommunication System) network of rain gauges available that makes possible a high resolution verification. In this work we compare the performances of the latest three years of QPF derived from the meteorological models COSMO-I7 (the Italian version of the COSMO Model, a mesoscale model developed in the framework of the COSMO Consortium) and IFS (the ECMWF global model) with the HQPF. In this analysis it is possible to introduce the hypothesis test developed by Hamill (1999), in which a confidence interval is calculated with the bootstrap method in order to establish the real difference between the skill scores of two competitive forecasts. It is important to underline that the conclusions refer to the analysis of the Piemonte operational alert system, so they cannot be directly taken as universally true. But we think that some of the main lessons that can be derived from this study could be useful for the meteorological community. In details, the main conclusions are the following: <br><br> &ndash; despite the overall improvement in global scale and the fact that the resolution of the limited area models has increased considerably over recent years, the QPF produced by the meteorological models involved in this study has not improved enough to allow its direct use: the subjective HQPF continues to offer the best performance for the period +24 h/+48 h (i.e. the warning period in the Piemonte system); <br><br> &ndash; in the forecast process, the step where humans have the largest added value with respect to mathematical models, is the communication. In fact the human characterization and communication of the forecast uncertainty to end users cannot be replaced by any computer code; <br><br> &ndash; eventually, although there is no novelty in this study, we would like to show that the correct application of appropriated statistical techniques permits a better definition and quantification of the errors and, mostly important, allows a correct (unbiased) communication between forecasters and decision makers. Accadia, C., Casaioli, M., Mariani, S., Lavagnini, A., Speranza, A., de Venere, A., Inghilesi, R., Ferretti, R., Paolucci, T., Cesari, D., Patruno, P., Boni, G., Bovo, S., Cremonini, R.: Application of a statistical methodology for limited area model intercomparison using a bootstrap technique, Il Nuovo Cimento No. 26, 61–77, 2003a. Accadia, C., Mariani, S., Casaioli, M., and Lavagnini, A.: Sensitivity of precipitation forecast skill scores to bilinear interpolation and simple neareast-neighbor average method on high-resolution verification grids, Weather Forecast., 18, 918–832, 2003b. Baars, J. A. and Mass, C. F.: Performance of National Weather Service forecasts compared to operational, consensus, and weighted model output statistics, Weather Forecast, 20, 1034–1047, 2005. Campus, S., Barbero, S., Bovo, S., Forlati, F.: Evaluation and Prevention of Natural Risks, Taylor & Francis, 2007. Casati, B., Wilson, L. J., Stephenson, D. B., Nurmi, P., Ghelli, A., Pocernich M., Damrath, U., Ebert, E. E., Brown, B. G. and Mason, S.: Review Forecast verification: current status and future directions, Meteorol. Appl., 15, 3–18, 2008. Charba, J. P., Reynolds, D. W., McDonald, B. E. and Carter, G. M.: Comparative Verification of Recent Quantitative Precipitation Forecasts in the National Weather Service: A Simple Approach for Scoring Forecast Accuracy, Weather Forecast., 18, 161–183, 2003. Gigerenzer, G.: Reckoning with Risk. Learning to Live with Uncertainty, London Penguin, 2002. Göber M., Wilson, C. A., Milton, S. F., and Stephenson, D. B.: Fairplay in the verification of operational quantitative precipitation forecasts, J. Hydrol., 288, 225–236, 2004. Harvey, L. O. Jr., Hammond, K. R., Lusk, C. M., and Mross, E. F.: The application of signal detection theory to weather forecasting behavior, Mon. Weather Rev., 120, 863–883, 1992. Hamill, T. M.: Hypothesis tests for evaluating numerical precipitation forecasts, Weather Forecast., 14, 155–167, 1999. Jolliffe, I. T. and Stephenson, D. B.: Forecast Verification: A Practitioner's Guide in Atmospheric Science, John Wiley and Sons, 2003. Mason S.: Understanding forecast verification statistics, Meteorol. Appl., 15, 31–40, 2008. Rabuffetti, D. and Barbero, S.: Operational hydro-meteorological warning and real-time flood forecasting: the Piemonte Region case study, Hydrol. Earth Syst. Sci., 9, 457–466, 2005. Richardson, D. S.: Skill and relative economic value of the ECMWF ensemble prediction system, Q. J. Roy. Meteor. Soc., 126, 649–667, 2000. Roebber, P. J. and Bosart, L. F.: The contributions of education and experience to forecast skill. Weather and Forecasting No. 11, 21-40, 1996a. Roebber, P. J., Bosart, L. F. and Forbes, G. J.: Does distance from the forecast site affect skill? Weather Forecast., 11, 582–589, 1996b. Roebber, P. J. and Bosart, L. F.: The Complex Relationship between Forecast Skill and Forecast Value: A Real-World Analysis, Weather Forecast, 11, 544–559, 1996c. Sanders, F.: Trends in skill of Boston forecasts made at MIT, 1966–84. B. Am. Meteorol. Soc., 67, 170–176, 1986. Stephenson, D. B.: Use of the odds ratio for diagnosing forecast skill. Weather Forecast., 15, 221–232, 2000. Thornes, J. E. and Stephenson, D. B.: How to judge the quality and value of weather forecast products, Meteorol. Appl., 8, 307–314, 2001. Wilks, D. S.: Statistical Methods in the Atmospheric Sciences, Academic Press, 2006. World Meteorological Organization (WMO): Guidelines on communicating forecast uncertainty. WMO/TD No 1422, available at: www.wmo.int, 2008.