Advances in Geosciences www.adv-geosci.net 1680-7340 1680-7359 20 Observation, Prediction and Verification of Precipitation (EGU Session 2008) 2009 10.5194/adgeo-20-19-2009 http://www.adv-geosci.net/20/19/2009/ http://www.adv-geosci.net/20/19/2009/adgeo-20-19-2009.html http://www.adv-geosci.net/20/19/2009/adgeo-20-19-2009.pdf 19 23 2009-03-12 Relationship between forecast precipitation relative errors and skill scores: the case of rare event frequencies N. Tartaglione nazario.tartaglione@unicam.it Department of Physics, University of Camerino, Camerino, Italy School of Mathematical Sciences, University College Dublin, Dublin, Ireland This paper addresses the problem of the relationship between skill scores and forecast rainfall relative errors. The problem is approached by using synthetic time series of rainfall data representing the observations. It is assumed that the magnitude of the relative error is known. The forecasts are constructed by adding errors to the observations. We use a threshold to dichotomise forecasts and observations to obtain the skill scores. We perform 1000 simulations for each error magnitude in order to obtain the mean values and uncertainties of the scores. <br><br> We consider two different precipitation regimes, and we show the influence of these regimes on the precipitation. We find that the relationship between forecast errors and skill scores is strongly influenced by the event frequencies, which in turn depend on the precipitation regime. We find that only when the event frequency of the two regimes is made similar by changing the threshold, the relationship between the scores and relative errors is similar. This suggests that a comparison between two forecast precipitation datasets should account for the difference (if any) in precipitation regimes. Accadia, C., Mariani, S., Casaioli, M., Lavagnini, A., and Speranza, A.: Sensitivity of Precipitation Forecast Skill Scores to Bilinear Interpolation and a Simple Nearest-Neighbour Average Method on High-Resolution Verification Grids, Wea. Forecasting, 18, 918–932, 2003. Baldwin, M. E. and Kain, J. S.: Sensitivity of several performance measures to displacement error, bias, and event frequency, Wea. Forecasting, 21, 636–648, 2006. Briggs, W., Pocernich, M., and Ruppert, D.: Incorporating Misclassification Error in Skill Assessment, Mon. Weather Rev., 133, 3382–3392, 2005. Flores, C.: Multiplicative cascade models for rain in hydro-meteorological disasters risk management, 35. ASTIN-Kolloquium, Bergen, Norway, 6–9~June 2004 (available at http://www.actuaries.org/ASTIN/Colloquia/Bergen/Flores.pdf), 2004. Hanssen, A. W. and Kuipers, W. J. A.: On the relationshipship between the frequency of rain and various meteorological parameters. Koninklijk Nederlands Meteorologisch Institut, Meded. Verhand., 81, 2–15, 1965. Hamill, T. M.: Hypothesis tests for evaluating numerical precipitation forecasts, Wea. Forecasting, 14, 155–167, 1999. Hamill, T. M. and Juras, J.: Measuring forecast skill: is it real skill or is it the varying climatology?, Q. J. Roy. Meteor. Soc., 132, 2905–2923, 2006. Jolliffe, I. T.: Uncertainty and inference for verification measures, Wea. Forecasting, 22, 637–650, 2007. Jolliffe, I. T. and Stephenson, D. B.: Forecast Verification: A Practitioner's Guide in Atmospheric Science, Wiley: Chichester, 240 pp., 2003. Livezey, R. E., and Chen, W.: Statistical Field Significance and its Determination by Monte Carlo Techniques, Mon. Wea. Rev., 111, 46–59, 1983. Mason, S. J.: Understanding forecast verification statistics, Meteor. Appl., 15, 31–40, 2008. Mass, C. F., Ovens, D., Westrick, K., and Colle, B. A.: Does Increasing Horizontal Resolution Produce More Skillful Forecasts?, B. Am. Meteor. Soc., 83, 407–430, 2002. Tartaglione, N.: Relationship between Precipitation Forecast Errors and Skill Scores of Dichotomous Forecasts, Wea. Forecasting, submitted, 2009. Woodcock, F.: The Evaluation of Yes/No Forecasts for Scientific and Administrative Purposes, Mon. Weather Rev., 104, 1209–1214, 1976.