References

ADGEO

Advances in Geosciences

ADGEO

1680-7359

Copernicus GmbH

Göttingen, Germany

10.5194/adgeo-16-137-2008

Bias Adjusted Precipitation Threat Scores

Mesinger

NCEP/Environmental Modeling Center, Camp Springs, Maryland, and Earth System Science Interdisciplinary Center (ESSIC) University of Maryland, College Park, Maryland, USA

09 04 2008

16 16 137 142

This is an open-access article ditributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

This article is available from http://www.adv-geosci.net/16/137/2008/adgeo-16-137-2008.html

The full text article is available as a PDF file from http://www.adv-geosci.net/16/137/2008/adgeo-16-137-2008.pdf

Among the wide variety of performance measures available for the assessment of skill of deterministic precipitation forecasts, the equitable threat score (ETS) might well be the one used most frequently. It is typically used in conjunction with the bias score. However, apart from its mathematical definition the meaning of the ETS is not clear. It has been pointed out (Mason, 1989; Hamill, 1999) that forecasts with a larger bias tend to have a higher ETS. Even so, the present author has not seen this having been accounted for in any of numerous papers that in recent years have used the ETS along with bias "as a measure of forecast accuracy". A method to adjust the threat score (TS) or the ETS so as to arrive at their values that correspond to unit bias in order to show the model's or forecaster's accuracy in \textit{placing} precipitation has been proposed earlier by the present author (Mesinger and Brill, the so-called dH/dF method). A serious deficiency however has since been noted with the dH/dF method in that the hypothetical function that it arrives at to interpolate or extrapolate the observed value of hits to unit bias can have values of hits greater than forecast when the forecast area tends to zero. Another method is proposed here based on the assumption that the increase in hits per unit increase in false alarms is proportional to the yet unhit area. This new method removes the deficiency of the dH/dF method. Examples of its performance for 12 months of forecasts by three NCEP operational models are given.

References 1

Baldwin, M. E. and Kain, J. S.: Sensitivity of several performance measures to displacement error, bias, and event frequency, Wea. Forecasting, 21, 636–648, 2006.

Davis, C., Brown, B., and Bullock, R.: Object-based verification of precipitation forecasts. Part I: Methodology and application to mesoscale rain areas, Mon. Weather Rev., 134, 1772–1784, 2006.

Ebert, E. E. and McBride, J. L.: Verification of precipitation in weather systems: Determination of systematic errors, J. Hydrol., 239, 179–202, 2000.

GCWM Branch, EMC: The GFS Atmospheric Model. NCEP Office Note 442, 14 pp., available at: http://wwwt.emc.ncep.noaa.gov/officenotes, 2003.

Gilbert, G. F.: Finley's tornado predictions, Amer. Meteorol. J., 1, 166–172, 1884.

Hamill, T. M.: Hypothesis tests for evaluating numerical precipitation forecasts, Wea. Forecasting, 14, 155–167, 1999.

Janji\'c, Z. I.: A nonhydrostatic model based on a new approach, Meteorol. Atmos. Phys., 82, 271–285, 2003.

Mason, I.: Dependence of the critical success index on sample climate and threshold probability, Aust. Meteorol. Mag., 37, 75–81, 1989.

Mesinger, F. and Brill, K.: Bias normalized precipitation scores. Preprints, 17th Conf. on Probability and Statistics, Seattle, WA, Amer. Meteorol. Soc., CD-ROM, J12.6, 2004.

Pielke, R. A., Sr.: Mesoscale Meteorological Modeling, 2~ed., Academic Press, 676 pp., 2002.

Schaefer, J. T.: The critical success index as an indicator of warning skill, Wea. Forecasting, 5, 570–575, 1990.

Shuman, F. G.: A modified threat score and a measure of placement error. NOAA/NWS Office Note 210, 13 pp., available at: http://wwwt.emc.ncep.noaa.gov/officenotes, 1980.