Advances in Geosciences
www.adv-geosci.net
1680-7340
1680-7359
16
Observation, Prediction and Verification of Precipitation (EGU Session 2007)
2008
10.5194/adgeo-16-137-2008
http://www.adv-geosci.net/16/137/2008/
http://www.adv-geosci.net/16/137/2008/adgeo-16-137-2008.html
http://www.adv-geosci.net/16/137/2008/adgeo-16-137-2008.pdf
137
142
2008-04-09
Bias Adjusted Precipitation Threat Scores
F. Mesinger
NCEP/Environmental Modeling Center, Camp Springs, Maryland, and Earth System Science Interdisciplinary Center (ESSIC) University of Maryland, College Park, Maryland, USA
Among the wide variety of performance measures available
for the assessment of skill of deterministic precipitation forecasts, the
equitable threat score (ETS) might well be the one used most frequently. It
is typically used in conjunction with the bias score. However, apart from
its mathematical definition the meaning of the ETS is not clear. It has been
pointed out (Mason, 1989; Hamill, 1999) that forecasts with a larger bias
tend to have a higher ETS. Even so, the present author has not seen this
having been accounted for in any of numerous papers that in recent years
have used the ETS along with bias "as a measure of forecast accuracy".
<br><br>
A method to adjust the threat score (TS) or the ETS so as to arrive at their
values that correspond to unit bias in order to show the model's or
forecaster's accuracy in \textit{placing} precipitation has been proposed earlier by the
present author (Mesinger and Brill, the so-called <i>dH/dF</i> method). A serious
deficiency however has since been noted with the <i>dH/dF</i> method in that the
hypothetical function that it arrives at to interpolate or extrapolate the
observed value of hits to unit bias can have values of hits greater than
forecast when the forecast area tends to zero. Another method is proposed
here based on the assumption that the increase in hits per unit increase in
false alarms is proportional to the yet unhit area. This new method removes
the deficiency of the <i>dH/dF</i> method. Examples of its performance for 12 months of
forecasts by three NCEP operational models are given.
Baldwin, M. E. and Kain, J. S.: Sensitivity of several performance measures to displacement error, bias, and event frequency, Wea. Forecasting, 21, 636–648, 2006.
Davis, C., Brown, B., and Bullock, R.: Object-based verification of precipitation forecasts. Part I: Methodology and application to mesoscale rain areas, Mon. Weather Rev., 134, 1772–1784, 2006.
Ebert, E. E. and McBride, J. L.: Verification of precipitation in weather systems: Determination of systematic errors, J. Hydrol., 239, 179–202, 2000.
GCWM Branch, EMC: The GFS Atmospheric Model. NCEP Office Note 442, 14 pp., available at: http://wwwt.emc.ncep.noaa.gov/officenotes, 2003.
Gilbert, G. F.: Finley's tornado predictions, Amer. Meteorol. J., 1, 166–172, 1884.
Hamill, T. M.: Hypothesis tests for evaluating numerical precipitation forecasts, Wea. Forecasting, 14, 155–167, 1999.
Janji\'c, Z. I.: A nonhydrostatic model based on a new approach, Meteorol. Atmos. Phys., 82, 271–285, 2003.
Mason, I.: Dependence of the critical success index on sample climate and threshold probability, Aust. Meteorol. Mag., 37, 75–81, 1989.
Mesinger, F. and Brill, K.: Bias normalized precipitation scores. Preprints, 17th Conf. on Probability and Statistics, Seattle, WA, Amer. Meteorol. Soc., CD-ROM, J12.6, 2004.
Pielke, R. A., Sr.: Mesoscale Meteorological Modeling, 2~ed., Academic Press, 676 pp., 2002.
Schaefer, J. T.: The critical success index as an indicator of warning skill, Wea. Forecasting, 5, 570–575, 1990.
Shuman, F. G.: A modified threat score and a measure of placement error. NOAA/NWS Office Note 210, 13 pp., available at: http://wwwt.emc.ncep.noaa.gov/officenotes, 1980.