Circular Flip-Flop Index: quantifying revision stability of forecasts of direction
Deryn Griffiths
A * , Nicholas Loveday A , Benjamin Price A , Michael Foley A and Alistair McKelvie A
A Bureau of Meteorology, GPO Box 1289, Melbourne, Vic. 3001, Australia.
Journal of Southern Hemisphere Earth Systems Science 71(3) 266-271 https://doi.org/10.1071/ES21010
Submitted: 13 May 2021 Accepted: 22 September 2021 Published: 9 December 2021
© 2021 The Author(s) (or their employer(s)). Published by CSIRO Publishing on behalf of BoM. This is an open access article distributed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND)
Abstract
The Flip-Flop Index, designed to quantify the extent to which a forecast changes from one issue time to the next, is extended to a Circular Flip-Flop Index for use with forecasts of wind direction, swell direction or similar. The index was devised so we could understand the degree of stability in wind direction forecasts. The Circular Flip Flop Index is independent of observations, has a relatively simple definition and does not penalise a sequence of forecasts that show a trend as long as the forecasts stay within a 180° sector. The Circular Flip-Flop Index is interpreted in terms of the impact of changing forecasts on decisions made by users of the forecast. The Circular Flip-Flop Index has been used to compare the stability of sequences of automated forecast guidance to the official Australian Bureau of Meteorology forecasts, which are prepared manually. It is the first objective assessment of the stability of forecasts of direction. The results show that the forecasts of wind direction from the automated forecast guidance, itself a consensus of many numerical weather models, are more stable than the official, manual forecasts. The Circular Flip-Flop Index does not measure skill but can play a complementary role in characterising and evaluating a forecasting system.
Keywords: flip-flop, forecast assessment, forecast convergence, forecast oscillations, forecast stability, forecast volatility, wind verification.
References
Bureau of Meteorology (2014) Operations bulletin number 103: upgrades to the operational gridded OCF system. Available at http://www.bom.gov.au/australia/charts/bulletins/apob103.pdf [Accessed 4 February 2021]Ehret U (2010) Convergence index: a new performance measure for the temporal stability of operational rainfall forecasts. Meteorologische Zeitschrif 19, 441–451.
| Convergence index: a new performance measure for the temporal stability of operational rainfall forecasts.Crossref | GoogleScholarGoogle Scholar |
Fowler TL, Brown BG, Halley Gotway J, Kucera P (2015) Spare change: evaluating revised forecasts. Mausam 66, 635–644.
Griffiths D, Foley M, Ioannou I, Leeuwenburg T (2019) Flip-Flop index: quantifying revision stability for fixed-event forecasts. Meteorological Applications 26, 30–35.
| Flip-Flop index: quantifying revision stability for fixed-event forecasts.Crossref | GoogleScholarGoogle Scholar |
Huber PJ (1964) Robust estimation of a location parameter. The Annals of Mathematical Statistics 35, 73–101.
Ruth DP, Glahn B, Dagostaro V, Gilbert K (2009) The performance of MOS in the digital age. Weather and Forecasting 24, 504–519.
| The performance of MOS in the digital age.Crossref | GoogleScholarGoogle Scholar |
Zsoter E, Buizza R, Richardson D (2009) “Jumpiness” of the ECMWF and met office EPS control and Ensemble-Mean forecasts. Monthly Weather Review 137, 3823–3836.
| “Jumpiness” of the ECMWF and met office EPS control and Ensemble-Mean forecasts.Crossref | GoogleScholarGoogle Scholar |
