Interpretation of seasonal fire outlooks

1.Introduction

The Australian Fire Danger Rating System (AFDRS) is a nationally consistent approach to forecasting fire danger for all major vegetation types found in Australia (Hollis et al. 2024a, 2024b). The operational implementation of AFDRS for forecasting fire danger ratings by the Bureau of Meteorology, launched in September 2022, was the most significant change in fire danger rating systems in Australia since the McArthur system was introduced in the middle of last century. The McArthur system was based on two fuel types, and fire danger ratings were based on the Grass Fire Danger Index (GFDI) (McArthur 1960) or Forest Fire Danger Index (FFDI) (McArthur 1967). The fire danger index scales ranged from 1 to 100 where 100 should have approximately represented ‘bad’ summer concurrent conditions of a temperature of 35°C, average wind speed of 45 km h^–1 and relative humidity of 10%, coupled with an average summer drought (Matthews et al. 2019).

Climate change has altered fire weather, driven primarily by changes to temperature and relative humidity (Bureau of Meteorology and CSIRO 2022), which also alter vegetation (Hughes 2003). An increase in extreme fire weather (Fox-Hughes 2008; Abram et al. 2021) and in the length of the fire season (Dowdy 2018) has been observed since the 1950s across large parts of Australia. This has led to larger and more frequent fires, especially in southern Australia (Clarke et al. 2013; Harris and Lucas 2019; Canadell et al. 2021; Bureau of Meteorology and CSIRO 2022). The devastating Victorian Black Saturday fires (2009) triggered a Royal Commission, which recommended changes and improvements to the fire danger rating system (Teague et al. 2010), but ultimately it was recognised that the McArthur system was being used beyond its original design (L. Fogarty, A. Sullivan, S. Heemstra and M. Chladil, unpubl. data, 2010) and a new system was required (Australian Emergency Management Committee 2011). Various ratings methods used internationally were reviewed and important characteristics and requirements of a fire danger rating in Australia were identified (Matthews et al. 2019). The diverse Australian flora and related observed fire behaviour are recognised by the adoption of eight basic fuel models, with varying parameterisations allowing diversification to 22 fuel types.

With the development of a new system, an opportunity arose to create longer-term outlooks, extending the forecast horizon from 7 days (as provided by the McArthur system) to provide information on potential fire weather conditions out to several months ahead. The AFDRS Fire Danger Outlooks (FDOs) are the first such operational products of their kind, building on years of research and collaboration between the New South Wales (NSW) Rural Fire Service, the Australian and New Zealand National Council for Fire and Emergency Services (AFAC, previously the Australasian Fire and Emergency Service Authorities Council) and the Bureau of Meteorology (Marshall et al. 2022), with expert input from fire management agencies across Australia and from CSIRO. The products use the Bureau’s Australian Community Climate Earth simulator ACCESS-S2 seasonal climate prediction model (Wedd et al. 2022). Skill had been demonstrated for longer-term forecasting of the FFDI and was closely linked to the forecast skill of the atmospheric components used in the computations, particularly temperature and rainfall (Dowdy et al. 2016). The FDOs were developed through a co-design process (Gregory et al. 2024) and are currently available to fire agencies and partner agencies who are involved in land management and fire prevention activities.

2.Data and methods

The differences in hindcast temperature identified in Benger et al. (2022) varied spatially and temporally, and were significant enough to motivate further investigation. In this work, we determined how the hindcasts differ over the two periods and discuss the implications of this for the interpretation of the outlooks, by:

As the hindcasts constitute the technical foundations of the products, they guide how the products should be interpreted and therefore are of primary interest for this analysis. In practice, however, users rarely refer to the hindcasts. When users interpret products operationally, their interpretation of the ‘usual’ conditions (the median) is shaped by their experience, that is, their exposure to the observed conditions. Historical medians and 20th and 80th percentiles (top and bottom quintiles) are provided for rainfall and temperature (e.g. http://www.bom.gov.au/climate/outlooks/#/temperature/maximum/climatology/med/monthly/0), but quintiles are not included in the FDO product suite. We therefore examine observations for the two periods for the lower tercile, median and upper tercile to investigate if the hindcast differences are also reflected in the observations. Results confirming the hindcast analysis would further support the case of interpretive issues when users are estimating fire danger based on the different products; this is shown to be the case in the Results section.

Temperature and rainfall hindcasts of the 33rd, 50th and 66th percentiles (the lower tercile, median and upper tercile respectively) were created in NetCDF4 format using the Bureau’s ACCESS-S2 seasonal climate prediction model (Wedd et al. 2022) for the AFDRS years 2003–2017 (AFDRS hindcast period). These hindcasts were compared (by differencing) with the Bureau’s official temperature and rainfall hindcasts for the years 1981–2018 (official hindcast period).

Station data were extracted for sites that are reflective of diverse Australian climates and had continuous historical records to before 1981. Data were retrieved from the Bureau of Meteorology Climate Data Online (see http://www.bom.gov.au/climate/data/index.shtml). Site temperature and rainfall were analysed for: Pearce Royal Australian Air Force (RAAF) base and Manjimup in Western Australia (WA), Darwin Airport in the Northern Territory, Cape Willoughby and Eudunda in South Australia, Taroom and Oakey Aero in Queensland (Qld), Rutherglen in Victoria and Scottsdale in Tasmania (Tas.). Locations are shown as black dots in Fig. 1.

Fig. 1.

Differences in median precipitation hindcast (mm month^–1) for the official and AFDRS time windows. Green indicates the official hindcast period was wetter; brown indicates the official hindcast period was drier.

We compared the distributions of the monthly data for each period to determine if the distributions were different (with statistical significance), and then compared these results with ACCESS-S2-produced hindcast difference information.

The AFDRS hindcast period is a subset of the official hindcast, so the comparison was performed using the two sets of years: Set A is the AFDRS hindcast period (2003–2017) and Set B is the official hindcast period with the AFDRS years removed (1981–2003 and 2018) to ensure independent sets. (Comparison of Sets A and B gives the same result as comparing Set A with Set A + B, which is the full official hindcast period, but with the benefit of the assumption of independence.) The Mann–Whitney U test was used to compare the distribution of daily rainfall (precipitation is not normally distributed – confirmed by the Shapiro test, and our observation sets are independent by design).¹ The Mann–Whitney U test has a null hypothesis that the distributions of both groups are identical. If the P-value is small (<0.05), we can reject the null hypothesis that the difference is due to random sampling and conclude that the sets are distinct.

Both sets of experiments reinforced the findings from Benger et al. (2022), so the remainder of the current article only presents the results that relate to the rainfall analysis, and a single, selected observation case to illustrate the methods.

The sensitivity study by Matthews et al. (2019) showed that for all fuel models used in the AFDRS calculations, increasing rainfall (used directly or indirectly to approximate dead fuel moisture content) decreases the fire danger index (assuming all other inputs remain constant), and vice versa.

3.Results: hindcast rainfall comparison

Fig. 1 shows the differences of the medians (official period minus AFDRS period) where green (positive values) indicates that the official hindcast period is wetter, and brown (negative values) indicates that the official hindcast period is drier. The precipitation hindcast medians vary in space, time and trend – there are periods when the official hindcast is wetter and drier, rather than the change being uniformly in one direction, either wetter or drier, as was the case for maximum temperature (the AFDRS hindcast period is uniformly warmer).

Fig. 1 shows that the hindcasts are quite similar at the start of the northern wet season (November hindcasts are within ~10 mm) but the two hindcasts diverge significantly during the wet season, with some regions flipping from wetter to drier and vice versa. The official hindcast shows a drier signal for south-east Qld during January and February, which the Taroom, Qld, observation analysis supports (described in the following sections).

For southern Australia, the hindcasts are similar over winter; the official period is slightly wetter around southern WA, the south-eastern mainland and western Tas., with a shift to a drier Tas. during January for the official period.

In Fig. 2, we have an example of the comparisons of the lower tercile, median and upper tercile; December is chosen for illustrative purposes. The official hindcast period for December is generally wetter for all percentiles (with more than a 10-mm difference), with the median and upper hindcast terciles showing the most significant differences. Examination of the differences of all three percentiles for all months shows the hindcast difference trends appear generally uniform across the percentiles, as they do for December. The only exception to this is for one region in May: along the Victorian Alps, the lower tercile official hindcast is drier, but the median and upper terciles are wetter; the differences are otherwise consistent across the percentiles for all other regions.

Fig. 2.

December precipitation percentile differences (mm month^–1). Left, lower tercile; middle, median; right, upper tercile. Black points show locations for which observational data were also examined.

When the medians are similar, it is possible to use the chance of above median rainfall product with the FDO and maintain a consistent interpretation. When the hindcast medians differ, a more nuanced interpretation is required. Owing to the uniform difference in the maximum temperature hindcasts, it was possible to provide straightforward interpretive notes for using the FDO with the chance of above median maximum temperature; refer to Benger et al. (2022): as the AFDRS period is warmer, the FDO may indicate a lower chance of exceeding the median than might be expected based on interpretation of the ACCESS-S2 maximum temperature forecast alone. This is not practically feasible for rainfall (or for terciles) as there are too many cases to consider. To illustrate the complexity, we provide a comparative interpretive guide for using the chance of above median rainfall products with the FDO chance of above median Fire Danger Index. In Table 1, we examine if the official hindcast is wetter or drier, and if the rainfall outlooks show a high chance of above or below median with the possible AFDRS precipitation (conditional on the rainfall outlook value). We refer to the user interpretation of Fire Danger as ‘FD’ below and compare how a user might interpret the Fire Danger based on the rainfall forecast or FDO.

The information in Table 1 is expanded on below for clarity.

When the official hindcast is wetter:

When the official hindcast is drier:

When the official hindcast is drier than the AFDRS precipitation hindcast period, in a situation where the forecast shows a high chance of below median rainfall, then the forecast would necessarily be very much below the AFDRS rainfall median. This would result in more pronounced rainfall deficiency for the FDO and thus a stronger FD. Alternatively, the forecast rainfall could be above the official median, but below the AFDRS rainfall median, giving conflicting information.

The temperature difference highlighted by Benger et al. (2022) resulted in one key point to consider, that the FD may be lower using the FDO than when using the maximum temperature forecast when a high chance of above median maximum temperature is forecast. The rainfall difference illustrated here highlights six points to consider (interpretation notes in Table 1), and the relevant point to consider will vary in both space and time. The interpretation notes above illustrate the potential confusion for a user, with outcomes ranging from differing FD risk levels to conflicting information. For terciles, there would be a sixfold increase in the number of cases to be considered for consistent interpretation and messaging – this is clearly not viable as a ‘rule of thumb’ operationally. However, it is possible to interpret the outlooks together with the assistance of the difference plots. To illustrate, we provide an interpretive analysis of the December 2023 outlook for the temperature, FDO and rainfall outlooks below.

Fig. 3 shows an example of the rainfall, FDO and temperature chance of above median products for December 2023 (generated early November 2023). The circled regions show potential areas of interest in the FDO; those circled in yellow (and numbered) are locations for which the FDO may have unexpected values, those circled in green are likely to align with user’s expectations. Fig. 4 shows that the official hindcast for December is generally cooler and, in some regions, wetter than the AFDRS hindcast.

Fig. 3.

December forecast chance of above median for: maximum temperature (left), FDO (centre) and rainfall (right), created early November 2023. Yellow circles indicate locations for which the FDO may have unexpected values, green circles indicate locations for which FDO values are likely to align with user’s expectations. See text for explanations of features referencing numbered circles.

Fig. 4.

Hindcast differences for December: official hindcast cooler (blue on left plot) from Benger et al. (2022) and wetter (green in right plot) than the AFDRS hindcast.

The green circled regions of Fig. 3 both show a moderate chance of exceeding the median AFDRS index for December. Those regions also have a high chance of exceeding the median maximum temperature (which would elevate the AFDRS index), and no signal towards wetter or drier conditions. The difference plots in Fig. 4 show that the hindcasts for those regions are also similar, both for rainfall and temperature, so a user can be confident in their interpretation, and that a likely contributing factor of the elevated chance of exceeding the median AFDRS index is the forecast warmer conditions.

The yellow circled regions of Fig. 3 correspond to areas with differences in the hindcasts, some of which result in potentially conflicting interpretation without understanding the role of the hindcast. In regions 1 and 3, the official hindcast is both cooler and wetter; in region 2, the official hindcast is wetter. In Table 2 we provide notes on the interpretation of the outlook products for the three regions. The symbols ‘↑, ↓’ and ‘–’ indicate a ‘high chance of above or below median’ and ‘no indication of above or below median’ respectively.

In summary, as the interpretive notes in Table 2 show, it is highly likely that a user trying to interpret the outlooks without knowledge of the hindcast differences would find the information quite conflicting. It is also clear, however, that there is no simple rule of thumb that can be easily applied. The authors recommend using the difference plots for interpretive support, particularly in the case of using FDOs with rainfall outlooks.

4.Results: comparison of observations

Overall, the observations confirmed the hindcast analysis of differences between the rainfall and temperature for the two periods (years in Sets A and B), agreeing in direction and approximate magnitude. Observation data enabled us to investigate the statistical significance of the differences.

For some months, the difference in the distributions of rainfall or temperature was not statistically significant, but rarely (for any of the selected locations) were the differences in both the distributions of rainfall and temperature statistically insignificant for a given month. That is, generally, at least one of the rainfall or temperature distributions was significantly different for most months for each location. For the example location below, only 2 months had neither temperature nor precipitation being significantly different (see Table 3). In some months and locations when the rainfall distribution was identified as different, the actual differences in the terciles and median were so small that the difference was arguably indistinguishable for operational purposes. During those times and at those locations, it would be possible to use the rainfall outlooks and FDO together with confidence – but this is not uniform through time or space, so should not be assumed when using the products for decision making.

The example location is Pearce RAAF base in WA, chosen for illustrative purposes; the analysis for the other locations similarly supported the hindcast results. In Table 3, the test statistics are provided for the comparison of the two sets of observations for rainfall and temperature; the months highlighted in bold have P-value <0.05 so the differences in the distributions are considered statistically significant. The months where a difference in the distribution of rainfall totals can be assumed to exist between the hindcast periods for Pearce RAAF in WA are Feb, May, Jun, Jul, Aug, Oct and Nov. This is consistent with the difference in medians of the ACCESS-S2 data shown in Fig. 1, but not directly obvious for the non-winter months as the differences are not large compared with the national scale and spatial resolution of the hindcasts. The months with significant differences varied across the locations. All the shifts in distribution for the Pearce location are to a drier period for the AFDRS hindcast years, consistent with the general pattern in the hindcast differences. The region has observed a decreasing trend in rainfall for the period from May to August (Bureau of Meteorology and CSIRO 2022), so these results are unsurprising for this location. Other locations investigated showed both wetter and drier periods, also consistent with the hindcast comparisons.

For May–July, the differences were more significant. The differences in lower tercile, median, upper tercile (mm month^–1) for these months and the percentage changes are:

The differences for June are visualised in Fig. 5.

Fig. 5.

Normalised histogram of June observed rainfall at Pearce RAAF in WA for the AFDRS hindcast period (2003–2017) and the ‘other hindcast years’ (1981–2003 and 2018).

In Fig. 5, the ‘other years’ lower tercile almost coincides with the median for the AFDRS hindcast period, and the ‘other years’ median coincides with the upper tercile – making it difficult to see on the plot – for the AFDRS hindcast period. This is a significant shift in the distribution.

In summary, the observational comparisons reinforce the variability highlighted in the hindcast analysis, demonstrating that temperature and rainfall differences in the hindcasts can lead to significantly different interpretation of fire danger based on FDO, temperature and rainfall outlooks. The differences are not uniform in space or time, and thus no simple interpretive rule of thumb can be provided.

5.Conclusion and recommendations

Climate outlooks should be used with other sources of intelligence to understand which components might be contributing to forecast outcomes and the associated risk. It is wise to use temperature and rainfall outlooks with the FDOs as temperature and rainfall are contributing factors to fire danger conditions (Jain et al. 2022), but the FDOs have differing data foundations that need to be understood to interpret them correctly. In Benger et al. (2022), the temperature hindcasts for the AFDRS hindcast period (2003–2017) were shown to be generally warmer than the official hindcast period (1981–2018). The results of the experiments presented here show that the differences in precipitation hindcasts for the official and AFDRS hindcast periods vary in space, time and trend (potentially both wetter or drier, varying by region and month).

Users often rely on their own understanding of the climate, shaped by observations, to interpret outlook products (rather than referring to a specific hindcast). The comparison of observations over the hindcast periods reinforced the results of the hindcasts comparisons. The differences in rainfall distributions for the examined locations were determined to be of statistical significance for most months for temperature and approximately half of the months for rainfall. Significant distribution shifts were identified for the median and tercile thresholds used in the climate outlooks. These differences can significantly influence the interpretation of fire danger when using the temperature and rainfall outlooks and the FDOs, particularly as users tend to rely on their experience and own understanding of the median, shaped by the conditions experienced (observed), rather than refer to historical values. Without an understanding of these differences, a user could potentially have conflicting interpretations of the outlooks. This could lead to confusion for users and hinder, rather than aid, in operational preparedness.

As there is no regular pattern in time, space or trend of change between the hindcasts that can help define a rule of thumb to account for these differences, it is not straightforward to interpret the FDOs with the rainfall outlooks in a consistently compatible manner.

One solution would be the development of temperature and rainfall outlooks with the same hindcast period. Although a shorter hindcast is not favourable from a climate science perspective, the operational benefits from having compatible products available should be investigated. It may also be beneficial to consider the development of quintile FDO products and a suite of unusually wet, dry, warm or cool products (see http://www.bom.gov.au/climate/outlooks/#/rainfall/extremes/p20/weekly/0) (rainfall and maximum temperature quintile products, with the same hindcast period).