An evaluation of wildland fire simulators used operationally in Australia

P. Fox-Hughes; C. Bridge; N. Faggian; C. Jolly; S. Matthews; E. Ebert; H. Jacobs; B. Brown; J. Bally

doi:10.1071/WF23028

RESEARCH ARTICLE (Open Access)

Previous Next Contents Vol 33(4)

An evaluation of wildland fire simulators used operationally in Australia

P. Fox-Hughes

^A ^* , C. Bridge ^B , N. Faggian ^B , C. Jolly ^A , S. Matthews ^C , E. Ebert ^B , H. Jacobs ^B , B. Brown ^D and J. Bally ^E

+ Author Affiliations

- Author Affiliations

^A Bureau of Meteorology, 111 Macquarie Street, Hobart, Tas. 7001, Australia.

^B Bureau of Meteorology, 700 Collins Street, Docklands, Vic. 3001, Australia.

^C New South Wales Rural Fire Service, 15 Carter Street, Lidcombe, NSW 2141, Australia.

^D National Center for Atmospheric Research, 3090 Center Green Drive, Boulder, CO 80301, USA.

^E Australasian Fire and Emergency Service Authorities Council (AFAC), 340 Albert Street, East Melbourne, Vic. 3002, Australia.

^* Correspondence to: paul.fox-hughes@bom.gov.au

International Journal of Wildland Fire 33, WF23028 https://doi.org/10.1071/WF23028

Submitted: 1 March 2023 Accepted: 22 March 2024 Published: 12 April 2024

© 2024 The Author(s) (or their employer(s)). Published by CSIRO Publishing on behalf of IAWF. This is an open access article distributed under the Creative Commons Attribution-NonCommercial 4.0 International License (CC BY-NC)

Abstract

Background

Fire simulators are increasingly used to predict fire spread. Australian fire agencies have been concerned at not having an objective basis to choose simulators for this purpose.

Aims

We evaluated wildland fire simulators currently used in Australia: Australis, Phoenix, Prometheus and Spark. The evaluation results are outlined here, together with the evaluation framework.

Methods

Spatial metrics and visual aids were designed in consultation with simulator end-users to assess simulator performance. Simulations were compared against observations of fire progression data from 10 Australian historical fire case studies. For each case, baseline simulations were produced using as inputs fire ignition and fuel data together with gridded weather forecasts available at the time of the fire. Perturbed simulations supplemented baseline simulations to explore simulator sensitivity to input uncertainty.

Key results

Each simulator showed strengths and weaknesses. Some simulators displayed greater sensitivity to different parameters under certain conditions.

Conclusions

No simulator was clearly superior to others. The evaluation framework developed can facilitate future assessment of Australian fire simulators.

Implications

Collection of fire behaviour observations for routine simulator evaluation using this framework would benefit future simulator development.

Keywords: Australis, evaluation framework, fire behaviour modelling, fire simulation modelling, operational fire modelling, Phoenix, Prometheus, Spark.

References

Ager AA, Vaillant NM, Finney MA (2010) A comparison of landscape fuel treatment strategies to mitigate wildland fire risk in the urban interface and preserve old forest structure. Forest Ecology and Management 259, 1556-1570.
| Crossref | Google Scholar |

Allaire F, Filippi J-B, Mallet V (2020) Generation and evaluation of an ensemble of wildland fire simulations. International Journal of Wildland Fire 29, 160-173.
| Crossref | Google Scholar |

Allaire F, Filippi J-B, Mallet V, Vaysse F (2022) Simulation-based high-resolution fire danger mapping using deep learning. International Journal of Wildland Fire 31(4), 379-394.
| Crossref | Google Scholar |

Anderson WR, Cruz MG, Fernandes PM, McCaw L, Vega JA, Bradstock RA, Fogarty L, Gould J, McCarthy G, Marsden-Smedley JB, Matthews S, Mattingley G, Pearce HG, van Wilgen BW (2015) A generic, empirical-based model for predicting rate of fire spread in shrublands. International Journal of Wildland Fire 24(4), 443-460.
| Crossref | Google Scholar |

Arca B, Duce P, Laconi M, Pellizzaro G, Salis M, Spano D (2007) Evaluation of FARSITE simulator in Mediterranean maquis. International Journal of Wildland Fire 16, 563-572.
| Crossref | Google Scholar |

Bachmann A, Allgöwer B (2002) Uncertainty propagation in wildland fire behaviour modelling. International Journal of Geographical Information Science 16(2), 115-127.
| Crossref | Google Scholar |

Benali A, Ervilha AR, Sá ACL, Fernandes PM, Pinto RMS, Trigo RM, Pereira JMC (2016) Deciphering the impact of uncertainty on the accuracy of large wildfire spread simulations. Science of the Total Environment 569–570, 73-85.
| Crossref | Google Scholar | PubMed |

Bridge C (2015) Why are temperature forecasts from the Australian Digital Forecast Database poorer on summer afternoons? Australian Meteorological and Oceanographic Journal 65(2), 176-194.
| Crossref | Google Scholar |

Brotak EA (1980) A comparison of the meteorological conditions associated with a major wildland fire in the United States and a major bushfire in Australia. Journal of Applied Meteorology and Climatology 19(4), 474-476.
| Crossref | Google Scholar |

Bureau of Meteorology (2015) Australian Digital Forecast Database (ADFD) User Guide. Available at http://www.bom.gov.au/catalogue/adfdUserGuide.pdf

Bureau of Meteorology (2017) Bushfire Predictive Services Final Report: An evaluation of fire spread simulators used in Australia. Report to NSW Rural Fire Service and Australasian Fire and Emergency Service Authorities Council. Available at http://www.bom.gov.au/research/publications/otherreports/FPS_Final_Report_v1.81_Evaluation_Of_Simulators_Release.pdf

Burrows ND, Ward B, Robinson A (2009) Fuel dynamics and fire spread in spinifex grasslands of the Western Desert. In ‘Proceedings of the Royal Society of Queensland: Bushfire 2006 Conference’, 6–9 June 2006, Brisbane, Qld. (Ed. C Tran.) pp. 69–76. (Royal Society of Queensland Inc.: St Lucia, Qld, Australia)

Carmona-Moreno C, Belward A, Malingreau JP, Hartley A, Garcia‐Alegre M, Antonovskiy M, Buchshtaber V, Pivovarov V (2005) Characterizing interannual variations in global fire calendar using data from Earth observing satellites. Global Change Biology 11(9), 1537-1555.
| Crossref | Google Scholar |

Cheney NP (1976) Bushfire disasters in Australia, 1945–1975. Australian Forestry 39(4), 245-268.
| Crossref | Google Scholar |

Cheney NP, Gould JS, Catchpole WR (1998) Prediction of fire spread in grasslands. International Journal of Wildland Fire 8(1), 1-13.
| Crossref | Google Scholar |

Cheney NP, Gould JS, McCaw WL, Anderson WR (2012) Predicting fire behaviour in dry eucalypt forest in southern Australia. Forest Ecology and Management 280, 120-131.
| Crossref | Google Scholar |

Cruz MG, Alexander ME (2013) Uncertainty associated with model predictions of surface and crown fire rates of spread. Environmental Modelling & Software 47, 16-28.
| Crossref | Google Scholar |

Cruz MG, Sullivan AL, Gould JS, Sims NC, Bannister AJ, Hollis JJ, Hurley RJ (2012) Anatomy of a catastrophic wildfire: the Black Saturday Kilmore East fire in Victoria, Australia. Forest Ecology and Management 284, 269-285.
| Crossref | Google Scholar |

Cruz MG, McCaw WL, Anderson WR, Gould JS (2013) Fire behaviour modelling in semi-arid mallee-heath shrublands of southern Australia. Environmental Modelling & Software 40, 21-34.
| Crossref | Google Scholar |

Cruz MG, Sullivan AL, Leonard R, Malkin S, Matthews S, Gould JS, Alexander ME (2014) ‘Fire behaviour knowledge in Australia: a synthesis of disciplinary and stakeholder knowledge on fire spread prediction capability and application.’ (Bushfire Cooperative Research Centre: Melbourne, Vic.)

Cruz MG, Gould JS, Alexander ME, Sullivan AL, McCaw WL, Matthews S (2015) Empirical-based models for predicting head-fire rate of spread in Australian fuel types. Australian Forestry 78(3), 118-158.
| Crossref | Google Scholar |

Cui W, Perera AH (2010) Quantifying spatio-temporal errors in forest fire spread modelling explicitly. Journal of Environmental Informatics 16(1), 19-26.
| Crossref | Google Scholar |

DeCastro A, Siems-Anderson A, Smith E, Knievel JC, Kosović B, Brown BG, Balch JK (2022) Weather research and forecasting – fire simulated burned area and propagation direction sensitivity to initiation point location and time. Fire 5, 58.
| Crossref | Google Scholar |

Doerr SH, Santín C (2016) Global trends in wildfire and its impacts: perceptions versus realities in a changing world. Philosophical Transactions of the Royal Society. Series B, Biological Sciences 371(1696), 20150345.
| Crossref | Google Scholar | PubMed |

Duff TJ, Chong DM, Taylor P, Tolhurst KG (2012) Procrustes based metrics for spatial validation and calibration of two-dimensional perimeter spread models: a case study considering fire. Agricultural and Forest Meteorology 160, 110-117.
| Crossref | Google Scholar |

Duff TJ, Chong DM, Tolhurst KG (2013) Quantifying spatio-temporal differences between fire shapes: estimating fire travel paths for the improvement of dynamic spread models. Environmental Modelling & Software 46, 33-43.
| Crossref | Google Scholar |

Duff TJ, Chong DM, Tolhurst KG (2016) Indices for the evaluation of wildfire spread simulations using contemporaneous predictions and observations of burnt area. Environmental Modelling & Software 83, 276-285.
| Crossref | Google Scholar |

Duff TJ, Cawson JG, Cirulis B, Nyman P, Sheridan GJ, Tolhurst KG (2018) Conditional performance evaluation: using wildfire observations for systematic fire simulator development. Forests 9(4), 189-204.
| Crossref | Google Scholar |

EM-DAT (2018) The Emergency Events Database – Université Catholique de Louvain (UCL) - CRED, D. Guha-Sapir. (Brussels, Belgium) Available at www.emdat.be

Filippi J-B, Mallet V, Nader B (2014) Representation and evaluation of wildfire propagation simulations. International Journal of Wildland Fire 23, 46-57.
| Crossref | Google Scholar |

Filkov A, Duff T, Penman T (2018) Improving fire behaviour data obtained from wildfires. Forests 9, 81.
| Crossref | Google Scholar |

Finney MA (2000) Efforts at comparing simulated and observed fire growth patterns. Report INT-95066-RJVA. (Systems for Environmental Management: Missoula, MT, USA)

Finney MA (2004) FARSITE: Fire Area Simulator – Model Development and Evaluation. Research Paper. RMRS-RP-4 Revised March 1998, revised February. (USDA Forest Service Rocky Mountain Research Station)

Finney MA, Grenfell IC, McHugh CW, Seli RC, Trethewey D, Stratton RD, Brittain S (2011) A method for ensemble wildland fire simulation. Environmental Modeling & Assessment 16, 153-167.
| Crossref | Google Scholar |

Forestry Canada (1992) Development and structure of the Canadian forest fire behavior system. Information Report ST-X-3. (Forestry Canada Fire Danger Group: Ottawa, Canada)

Fujioka FM (2002) A new method for the analysis of fire spread modeling errors. International Journal of Wildland Fire 11(4), 193-203.
| Crossref | Google Scholar |

Giannaros TM, Lagouvardos K, Kotroni V (2020) Performance Evaluation of an operational rapid response fire spread forecasting system in the southeast Mediterranean (Greece). Atmosphere 11, 1264.
| Crossref | Google Scholar |

Giglio L, Randerson JT, van der Werf GR (2013) Analysis of daily, monthly, and annual burned area using the fourth‐generation global fire emissions database (GFED4). Journal of Geophysical Research: Biogeosciences 118(1), 317-328.
| Crossref | Google Scholar |

Gleckler PJ, Taylor KE, Doutriaux C (2008) Performance metrics for climate models. Journal of Geophysical Research: Atmospheres 113(D6), D06104.
| Crossref | Google Scholar |

Haas JR, Calkin DE, Thompson MP (2013) A national approach for integrating wildfire simulation modeling into wildland urban interface risk assessments within the United States. Landscape and Urban Planning 119, 44-53.
| Crossref | Google Scholar |

Hilton JE, Miller C, Sullivan AL, Rucinski C (2015) Effects of spatial and temporal variation in environmental conditions on simulation of wildfire spread. Environmental Modelling & Software 67, 118-127.
| Crossref | Google Scholar |

Johnston P, Kelso J, Milne GJ (2008) Efficient simulation of wildfire spread on an irregular grid. International Journal of Wildland Fire 17(5), 614-627.
| Crossref | Google Scholar |

Kelso JK, Mellor D, Murphy ME, Milne GJ (2015) Techniques for evaluating wildfire simulators via the simulation of historical fires using the Australis simulator. International Journal of Wildland Fire 24, 784-797.
| Crossref | Google Scholar |

Knight I, Coleman J (1993) A fire perimeter expansion algorithm-based on Huygens wavelet propagation. International Journal of Wildland Fire 3(2), 73-84.
| Crossref | Google Scholar |

Louis S, Matthews S (2015) Fire spread prediction using a lagged weather forecast ensemble. In ‘Proceedings of the 21st International Congress on Modelling and Simulation’, 29 November–4 December 2015, Gold Coast, Australia. (Eds T Weber, MJ McPhee, RS Anderssen) pp. 236–242. (Modelling and Simulation Society of Australia and New Zealand)

Marsden-Smedley JB, Catchpole WR (1995) Fire behaviour modelling in Tasmanian buttongrass Moorlands. II. Fire behaviour. International Journal of Wildland Fire 5, 215-228.
| Crossref | Google Scholar |

McArthur AG (1967) Fire behaviour in Eucalypt forests. Forestry and Timber Bureau, Leaflet 107. (Department of Natural Development: Canberra, ACT, Australia)

Miller C, Hilton J, Sullivan A, Prakash M (2015) SPARK – A bushfire spread prediction tool. In ‘Proceedings of the International Symposium on Environmental Software Systems 2015: Environmental Software Systems. Infrastructures, Services and Applications’. (Eds R Denzer, RM Argent, G Schimak, J Hřebíček) pp. 262–271. (Springer: Cham, Switzerland) doi:10.1007/978-3-319-15994-2_26

Miller SI, Carter W, Stephens RG (1984) ‘Report of the Bushfire Review Committee: On Bush Fire Disaster Preparedness and Response in Victoria, Australia, Following the Ash Wednesday Fires 16 February 1983.’ (Government Printer: Melbourne, Victoria)

Mills GA (2005) A re-examination of the synoptic and mesoscale meteorology of Ash Wednesday 1983. Australian Meteorological Magazine 54(1), 35-55.
| Google Scholar |

Opperman T, Gould J, Finney M, Tymstra C (2006) Applying fire spread simulators in New Zealand and Australia: results from an international seminar. In ‘Fuels Management – How to Measure Success: Conference Proceedings’, 28–30 March 2006, Portland, OR. RMRS-P-41. (Eds PL Andrews, BW Butler) pp. 201–212. (USDA Forest Service, Rocky Mountain Research Station: Fort Collins, CO)

Owens D, O’Kane M (2020) ‘Final Report of the NSW Bushfire Inquiry’. 466 p. (Sydney, Australia) Available at https://www.dpc.nsw.gov.au/assets/dpc-nsw-gov-au/publications/NSW-Bushfire-Inquiry-1630/Final-Report-of-the-NSW-Bushfire-Inquiry.pdf [accessed 1 February 2023]

Ozaki M, Aryal J, Fox-Hughes P (2019) Dynamic wildfire navigation system. ISPRS International Journal of Geo-Information 8, 194.
| Crossref | Google Scholar |

Penman TD, Ababei DA, Cawson JG, Cirulis BA, Duff TJ, Swedosh W, Hilton JE (2020) Effect of weather forecast errors on fire growth model projections. International Journal of Wildland Fire 29, 983-994.
| Crossref | Google Scholar |

Pinto RM, Benali A, Sá AC, Fernandes PM, Soares PM, Cardoso RM, Trigo RM, Pereira JM (2016) Probabilistic fire spread forecast as a management tool in an operational setting. SpringerPlus 5(1), 1205-1228.
| Crossref | Google Scholar | PubMed |

Plucinski MP, Sullivan AL, Rucinski CJ, Prakash M (2017) Improving the reliability and utility of operational bushfire behaviour predictions in Australian vegetation. Environmental Modelling & Software 91, 1-12.
| Crossref | Google Scholar |

Roebber PJ (2009) Visualizing multiple measures of forecast quality. Weather and Forecasting 24, 601-608.
| Crossref | Google Scholar |

Rothermel RC (1983) How to predict the spread and intensity of forest and range fuels. General Technical Report INT-143. (USDA Forest Service, Intermountain Forest and Range Experiment Station: Ogden, UT)

Rothermel RC, Rinehart GC (1983) Field procedures for verification and adjustment of fire behavior predictions. General Technical Report INT-142. (USDA Forest Service, lntermountain Forest and Range Experiment Station: Ogden, UT)

Russell-Smith J, Yates CP, Whitehead PJ, Smith R, Craig R, Allan GE, Thackway R, Frakes I, Cridland S, Meyer MC, et al. (2007) Bushfires 'Down Under': patterns and implications of contemporary Australian landscape burning. International Journal of Wildland Fire 16(4), 361-377.
| Crossref | Google Scholar |

Scott AC, Bowman DM, Bond WJ, Pyne SJ, Alexander ME (2013) ‘Fire on Earth: an introduction.’ (John Wiley & Sons)

State Fire Management Council (2014) Bushfire in Tasmania: a new approach to reducing our statewide relative risk. (State Fire Management Council Unit, Tasmania Fire Service: Hobart, Tas.)

Stretton LEB (1939) ‘Report of the Royal Commission to inquire into the causes of and measures taken to prevent the bush fires of January, 1939 and to protect life and property and the measures to be taken to prevent bush fires in Victoria and to protect life and property in the event of future bush fires.’ (Government Printer: Melbourne, Vic.). Available at http://royalcommission.vic.gov.au/getdoc/fbe8a952-b2fc-4346-9c60-805faab437d9/TEN.028.001.0001.pdf

Teague B, McLeod R, Pascoe S (2010) ‘2009 Victorian Bushfires Royal Commission Final Report.’ (Parliament of Victoria) Available at http://royalcommission.vic.gov.au/finaldocuments/summary/PF/VBRC_Summary_PF.pdf

Tolhurst K, Shields B, Chong D (2008) Phoenix: development and application of a bushfire risk management tool. Australian Journal of Emergency Management 23(4), 47.
| Google Scholar |

Tomašević IČ, Cheung KKW, Vučetić V, Fox-Hughes P (2022) Comparison of wildfire meteorology and climate at the Adriatic Coast and southeast Australia. Atmosphere 13, 755.
| Crossref | Google Scholar |

Tymstra C, Bryce RW, Wotton BM, Taylor SW, Armitage OB (2010) Development and structure of Prometheus: the Canadian wildland fire growth simulation model. Information Report NOR-X-417. (Natural Resources Canada, Canadian Forest Service, Northern Forestry Centre: Edmonton, AB, Canada)