The effect of ignition protocol on the spread rate of grass fires: a comment on the conclusions of Sutherland et al. (2020)

Additional keywords: empirical modelling, experimental methods, fire behaviour experiments, headfire, ignition pattern.

The open and rigorous criticism of concepts or research approaches is a key ingredient for the advancement of science. In the recent past, simulation modelling has been used to identify perceived issues with experimental designs used in field-scale fire experiments and to suggest methodological improvements (e.g. Linn et al. 2012; Pimont et al. 2017). Motivated by the stated differences in experimental fire ignition methods (line length, ignition direction, ignition rate) and plot layout (width and length) used in Cheney et al. (1993) and Cruz et al. (2015), Sutherland et al. (2020) used simulations from the Wildland–urban interface Fire Dynamics Simulator (WFDS) (Mell et al. 2007) to investigate the impact of the different ignition protocols on fire spread rates across a virtual experimental plot. Despite the lack of any validation measurements to substantiate the simulation findings and the fact that most of the simulations conducted did not replicate the field methods used by Cruz et al. (2015), Sutherland et al. (2020) conclude that the ‘inward ignition’ method (i.e. igniting from the outer edge towards the centre of the ignition line) used by Cruz et al. (2015) is inadequate to achieve a fire spreading at its steady-state rate in 33-m-long plots used by the latter authors and could have ‘adversely affected’ the experimental results, leading to incorrect conclusions about the relationships between fuel, wind speed and rate of fire spread. These authors further state that ‘the inward ignition protocol should not be used’.

It is important for several reasons to comment on the results and conclusions of Sutherland et al. (2020). The results from Cruz et al. (2015, 2018) describing the effects of grass curing and grass fuel load on fire behaviour are currently used operationally in Australia to rate fire danger and to predict the spread rate of free-burning fires in grasslands. Fallacious questions about the validity of these models can raise unfounded distrust in model outputs, leading to potentially hazardous situations. Importantly, Sutherland et al. (2020) do not provide the necessary evidence that validates their simulation results. Are their results realistic and valid, or a consequence of the modelling approach? And do their simulation results warrant a rethink on the value of the experimental data collected, as they suggest? The aim of the present paper is to clarify the true effect of ignition protocol on grass fire rate of spread through an evidence-based approach. By exploring existing data, we aim to eliminate any doubts as to the validity of the data and modelling results of Cruz et al. (2015, 2018).

Experimental fires conducted to study the flame propagation processes and dynamics observed in wildfires need to replicate the behaviours of the attributes being studied. At the same time, there is a need to avoid the effect of constraining factors, such as scale and boundary effects, that could arise from the experimental design used.

One important experimental design consideration when conducting a field burning research program relates to the size of the burn plots where measurements of fire propagation will be made. For experimental fires involving the measurement of rate of fire spread, it is of interest that the burn plot be as small as possible while ensuring the fire still spreads at a pseudo- or ‘quasi’-steady state.^A In this context, there are several advantages of smaller versus larger burn plots. Smaller burn plots can ensure more homogeneous fuels, which are relevant when one aims to control the effect of fuel structure on fire propagation, either by keeping variation to a minimum or more easily allowing manipulation of the fuel. Smaller plots allow for more detail and precision in measuring fire behaviour (Fernandes et al. 2000; Mell and Linn 2017) and wind speed (Sullivan and Knight 2001). Smaller plot sizes also allow for more replicates within a given experimental area (Cruz et al. 2020). Another point of paramount importance is the obvious safety considerations (Alexander and Quintilio 1990). If one aims to conduct experimental fires under heightened fire danger conditions, a smaller burn plot will increase suppression effectiveness and reduce the risk of fires escaping the experimental study area. A disadvantage of relatively small experimental fires is that the data become ‘noisier’ owing to the run not incorporating at least one wind gust–lull cycle (Linn et al. 2012; Cruz et al. 2013). A small experimental fire will sample rate of spread over a section of this cycle. Factors such as the distance between the flame front and the wind measuring location, the location of the fire within the gust–lull cycle, the frequency and amplitude of this cycle, and the state of the fire relative to the speed of the wind (i.e. accelerating or decelerating), will add uncertainty to the rate of spread–wind speed relationship. This will result in wider confidence intervals and larger absolute residuals, although if enough replicates are conducted, not a bias, as shown by Cruz et al. (2018).

A constraint of small experimental fires is the need to produce a fire with a spreading flame front that represents its wildfire counterpart. It is known that fireline width affects the spread rate of experimental fires, with smaller fires potentially spreading below the pseudo-steady-state rate of propagation (Cheney et al. 1998; Wotton et al. 1999; Anderson et al. 2015). An experimental design should ensure plot width does not constrain the attainment of the pseudo-steady-state condition. A fire will also need to be allowed to spread for a given distance after ignition to attain the pseudo-steady-state condition and then spread in this state for enough time to allow multiple measurements of fire behaviour to be made. Reducing the time taken for the fire to reach its pseudo-steady-state rate of spread is particularly critical in smaller burn plots. An obvious aim is to have a continuous instantaneous ignition along the upwind edge of the burn plot (e.g. Viegas et al. 2002) to minimise a fire’s development time. In the absence of a reliable instantaneous ignition method, it is common to see the ignition of an experimental fire being carried out by individuals moving quickly with free-flowing drip torches, either on foot (Cheney et al. 1998; Butler et al. 2013; Cruz et al. 2015, Ottmar et al. 2016) or in motorised vehicles (Butler et al. 2016), or using a vehicle-mounted flame thrower (Bradshaw and Tour 1993; Stocks et al. 2004).

A robust field-based experimental design will thus need to be informed by an understanding of the expected fire dynamics in the fuels under study for the targeted weather conditions driving fire propagation and in finding a compromise between resources available, number of replicates needed, fire behaviour desired, fire measurement methods employed and the final use of the data collected.

The methods used in carrying out grass fire behaviour experiments in Australia have changed over time as a result of evolving technologies, a refinement of the understanding of fire dynamics in grass fuels, and the logistical constraints related to conducting fires under very high to extreme fire danger conditions (in the sense of McArthur 1966). The Annaburroo experiments of Cheney et al. (1993) utilised large plots (≥100 × 100 m) and an ignition protocol using two igniters lighting outward from the centre of the upwind edge of the plot toward the plot corners (‘outward’ ignition method). This method resulted in an initial pointed headfire and, depending on the prevailing wind speed and associated rate of fire spread, the fire took more or less time to achieve a parabolic headfire shape and pseudo-steady-state spread rate.

In contrast, the experimental approach of Cruz et al. (2015) used smaller burn plots (33 × 33 m), which arose from logistical constraints related to the need to treat the fuel with herbicides to artificially induce variations in the degree of curing. The ‘outward’ ignition method used by Cheney et al. (1993) was not considered appropriate in these smaller plots as under the moderate to strong winds required, the fire was not guaranteed to reach the pseudo-steady-state condition within the burn plot boundaries. An alternative ignition method was required that would quickly produce a fire with a pseudo-steady-state rate of spread.

The ‘inward’ ignition method (i.e. two igniters travelling quickly from the outer edge of the ignition line towards its centre) was found to produce a faster build-up of the flame front than that obtained with the ‘outward’ ignition. The effectiveness of this ignition method requires the igniters to move at a fast pace, i.e. faster than 2 m s⁻¹. The faster the ignition, the quicker the flame front build-up, with an experimental fire generally attaining a parabolic headfire shape before the fire front had travelled half the plot length, after which fire behaviour measurements could be conducted with confidence. Important to this discussion is that in the work of Cruz et al. (2015, 2016, 2018), only fires that had attained this parabolic-shaped head were used in subsequent data analysis.

Sutherland et al. (2020) have suggested that the data from the two ignition methods – i.e. ‘outward’ (Cheney et al. 1993) and ‘inward’ (Cruz et al. 2015) – were collected at different stages of fire development and thus dynamics, with the data from Cruz et al. (2015) not necessarily reflecting a pseudo-steady-state rate of spread. Consequently, they claim the data from the different studies should not be combined in a modelling analysis. An analysis by Cruz et al. (2018), in which the data from the fully cured experimental fires in Cruz et al. (2015, 2016) were contrasted with the natural grass fire spread rate model of Cheney et al. (1998), found no bias in model predictions. As the model of Cheney et al. (1998) was developed from a combined dataset of observations of wildfires and large experimental fires (Cheney et al. 1993), the absence of any bias suggests that the experiments carried out in the 33 × 33-m plots with the ‘inward’ ignition method were in fact capturing the essential fire spread dynamics responsible for the flame front propagation of a wildfire in these fuels. If the experimental fires from the study of Cruz et al. (2015) did not reach their pseudo-steady-state condition or were in the so-called ‘surge’ phase (Sutherland et al. 2020), then the model of Cheney et al. (1998) would over- or under-predict respectively the spread rate of the Cruz et al. (2015, 2016) experimental fires.

To further investigate if the two experimental datasets do in fact have diverging characteristics, we compared the predictions of the Cheney et al. (1998) natural grass fire spread model against the datasets of Cheney et al. (1993) and Cruz et al. (2015, 2016) separately. Fig. 1a, b shows the distribution of the predicted and observed rates of fire spread for both datasets and the prediction residuals. Visually, one cannot observe any differing trends in the spread of residuals (Fig. 1b), with both data groups seemingly integrated. If the claims of Sutherland et al. (2020) were correct, we would observe the Cruz et al. (2015, 2016) data in Fig. 1a clearly to the left of the 1 : 1 line (the model would underpredict the data as the observations were made in the ‘surge’ phase). A t-test for the distribution of the residuals (Fig. 1c) found no significant differences between the two populations (P = 0.29), with the average residual being –4.3 and –9.1 m min⁻¹ respectively for the datasets of Cheney et al. (1993) and Cruz et al. (2015, 2016). Clearly, and contrary to the assertion of Sutherland et al. (2020), the two datasets do not have distinctly different fire behaviour characteristics.

Fig. 1.  Plots of (a) observed rate of fire spread in fully cured experimental plots (Cruz et al. 2015, 2016) v. predicted value according to Cheney et al. (1998) natural grass fire spread model (the dashed lines around the line of perfect agreement indicate the 35% error interval as per Cruz and Alexander 2013); (b) residual distribution as a function of observed rate of fire spread; and (c) boxplot describing the distribution of residuals (the box defines the upper and lower quartiles and the outer horizontal lines outside the box indicate the 90th and 10th percentiles; the closed circles are points that fall more than 1.5 times the interquartile range above the third quartile or below the first quartile).

Sutherland et al. (2020) suggest that the fire spread data collected by Cruz et al. (2015) in plots smaller than 50 m long were not from fires spreading at their potential pseudo-steady rate. Cruz et al. (2020) conducted sequential measurements of rate of fire spread in experimental fires lit from ignition lines of 33 m along a 50-m downwind run, with measurements conducted in three distinct 10-m segments after a fire front development section of 19 m (data provided in Cruz et al. 2020). The 19-m length of this section was an informed choice that took into account the need for three 10-m long fire spread measurement segments and observations from previous experimental fires. Fig. 2a shows the observed variation in rate of fire spread for individual fires conducted in harvested and unharvested wheat. Contrary to the artificial, constant wind experiments in Sutherland et al. (2020), the measured data show no trend in rate of spread variation with position in the plot. A detailed analysis of these data by Cruz et al. (2020) found no significant differences between the rates of fire spread measured in the three distinct segments (Fig. 2b). As a population, fires were not accelerating or decelerating once they passed the first interval marker at 19 m from the ignition line (Cruz et al. 2020). The variation in rate of fire spread observed in Fig. 2a is best explained as being a function of the random variation in wind speed. Measurements of rate of fire spread in Cruz et al. (2015, 2016) started after the flame front had progressed at least 18 m, making the findings in Cruz et al. (2020) applicable to this dataset.

Fig. 2.  Plot of (a) variation in measured rate of spread with distance along plot from ignition for each spread period for harvested and unharvested wheat fuels for the experimental fires of Cruz et al. (2020). Spread period distances from ignition line: spread period 1: 19–29 m; spread period 2: 29–39 m, spread period 3: 39–49 m. (b) Boxplot describing the distribution of the variation of measured period rate of spread for the experimental fires of Cruz et al. (2020) as a percentage of the plot mean. Box defines the upper and lower quartiles. The outer horizontal lines outside the box indicate the 90th and 10th percentiles. Closed circles are points that fall more than 1.5 times the interquartile range above the third quartile or below the first quartile.

Clearly, these real-world data indicate that the suggestion by Sutherland et al. (2020) that grass fires ignited by the inward ignition method need at least a 50-m run to achieve pseudo-steady state is incorrect. Interestingly, the Sutherland et al. (2020) simulation that most resembles the experimental protocol of Cruz et al. (2015, 2016, 2018, 2020) is the one where the ignition is carried out at an assumed fast walking pace (i.e. 2.4 m s⁻¹). In this case, the model simulation ‘achieve a quasi-equilibrium within ~20 m’, which fits the experimental observations, but this very relevant fact was for some reason not discussed in the conclusions of Sutherland et al. (2020).

Fire behaviour simulation models have been used to inform experimental design and the desired environmental conditions necessary to attain a level of fire propagation that meets research objectives (e.g. Linn et al. 2012; Clements et al. 2019). The use of physics-based fire behaviour models in this context can certainly benefit a field program aimed at investigating particular aspects of fire propagation. Nonetheless, it is important that the models provide simulations that are in accordance with reality if they are to be of value. The use of untested models or the use of a model parameterisation that aims to fit a narrative that is not necessarily true can result in erroneous findings that might negatively impact an experimental burning research program.

Here, we have shown conclusively through empirical evidence that the conclusions reached by Sutherland et al. (2020) pointing out the inadequacy of the experimental methods used by Cruz et al. (2015, 2018) are wrong. The evidence from two field campaigns with measured fire spread data indicate that experimental fires in Cruz et al. (2015, 2018) using the inward ignition method attained potential pseudo-steady-state spread in approximately the first half of the plot, allowing unbiased fire behaviour measurements to be made in the second half of the experiment. Our analysis also showed that the trends in fire spread observed in Cruz et al. (2015, 2016) are consistent with the ones observed by Cheney et al. (1993) with their larger experimental fire plots.

Although the conclusions of Sutherland et al. (2020) regarding the adequacy of the methods of Cruz et al. (2015) are erroneous, the same cannot be said about their simulation results, although only one of their eight inward ignition simulations resembled the conditions as typically attained by Cruz et al. (2015). There are several reasons that might explain why some of the simulations of Sutherland et al. (2020) depart from the documented experimental fire observations of Cruz et al. (2015). Given the exposure of grass fuels to the open wind, variation in wind direction and speed are known to have an immediate and strong impact on the shape and characteristics of grass fire flames, leading to short build-up times (Cheney and Gould 1995; Cheney and Sullivan 2008). The fact that the simulations of Sutherland et al. (2020) do not consider the input wind as a dynamic variable with continual changes in speed and direction as is observed in nature (e.g. as simulated by Linn et al. 2012) but treat it as a constant inlet flow without dynamic variation may explain why their simulations point to longer runs being necessary to achieve a pseudo-steady-state. This fixed inlet wind speed was justified by Moinuddin et al. (2018) owing to the model’s lack of response to artificial variations in wind speed. Plotted results published by Moinuddin et al. (2018) show the physical model to under-represent the effect of wind speed in grass fire spread, as compared with experimental observations and derived empirical models. This might explain why fire in the model does not respond appropriately to changes in wind speed and direction as observed in nature.

There are also results contained in Sutherland et al. (2020) that hint at artificial, non-fire-physics-related factors influencing model outcomes. The authors point out that the choice of the size of the simulation domain can lead to differences in rate of fire spread of 100%. These authors also discuss the effect of the grid size in the simulated rate of fire spread, mentioning that halving the resolution from 0.5 to 0.25 m results in an approximate doubling of the equilibrium rate of fire spread, pointing to an apparent lack of scale invariance (Sullivan 2019) in the model. Nonetheless, the results in Sutherland et al. (2020) using the 0.25-m resolution appear to closely match those of Mell et al. (2007) using a 1-m resolution grid. This seemingly nonsensical result is not explained in Sutherland et al. (2020). It is also unclear why their simulations artificially converge to a comparable rate of fire spread after a 100-m run (the length of their burnable grid), independently of the ignition line length and prevailing wind speed. Unless a physics-based fire behaviour model is able to produce simulations that are solely the result of the physical processes driving the phenomena under study, one will never be certain if the results, namely relationships between fire characteristics and environment variables, are due to a real effect or are an artificial construct arising from the model formulation.

The authors declare no conflicts of interest.