Generating fuel consumption maps on prescribed fire experiments from airborne laser scanning

Study area

Stand-replacement fires are operationally prescribed on subalpine forests in the Fishlake National Forest, with a primary objective of removing older beetle-killed or encroaching younger conifers and reestablishing quaking aspen (Populus tremuloides). Five prescribed fires in 2017–2020 provided FASMEE researchers with opportunities to collect measurements on fuels, fire behaviour, heat release, plume processes, smoke emissions, and fire effects on vegetation and soil (Ottmar et al. 2021). The main FASMEE burn units (Fig. 1) were the Manning Creek Unit (MCU) burned on 20 June 2019, the Langdon Mountain Unit (LMU) burned on 7 November 2019, and the Annabella Reservoir Unit (ARU) burned on 5 November 2020. Two additional prescribed burns were conducted adjacent and prior to (October 2017 and 21 November 2018) the MCU burn to act as firebreaks, referred to hereafter as Blackline (BL) units (i.e. BL2017 and BL2018; Fig. 1). Fuel moisture on the day of burning was much higher at ARU than all other units due to snow (~35% for 1- and 10-h fuels). LMU had the lowest fuel moisture (~10% for 1- and 10-h fuels; MCU: ~15%). See DRIScience (2020) for an overview of the FASMEE project and video footage from within the MCU as an example of fire behaviour.

Fig. 1.

Overview of the three burn units on Monroe Mountain in the Fishlake National Forest south of Richfield, UT: (a) Annabella Reservoir (ARU), (b) Manning Creek (MCU) and adjacent blackline units (burned in 2017 on the northeast edge of the MCU and in 2018 on the southern edge of the MCU), (c) Langdon Mountain (LMU). Plots (n = 61) were clustered into seven sampling areas.

Elevations ranged from 2449 m at MCU to 3212 m at ARU, with a mean elevation of 2908 m across all units. The burn units were mostly forested with quaking aspen, sub-alpine fir (Abies lasiocarpa), and Engelmann spruce (Picea engelmannii), with open areas of sagebrush (Artemisia sp.) acting as natural firebreaks on the unit edges. None of the burn units had experienced fire for over 150 years (Linda Chappell, personal communication).

Field data

Between 2016 and 2021, 61 plots were sampled twice (i.e. pre and post-fire) in MCU (n = 20), LMU (n = 10), ARU (n = 20), BL2017 (n = 6), and BL2018 (n = 5) by the Fire and Environmental Research Applications Team of the USDA Forest Service (Fig. 1). Plots were divided between seven sampling areas (5–20 plots per area) and systematically arranged on a grid within each sampling area. These sampling areas were distributed across the three main burn units and two blackline units to represent the range of forest types (Ottmar et al. 2021). Each plot was defined by a centre point geolocated using a Javad Triumph-2 differential GPS (Javad GNSS, Inc., San Jose, CA). To estimate litter and duff fuel loads, we measured litter and duff depths at eight points per plot, computed an average depth for each material, then multiplied it by published material-specific fuel bed density values (Prichard et al. 2017). Coarse (100- and 1000-h) down woody debris (DWD) was calculated following Brown et al. (1982) using measurements along three 10 m transects extending radially from plot centre, based on a single random azimuth then spaced equally 120° apart. Each plot had two nested 1 m² clip plots placed 5 m from plot centre, oriented at 0° (north) and 180° (south). The northern clip plot was used to collect pre-fire fuels and the southern for post-fire fuels. Herbaceous fuels, shrubs, and seedlings were clipped (destructively sampled) then oven-dried and weighed to obtain fuel loading, while fine DWD (1- and 10-h) were collected (destructively sampled) within one quadrant (0.25 m²) of the clip plot. For canopy fuel, we recorded species, status (live/dead), and measured the DBH of all trees (>12 cm DBH) within an 8 m radius from plot centre. Additionally, total height, height to live crown, and canopy base height were measured for a subset of live trees. Saplings (≤12 cm DBH and >1.37 m tall) were tallied within a 5.6 m radius and an average sapling DBH was used. We calculated fuel load for trees (i.e. boles, snags, and crown material), saplings, and available canopy fuel (ACF) (i.e. foliage plus 50% of fine branches) using the Fire and Fuels Extension of the Forest Vegetation Simulator (Reinhardt and Crookston 2003). Additional details on field data collected and plot design can be found in Ottmar et al. (2021).

ALS data

Pre-fire ALS data were collected by Digital Mapping Inc. using an Optech ALTM 167 GEMINI sensor between 27 August and 11 September 2016 at ≥8 pulses/m² and with 60% overlap. Post-fire ALS data were collected by Technical Applications and Consulting, LLC using an Optech Galaxy T500 sensor. Post-fire ALS data for MCU, LMU, BL2017, and BL2018 were collected on 30 September 2020 at ≥8 pulses/m² and with 80% overlap, while data for ARU were collected on 15 June 2021 at ≥7 pulses/m² and with 100% overlap. Using LAStools (Isenburg 2013), we normalised the height of the ALS point clouds, then calculated statistical metrics (e.g. height percentiles, canopy cover, point density in height strata; Supplementary Table S1) within 8 m circles at each plot (i.e. the area within which trees were tallied) from both the pre- and post-fire ALS acquisitions. The same statistical metrics were calculated for 10 m raster grids across each ALS acquisition.

Data analysis

We trained RF models to predict fuel loading of canopy (trees and saplings), shrub/seedling, herbaceous, DWD, litter, duff, total fuel, subcanopy fuel (shrub/seedling, herbaceous, DWD, litter, and duff), and ACF using the combined set of pre- and post-fire field data as the independent observations and the corresponding pre- and post-fire ALS statistical metrics within the 8 m circles of each plot as the predictor variables. We used an iterative variable selection process to reduce the number of final predictor variables (McCarley et al. 2020, 2022; Supplementary Table S1). To estimate fuel consumption across the burn units, we predicted pre- and post-fire fuel load using the RF models to 10 m rasters, then differenced them.

To assess the quality of the RF fuel loading models, we evaluated variance explained (i.e. pseudo-R²) reported in the RF outputs. We validated consumption estimates in three ways: (1) comparing estimated fuel consumption to observed fuel consumption at all the plots; (2) performing 3-fold cross-validation, where data were selected for each fold at random, irrespective of burn unit; and (3) performing a geographic 3-fold cross-validation, where the training and validation plot subsets were separated between the burn units of unequal sample size (Fig. 1). Fit statistics of root mean squared error (RMSE) and %RMSE (i.e. RMSE divided by the mean of observed values) and mean bias estimate (MBE) (i.e. the mean of predicted minus observed values) and %MBE (i.e. MBE divided by the mean of observed values) were used to assess the models and make comparisons. The first approach tested how well the predictions matched the observations as a general model validation. The random 3-fold cross-validation provided a more robust validation by withholding independent plots, such that any spatial dependence between training and validation plots was lessened. The geographic 3-fold cross-validation tested spatial transferability, or the ability of the models fit from data in only two burn units to provide accurate predictions in the third burn unit situated well beyond the range of spatial dependence. We expected that the: (1) general model validation would produce the best fit statistics, but be potentially overfit; (2) random 3-fold cross-validation would provide the most defensible quality assessment for consumption estimates derived from a model simultaneously predicting fuel loads in these three FASMEE burn units; and (3) geographic 3-fold cross-validation would provide the most defensible quality assessment for consumption estimates derived at other fires prescribed elsewhere across Monroe Mountain.

Fuel consumption

Canopy fuel variables (i.e. canopy fuel, total fuel, and ACF) had the best model performance (highest pseudo-R²) and were most precise (lowest %RMSE) and accurate (lowest absolute %MBE) upon validation (Table 1). Others have also demonstrated that ALS data is less accurate when modelling subcanopy variables due to canopy occlusion and inability to directly capture the depth of some variables such as litter and duff (Bright et al. 2022; McCarley et al. 2022). Our models for DWD, litter, duff, and subcanopy fuel explained between 25.6 and 42.9% of variance. A review of literature on predicting subcanopy fuels from ALS data by Bright et al. (2022) found the range of variance explained to be 14–71%, suggesting our results are typical. Our model explained between 64.9 and 76.0% variance in total fuel, ACF, and canopy fuel. This is also typical in literature, with others demonstrating better fitting models of canopy than subcanopy fuels (Price and Gordon 2016; McCarley et al. 2020; Mauro et al. 2021). It follows that the models for subcanopy fuels performed worse when validating fuel consumption (Table 1). Therefore, users of the data we generated should know that there are potentially greater errors in the fuel consumption estimates for DWD, litter, duff, and subcanopy fuel.

For maximising accuracy and incorporating subcanopy fuels, estimates of total fuel were best. In validating total fuel consumption, it had the second lowest %RMSE and the third lowest absolute %MBE, behind canopy fuel and ACF (Table 1). The total fuel model likely had higher relative accuracy because of the stronger statistical relationship between ALS data and overstory canopy, as discussed previously, and the ecological relationship between the canopy and subcanopy, which has been demonstrated by others (e.g. Lydersen et al. 2015; Cansler et al. 2019).

Spatial transferability

In comparing consumption modelled from the full set of plots with the geographic cross-validation, the lower RMSE and MBE indicated that spatial transferability would result in loss of precision and accuracy. That is, parameters from RF models trained in a certain area are likely to result in poorer predictions when applied to a novel area without coincident fuel measurements on the ground. However, the extent to which model performance suffers was demonstrated to vary by fuel component (Table 1). There may be cases where some loss of predictive accuracy is an acceptable trade-off to acquiring field data.

A notable complication in our results is that the sample design of the plots in our study was opportunistic from a safety and logistics standpoint but not well-designed to capture the range of forest conditions in each unit. The clustered design (Fig. 1) introduced spatial autocorrelation, limiting the true independence of the observations. When we removed plots from the models at random, the statistics for consumption were better than if all plots for a given burn unit were removed. Such a problem may lead to worse out-of-sample predictions when observation independence is further reduced, as we observed in our geographic cross-validation. In this respect, our results are not a robust test of spatial transferability, but they emphasise the need for an unbiased and spatially representative sampling design, in which (1) sample plot locations are well distributed spatially and (2) each fuel condition is sampled in proportion to its occurrence on the landscape (Goodbody et al. 2023), in future efforts to develop generalised models of fuel consumption.

Fekety et al. (2018) performed a similar geographic cross-validation analysis to test the spatial transferability of lidar-trained RF models to predict basal area and stem density. They concluded that making RF model predictions onto a novel area without training data and within the same ecoregion was feasible but would likely result in greater errors than if training data were added to the novel area. Mauro et al. (2021) explored the potential for spatial transferability across multiple ALS datasets, finding that ecosystem differences and modelling techniques affected performance the most. Tompalski et al. (2019) showed average RMSE and MBE increased by as much as 29.31 and 22.04%, respectively, when ALS models were transferred to a new area. Similarly, we found an average increase in RMSE of 11.0% and MBE of 13.9%. These findings demonstrate the limitations of spatial transferability in developing generalised models and reinforce the importance of capturing the range of biophysical conditions in the training data to achieve the most accurate predictions.