Accounting for among-sampler variability improves confidence in fuel moisture content field measurements

Citizen science for fuel moisture monitoring

Citizen science is a powerful approach for collecting large quantities of environmental data and is critical for understanding processes operating across large spatio-temporal scales (Devictor et al. 2010; Isaac et al. 2014). Among-sampler variability is inherent in field studies involving multiple individuals, as people naturally carry out tasks differently, and can arise through different levels of experience, motivation, and different interpretations of protocols. The importance of among-sampler variability has been reported in previous environmental citizen science projects, particularly for studies monitoring species presence-absence and quantifying flora percentage cover (e.g. Morrison 2016 and references therein; Sicacha-Parada et al. 2021; Nolan et al. 2022b). Some sources and impacts of among-sampler variability can be lessened through careful experimental design (e.g. developing effective training protocols, recruiting individuals with similar levels of prior experience). However, among-sampler variability cannot be completely eliminated, and this error is incorporated into the resulting measurements (Bird et al. 2014; August et al. 2020).

Accounting for among-sampler variability allows us to optimise citizen science opportunities to understand fuel moisture dynamics at broad spatio-temporal scales that cannot be captured by traditional, controlled field experiments (Dickinson et al. 2010; Arazy and Malkinson 2021). This is important for developing robust fuel models and is especially important in environments where cross-scale fuel moisture dynamics are not fully understood. Considering potential among-sampler variability prior to conducting large-scale fuel moisture campaigns can allow for targeted collection times, locations, and sampler metadata to isolate spatio-temporal fuel moisture dynamics.

Research questions

We conducted an intensive sampling campaign to determine the magnitude of among-sampler variability in measured fuel moisture at the plot scale within a Calluna vulgaris dominated temperate fire-prone landscape (cf. Davies et al. 2010). Our findings are widely applicable for estimating among-sampler variability in fuel moisture measurements and enabling a confidence range to be applied to estimates used in a practical setting. Accounting for among-sampler variability also opens opportunities for large-scale fuel moisture monitoring campaigns using citizen science, which are necessary for understanding fuel moisture dynamics at regional and national scales.

Study site

The field campaign was completed by an undergraduate geography field class (n = 17) in the Lickey Hills Country Park, Birmingham, England (52.3723°N, 2.0045°W). The site was selected as it is representative of the type of heathland landscapes that are found throughout temperate fire-prone environments (Glaves et al. 2020). Two Calluna vulgaris-dominated plots were selected, and samplers were evenly split across the two plots to minimise overall destruction of Calluna in one location. The two plots were both ~300 m² situated on hillslopes on the same shallow, acidic, peaty soils (Farewell et al. 2011) ~1000 m apart. Plots were dominated by Calluna and interspersed with Vaccinium myrtillus (common bilberry) and Pteridium aquilinum (bracken).

Fuel moisture sampling campaign

Students sampled six Calluna fuel layers each hour (Fig. 2). These layers were: live canopy; live stems; dead canopy; dead stems; surface moss (Kindbergia praelonga dominant species); and surface litter. Each sampler collected one set of samples every hour between 10:00 hours and 18:00 hours, resulting in nine sets of samples per sampler and a total of 918 samples overall. None of the samplers had monitored fuel moisture before, and all samplers received the same protocol (see Supplementary Material and Fig. S1) adapted from Norum and Miller (1984). We provided a briefing prior to beginning sampling and advice during the first hour of sampling to ensure correct species identification and sample size for laboratory analysis. Each sampler was instructed to haphazardly collect fuel clippings from each fuel layer across the entire plot area (~ 10 different plants) in accordance with Norum and Miller (1984). The undergraduate students stored clippings in an aluminium tin with a screw-fit lid sealed with masking tape. We calculated gravimetric fuel moisture content (FMC, mass of water per mass of dried sample, %) following Norum and Miller (1984). We weighed the tinned samples (wet weight) as soon as possible the morning after collection. We then dried the samples for at least 48 h at 80°C and reweighed them (dry weight). All fuel moisture contents are presented throughout as percentages; mass of water as a percentage of the mass of the dried sample (%).

Fig. 2.

Visual depiction of the six different fuel layers sampled by the students each hour through the sampling period. Red line depicts the boundary between the canopy above and the stems below (Photo credit: Kerryn Little).

Data analysis

We used mixed effects models with time as a fixed effect and sampler identity as a random effect. By including sampler as a random effect, this allowed us to estimate the among-sampler variability (Fig. 1), where the actual identity of the sampler is not important. The standard deviation of the random effect can be interpreted as the among-sampler variability once fixed effects (in this case time of day) have been accounted for. We also calculated the coefficient of variation (standard deviation of the random effect divided by the mean FMC% for a given layer) to facilitate comparability across fuel layers of different fuel moisture content ranges.

Finally, we calculated the model marginal R² (variation explained by the fixed effects) and conditional R² (variation explained by both fixed and random effects) (Nakagawa and Schielzeth 2013). The difference represents the amount of variation explained by among-sampler variability, and therefore gives an understanding of how important accounting for among-sampler variability is for a given layer. We conducted all statistical analyses in R version 4.1.2 (R Core Team 2022) using packages lme4 (Bates et al. 2015) and MuMIn (Bartoń 2022).

Fuel moisture measurement variability

High fuel moisture content measurement variability was observed across all fuel layers (Fig. 3). Individuals sampling within the same plot at the same time obtained different fuel moisture content measurements up to a maximum range of 320% (moss), 249% (litter), 76% (live canopy), 72% (dead canopy), 68% (dead stems), and 39% (live stems). Most fuel layers had a right-skewed distribution of measured fuel moisture content, with a high upper quartile, upper extreme and high fuel moisture outliers. Live Calluna had more of an even distribution of above and below median measured fuel moisture content. Measured live and dead Calluna fuel moisture content was highest at 10:00 hours and generally decreased throughout the day before starting to increase again at the end of the sampling period. This diurnal pattern was not evident in the wettest fuel layers (Fig. S2).

Fig. 3.

Measurement variability in fuel moisture content measurements collected by each sampler from 10:00 hours to 18:00 hours by fuel layer. Each y-axis is scaled independently to clearly visualise within-fuel layer measurement variability.

Among-sampler variability in fuel moisture content

Among-sampler variation, measured as the standard deviation of the random effect, ranged from 1.25 (dead canopy layer) to 32.13 (moss layer). Because each layer has a very different mean value for FMC%, we also present the coefficient of variation, which allows us to provide relative estimates of among-sampler variation. These range from 0.04 (live stems and dead canopy) to 0.56 (litter layer). In general, among-sampler variation is larger in the wetter layers (moss and litter) (Table 1). The amount of variance explained by the random effects (conditional R² – marginal R²) can provide us with information about how important it is to take account of the among-sampler variability. This was greater than the amount of variation explained by the fixed effects for all layers except dead canopy, suggesting it is crucial to account for among-sampler variability in such studies. It should be noted, however, that the R² values are sensitive to the sample size and study specifics. Time of sampling explained up to 14% (live canopy layer) of the variability in the data.

Quantifying among-sampler variability in fuel moisture estimates

Seventeen samplers collecting fuel moisture samples at the same time within the same site measured very different fuel moisture contents. With the exception of dead canopy material, among-sampler variability was more important than time-of-day in explaining the total measured fuel moisture variation of each fuel layer at the plot scale. Significant attention is given to diurnal variability in fuel moisture content in rapidly drying fine fuels (e.g. Slijepcevic et al. 2013; Bilgili et al. 2019; Zhang and Sun 2020). However, we have shown that among-sampler variability can exceed diurnal drying patterns and should also be considered in fuel moisture dynamics studies.

Among-sampler variability explained the greatest amount of variation in litter fuel moisture, followed by moss and live Calluna stems and canopy. Importantly, it is not exclusively the highest absolute values of fuel moisture that are associated with large among-sampler variability. We hypothesise that some fuel layers are harder to sample and require more subjective decision-making by the sampler. Even with protocols and training, any such subjective decision-making and variation in sampling effort will produce among-sampler variability in fuel layers. For instance, samplers must identify the top 2 cm of moss and litter material, remove any attached decomposing material, and ensure fuel sample separation where litter material is interspersed in patches of moss. Subjectivity in clipping live Calluna can also incorporate additional among-sample variability due to the length of sprigs collected, where samplers choose to separate the live canopy from the live stems, and even correctly identifying live from dead Calluna. A lack of confidence in the latter could lead to subconscious targeting of the greenest live material and missing the brown live material that is harder to identify.

At the other end of the scale, among-sampler variability was low in dead Calluna but the coefficient of variation was higher than expected for such a low among-sampler variability. This is likely attributable to outliers resulting from the misidentification of live Calluna as dead. Where dead fuel is correctly identified, this material is considered easy to collect following the sampling protocol and among-sampler variability is low. Among-sampler variability in this case is mainly a concern where brown Calluna is incorrectly identified as dead. Where dead fuel moisture is the most important variable, among-sampler variability may be less important to account for than time-of-day and illogical values from mis-identifications can be filtered out of the dataset.

There may also be variability within individual samplers through time. For instance, sampler accuracy may increase with experience gained or decrease due to fatigue as the day goes on. These changes may be intertwined with diurnal fluctuations in fuel moisture variability, as was the case in this study, and so are unable to be disentangled. However, temporal changes in individual sampler variability should be considered during sampling campaign design to minimise this influence (e.g. allow time for instructions, practice, and feedback before beginning and consider the required length of sampling campaigns to manage fatigue and comfort of samplers).

Considering among-sampler variability in survey design

Carefully considered sampling protocols can minimise sources of among-sampler variability prior to field collection (Dickinson et al. 2010; Morrison 2016). We aimed to reduce sampling effort variability by having a clear protocol for where, when, and how samples should be collected. We also controlled for among-sampler differences by recruiting volunteers from the same cohort with no prior fuel moisture sampling experience and provided them with the same level of training. Where sources of variability cannot be minimised through sampling protocols, statistical tools can sometimes be used to account for among-sampler variability from other sources of error (Bird et al. 2014; August et al. 2020). Statistical models such as mixed effects models (Aagaard et al. 2018) and machine learning tools like boosted regression trees (Cox et al. 2012), random forests, and artificial neural networks (Fink and Hochachka 2012) have been used in citizen science ecological studies to account for this variability. However, statistical tools can only be utilised when it is possible to isolate sampler identity from other covariates. Where sampler identity is not known or is confounded with other variables such as geographic location or time, the underlying controls on fuel moisture variability cannot be disentangled.

Larger fuel moisture sampling campaigns that aim to quantify the spatio-temporal variability in fuel moisture content will likely require greater flexibility in where and when samples are collected and who is recruited to collect samples. In these situations, the collection of sampler metadata (e.g. sampler experience, training received, and profession (e.g. heathland land managers may have greater familiarity and confidence in identifying fuel layers than others)) could also be collected to further quantify among-sampler variability (Kelling et al. 2015). Fuel moisture samples should have a sampler identifier to relate metadata metrics to fuel moisture content. Sampler metadata can be used to control sampling designs to prevent confounding with covariates, filter databases for analyses, and include metrics in models to isolate fuel moisture measurements from sources of sampler variability (August et al. 2020).

Implications

Among-sampler variability can lead to high variability in fuel moisture content measurements within the same plot, fuel layer, and time of day. With this knowledge, we can give a range of confidence in fuel moisture estimates associated with among-sampler variability that will be more accurate than a single value. Accounting for among-sampler variability opens opportunities to maximise the potential of citizen science research to characterise fuels and understand regional fuel dynamics beyond the capability of most traditional field experiments. For instance, capturing fuel load and fuel flammability are also essential to develop fuel models that are representative of the range of regional ecosystems they are being developed for. Fuel height and fuel samples could be collected by citizen scientists for research developing wider aspects of fuel and fire behaviour models. Furthermore, community hubs of citizen scientists could implement long-term fuel moisture monitoring campaigns to assess local wildfire danger, thereby creating wildfire-aware community networks within rural–urban interfaces and promoting risk reduction strategies.