Operationalising homeowner wildfire risk mitigation in fire-prone areas

Keywords: natural hazards, vegetation management, wildfire preparedness, wildland urban interface.

Understanding how communities can better prepare for wildfires is important, given the increasing effects of wildfire on an array of social values, including threats to personal safety, property and livestock and the integrity of place. Although a range of activities can increase preparedness to meet these threats – from taking action to reduce the likelihood of ignitions around the home to practicing an evacuation plan – in the USA the primary emphasis for individual resident preparedness tends to focus on evacuation and activities undertaken before a fire to increase the likelihood of structural survival via vegetation management around the structure and making structures more ignition resistant (Cohen 2000, 2010). More recent work from Australia has taken a broader notion of preparedness, including measures around evacuation decisions and active defence (see for instance, Penman et al. 2013).

Overall, the body of research in the USA has identified a range of factors that influence the decision of owners to engage in mitigation activities on their property, including risk perception, response and self-efficacy, social norms and trade-offs with other property values such as aesthetics and privacy (Toman et al. 2013). However, the influence of various sociodemographic factors on homeowner mitigation decisions varies across quantitative studies. For instance, many demographic factors, such as age and income, do not have consistent effects (McCaffrey and Olsen 2012). Some studies have found that older respondents completed more wildfire risk-mitigation activities than younger respondents (Collins 2008; Kyle et al. 2010; Brenkert-Smith et al. 2012; McNeill et al. 2013), but others found no effect of age (Martin et al. 2009; Fischer 2011). Still others found that older respondents were more likely to make structural changes to their home, but were not more likely to manage their vegetation (Dickinson et al. 2015).

Absher and Vaske (2007) suggest that demographic variables have less explanatory power than situational (e.g. renter or homeowner, seasonal or full-time) or psychological (e.g. response efficacy, knowledge) factors, but it is difficult to generalise across studies due to the range of ways that mitigation has been operationalised. Studies vary both in terms of the activities that have been considered and the ways in which these activities are ‘counted’ and subsequently analysed. Some studies do not specify how they chose which activities they would focus on, and others draw directly from specific outside sources and experts (e.g. Bright and Burtz 2006; Martin et al. 2007; Dickinson et al. 2015; Koksal et al. 2019). Some examine primarily vegetation management activities or do not distinguish between types of activities (Bright and Burtz 2006; Fischer 2011), while others examine mitigation across two dimensions, separating vegetation management and structural fire-resistance (Absher and Vaske 2007; Dickinson et al. 2015).

Interestingly, although a considerable body of research has addressed the question of which factors influence residents’ decisions to engage in mitigation activities, the question of how to identify and measure the extent of wildfire risk-mitigation activities is more tenuously addressed in the literature. One of the few efforts in this respect involved the development of a range of measures around wildfire preparedness, which was defined as ‘any prior cognitive or physical action that will reduce the risk to householders’ safety or the property in the event of a wildfire’ (Dunlop et al. 2014; p. 888). Specifically, operating in the Australian context where homeowners have the overt option to stay and defend their property, the authors developed measures for active defence, passive defence and evacuation that relate to the proportion of activities a respondent has completed that are applicable to them. Although this prior work made important progress by developing a consistent set of activities to assess preparedness, many of the measures, such as those around active defence and evacuation choices, are less relevant in the USA where the emphasis is on mass evacuation with little notion of homeowners engaging in active defence. Australian researchers have since used the measure developed by Dunlop et al. (2014) for their studies (e.g. Anton and Lawrence 2016; McNeill and Dunlop 2016; McNeill and Ronan 2017), but recent North American studies continue to use sets of items, drawn from a variety of sources, that focus on mitigating risk on individual properties (rather than evacuation or active defence) (e.g. Price et al. 2016; Olsen et al. 2017; Warziniack et al. 2019; Ghasemi et al. 2020).

In addition, there remain open questions about the consequences of differences in how risk-mitigation activities are operationalised and analysed. Although most studies ask whether an item has been completed (with a binary yes/no response), some focus on intention to complete risk mitigating activities (e.g. Hall and Slothower 2009; Ghasemi et al. 2020). To make a determination of the extent of mitigation, many researchers (in both qualitative and quantitative contexts) use a unidimensional measure calculated as a tally of the number of activities completed (e.g. Nelson et al. 2004; Collins 2008; Martin et al. 2009; Schulte and Miller 2010; Brenkert-Smith 2011; Dickinson et al. 2015; Koksal et al. 2019). Other researchers group respondents based on the number of activities completed (i.e. ‘highly prepared’ groups v. ‘less prepared’ groups) (Brenkert-Smith et al. 2012; Collins 2012; Price et al. 2016). Although tallies are still the more common way to measure risk mitigation, Australian researchers increasingly use the proportional measures created by Dunlop et al. (2014) who argued that, unlike the tally, this proportional measure does not ‘penalise’ respondents when an activity is incomplete because it is not applicable. For example, if a respondent does not have a deck and thus cannot clear vegetation out from under it, a tally effectively considers it to be a missing action.

Several questions also arise in relation to assessing the extent of mitigation. For vegetation management, assessing whether a resident has ‘completed’ an activity can be problematic because many activities require ongoing work and are never ‘complete’. Participants may differ in whether they tick a box indicating an activity as ‘completed’ based on their interpretation of what this means for something like vegetation management. A measure that provides some indication of the amount of vegetation management work that has been done may be a more accurate measure. That is, does it matter if participants interpret an item about completed activities as asking if they have done any work towards accomplishing it or if they have made a substantial or continual effort?

Finally, there is the question of the effect of differing analytical strategies. Many studies still use linear regression analysis in the context of wildfire risk-mitigation behaviours to model outcomes captured as counts or tallies (e.g. Collins 2008; Martin et al. 2009). However, that may not be the best method because the discrete (i.e. non-continuous) nature and frequent skew to count-level data can violate the distributional assumptions of linear regression (Allison 2009). We wanted to assess whether the interpretation of the results is substantively different using an analytical tool that is more statistically appropriate (though not necessarily more commonly used) for investigating count data in the Poisson family of regression models.

In the absence of a consistent way of representing the extent of wildfire risk mitigation, particularly in the North American context where existing efforts are less fully or directly applicable, we not only reduce our ability to make accurate comparisons across studies, but also risk working at cross-purposes between studies and contributing to contradictory findings in the broader wildfire preparedness and risk mitigation literature. We therefore wanted to examine the consequences of how different (and often arbitrary) operationalisations or interpretations of wildfire risk-mitigation items may be affecting the lessons learned across studies. In the following section we examine one of the most commonly measured activities, vegetation management, and how different strategies for counting, combining and analysing the same activities may influence the interpretation of results. We began by constructing several mitigation measures from the same set of items used in a survey of mitigation behaviours in three fire-prone areas in the USA. We then analysed the influence of contextual and demographic factors using different regression techniques, including linear multiple regression and negative binomial regression, a variant of Poisson regression, often used for tallies. We used this approach to address the following research questions:

1. RQ1: Are there meaningful differences in the interpretation of results when activities are counted towards a tally based on ‘at least some’ work put towards the activity versus counted only when respondents report undertaking ‘a lot’ of work?

2. RQ2: Are there meaningful differences in the interpretation of results when activities are simply tallied versus counted as a proportion of applicable activities completed?

3. RQ3: Are there meaningful differences in the interpretation of results when tallies or proportions of risk-mitigation activities are predicted through linear versus negative binomial logistic regression?

For our analysis we drew on a survey administered between October 2009 and April 2010 from four counties in three states: Alachua County, Florida; Ventura County, California; and the Jefferson and Lewis and Clark counties in Montana. These sites had been originally selected for their high wildfire risk, diverse population and mix of land use and ownership patterns. The survey was designed to assess influences on both homeowner mitigation behaviour and evacuation intentions and beliefs. Details of the survey design, implementation, and general results can be found in McCaffrey and Winter (2011). Sample statistics are presented in Table 1.

Table 1.  Summary of demographic variables of sample (restricted to homeowners)

To provide a baseline single item to measure the extent of risk mitigation, the survey asked to what extent respondents had managed vegetation to prepare for a wildfire that threatened their neighbourhood (i.e. 1–4, ‘Not at all’, ‘Only a little’, ‘Somewhat’, and ‘A great deal’, respectively). Because vegetation management is generally an ongoing process, the survey also asked respondents to indicate the amount of work undertaken (‘Have done a lot of work’, ‘Have done to some degree’, ‘Haven’t done at all’) for seven specific vegetation management activities (Table 2). Respondents could also indicate that the activity was not applicable for their home.

Table 2.  Percentage of respondents in each category for vegetation management activities

We then created four risk-mitigation outcome variables from responses to the seven specific vegetation management activities to reflect different types of counts in the wildfire risk-mitigation literature. Specifically, we created both a simple tally of all the possible activities completed as well as a proportion of all the applicable activities completed. We created two versions of each measure: one that counted activities as complete when participants indicated they put at least some work towards it (i.e. those who indicated they did some or a lot) and another that only counted an activity when a participant reported putting a lot of work towards it.

The independent variables of interest were gender (0 = female), education (dummy coded for either less than a college degree, college degree or more than a college degree), household income (6 levels ranging from <US$20 000 to >US$100 000), location (Florida, California or Montana, using Montana as the reference category), age (in years), length of time spent living in the home (in years) and a measure of belief about the person most responsible for protecting the property from wildfire (dummy coded for either more firefighter responsibility or more homeowner responsibility with equal responsibility as the reference category).

Having created five outcome variables (the single-item measure, the tallies and proportions based on at least some work or a lot of work) (see Table 3), we modelled each outcome variable using the independent variables described above as predictors using two different methods: linear regression and negative binomial logistic regression. Negative binomial logistic regression is a fixed-effects regression method in the Poisson family that is commonly used with highly variable count data, particularly where 0 counts are common, driving down count means while variances remain high, creating overdispersion that violates the assumptions of normal Poisson regression (Allison 2009).

Table 3.  Risk mitigation measures

Negative binomial logistic regression requires integer count outcome variables and is therefore not conducive to being used to predict proportions of applicable activities. However, negative binomial logistic regressions can be run using an ‘offset’ variable that takes into account differences in ‘exposure’ for each participant regarding the count variable of interest. In this way, you can take into account the maximum value that a count might reach for any given participant. If we create an offset variable from the number of behaviours that participants reported as applicable to them, it creates an analogous situation to predicting a proportion of applicable behaviours in the context of a count variable. In our negative binomial logistic regressions, we allowed the ancillary parameter that assesses the extent of overdispersion to be freely estimated in each of our models. Finally, because negative binomial logistic regression requires count data, we used only linear regression to model responses to the single-item measure.

As a result, we ran a total of nine models, investigating the single-item measure using linear regression and then tallies of the activities that participants put at least some work towards and those they put a lot of work towards as either simple tallies or proportions of applicable activities. In all the negative binomial logistic models described below, the coefficients are on a log scale and are difficult to interpret on their own. As a result, the exponentiated coefficients (exp(B)) are also provided and are interpretable as the proportion increase in the count that results from a 1-unit increase in the predictor in question. We used an identical set of predictors as for the linear regressions detailed above to aid in comparisons between models (Table 4).

Table 4.  Results of regression analyses predicting metrics of vegetation management from demographic and contextual variables
–/+ = negative/positive effect significant at P < 0.05; – –/+ + = negative/positive effect significant at P < 0.01;– – –/+ + + = negative/positive effect significant at P < 0.001; –*/+* = negative/positive effect marginally significant at P < 0.10

The sample for each site was wealthier, older and more educated than their respective county statistics, likely reflecting the sample focus on property owners (Table 1). A majority of the sample (61%) was male, with Montana having the largest proportion of males. Respondents in California reported the highest average income and level of educational attainment. The three areas were similar in age.

For the single-item measure, the majority (~60%) indicated that they had managed vegetation ‘a great deal’, while another 29% indicated that they had managed vegetation ‘somewhat’. The remainder was split among ‘only a little’ (6.7%) and ‘not at all’ (3.7%). Of the individual activities, the most common activity was removing vegetation within 30 feet (~9 m) of the home (92.3% reported they had done some or a lot of work) and the least common activity was removing or pruning vegetation near windows (27.8% reported they had done some or a lot of work; Table 2). Overall, the average respondent had done ‘some work’ or ‘a lot of work’ for 76% (5.3 of 7) of the possible activities (Table 2). When assessing the proportion of applicable activities completed, the level of engagement was even higher. On average, respondents put some or a lot of work towards 91% of applicable activities. Engagement remained high after limiting the scope to just those activities that respondents indicated they had put ‘a lot’ of work towards. The average respondent indicated they had put a lot of work towards 46% (3.2 of 7) of the possible activities and 57% of the personally applicable activities.

We first assessed the single-item measure of the extent to which respondents reported managing vegetation to prepare for a wildfire. The overall model was significant with an R² of 0.182. We saw significant effects of location such that those in California reported engaging in considerably less work managing vegetation for wildfire than those from Montana (b = −0.512, P < 0.0005), while those from Florida reported engaging in considerably more work than those from Montana (b = 0.234, P < 0.0005). This likely reflects differences among the three locations in vegetation and the extent of work required to maintain it. In Florida, for instance, vegetation regrowth can be quite rapid. Both age (b = 0.006, P = 0.016) and years owning the home (b = 0.006, P = 0.013) showed independent positive effects on the overall extent of vegetation management. We also saw a marginal effect of gender such that men reported putting more work into management activities than women (b = 0.083, P = 0.096). No other effects were significant.

When assessing mitigation as a tally of all activities that residents put at least some work towards, the model was overall significant; however, the R² value was quite low (R² = 0.022). Overall, we saw a significant effect of experience such that those who report experiencing a wildfire report engaging in a greater number of activities than those without experience (b = 0.264, P = 0.036). We also see a significant effect of gender such that men report engaging in more vegetation management activities than women (b = 0.293, P = 0.013). No other effects were significant.

When assessing mitigation as a tally of all activities that residents had put a lot of work towards, we got a significant model with a low R² of 0.037. Interestingly, we saw a differing pattern of effects such that the effects of gender (b = 0.273, P = 0.079) and experience (b = 0.308, P = 0.062) were only marginally significant. In addition, we now saw significant effects of household income (b = 0.117, P = 0.041) and education (b = 0.537, P = 0.006). Specifically, those with greater income reported engaging in more management activities than those with less income, when limiting the tally to those who reported a great deal of work, and those with less than a college degree reported engaging in more management activities than those with a college degree. There was no significant difference between having a college degree and more than a college degree. Finally, we saw a marginal effect of years owning the home (b = 0.015, P = 0.054), such that those who had owned their home for longer reported putting more work into a greater number of management activities.

When assessing mitigation as a proportion of applicable activities that residents had put at least some work towards, the model was significant with a small R² value (R² = 0.050). Overall, we saw significant effects of location, such that those in California reported engaging in a smaller proportion of applicable activities than those in Montana (b = −0.048, P = 0.001). We saw a significant effect of gender such that men reported engaging in a larger proportion of applicable activities than women (b = 0.029, P = 0.017). There was also a significant positive effect of age such that older participants reported engaging in a larger proportion of applicable activities than younger participants (b = 0.002, P = 0.002). Finally, there was a marginally significant effect of education such that those without a college degree reported engaging in a larger proportion of applicable activities than those with a college degree (b = 0.028, P = 0.069). There was no significant difference between those with a college degree and those with more than a college degree.

When assessing mitigation as a proportion of applicable activities that residents had put a lot of work towards, the model was significant with a marginally greater R² (R² = 0.057). Overall, there was a significant effect for location such that those from California reported engaging in a significantly smaller proportion of applicable activities than those from Montana (b = −0.083, P = 0.003). There was no significant difference between Montana and Florida. We also saw significant positive effects of age (b = 0.003, P = 0.020) and household income (b = 0.021, P = 0.014), such that older residents and those with greater income report engaging in a larger proportion of applicable activities. There was also a significant effect for education: those without a college degree engaged in a larger proportion of applicable activities than those with a college degree (b = 0.093, P = 0.002). There were no significant differences between those with a college degree and those with more than a college degree. Finally, we saw a marginal effect of responsibility such that those who reported that the responsibility for protecting property rests either mostly or solely with residents (as opposed to sharing the responsibility equally between residents and firefighters) reported engaging in a larger proportion of applicable activities (b = 0.048, P = 0.074).

When assessing mitigation as a tally of all activities that residents put at least some work towards and using the negative binomial regression model, the model was not significant, suggesting that the model was not better fitting than an intercept-only model (likelihood ratio Chi-square = 10.996, P = 0.444). However, when assessing mitigation as a tally of all activities that residents put a great deal of work towards, the model was significant (likelihood ratio Chi-square = 30.028, P = 0.002). We saw a significant effect for education such that those without a college degree reported engaging in around 17% more activities than those with a college degree (b = 0.157, exp(B) = 1.170, P = 0.016). There was no significant difference between those with a college degree and those with more than a college degree. There were also marginal effects of years owning the home (b = 0.005, exp(B) = 1.005, P = 0.070), household income (b = 0.035, exp(B) = 1.036, P = 0.059) and gender (b = 0.085, exp(B) = 1.088, P = 0.097). Specifically, we found that men, those who had owned their home for longer and those reporting greater income reported a greater number of management activities.

When assessing mitigation as an offset tally of activities (analogous to a proportion of applicable activities) that residents put at least some work towards, the model was not significant, suggesting it was not better fitting than an intercept-only model (likelihood ratio Chi-square = 10.201, P = 0.512). When assessing mitigation as an offset tally of activities (analogous to a proportion of applicable activities) that residents put a great deal of work towards the model was significant (likelihood ratio Chi-square = 48.977, P < 0.0005). We saw a significant effect of location such that those in California reported 16% fewer activities than those in Montana (b = −0.176, exp(B) = 0.839, P = 0.001). There was no significant difference between those from Florida and those from Montana. We also saw positive effects of age (b = 0.005, exp(B) = 1.005, P = 0.034) and household income (b = 0.036, exp(B) = 1.037, P = 0.023). Specifically, those who were older and those with greater income reported relatively more activities. We also saw an effect of education such that those without a college degree reported around 16% more activities than those with a college degree (b = 0.151, exp(B) = 1.163, P = 0.007). There was no significant difference between those with a college degree and those with more than a college degree. Finally, we saw a marginal effect of years in the home such that those who had owned their homes for longer reported marginally more activities (b = 0.004, exp(B) = 1.004, P = 0.069).

The results of all the regression analyses are summarised in Table 4, which shows the variables that had a significant effect on the dependent variable across each of the models to aid in comparison. Generally, the results shift depending on how risk mitigation was operationalised. This finding illuminates one likely reason why studies report inconsistency in the effect of common demographics such as gender, tenure, age, income and education. Future research should also consider how other effects related to the context or individual beliefs might vary across operationalisations as well, as these explanations have been argued as likely to be highly relevant (Absher and Vaske 2007).

This study sought to assess whether different ways that mitigation activities are operationalised and analysed affects the conclusions drawn. We found that, broadly speaking, there are consequences for the interpretation of how we count and combine activities, but that changing the type of statistical analysis generally just affected whether the resulting model was viable. These differences are readily apparent in this study only because we ran them all concurrently, whereas in general research practice the decisions governing how to operationalise the dependent variable tend to be made ahead of time. Understanding the potential consequences of different operationalisations is critical to making careful and considered choices about how to measure mitigation or preparedness in the first instance and how to interpret results that may be inconsistent across the literature. Generally, our findings supported the idea that researchers should consider adopting proportional measures of applicable activities in line with Dunlop et al. (2014) in order to account for variation in the applicability of mitigation activities. Furthermore, they suggest that researchers use more appropriate statistical techniques than linear regression to model count data and that they implement offsets where possible to approximate proportions of applicable activities. Finally, our results indicated that it may be important to consider developing measures that allow respondents to provide a more nuanced indications of the degree of their involvement in various mitigation activities than a simple binary indication of whether it exists or not, as our analyses suggested that different levels of work required for an activity to ‘count’ affect the outcomes of the models. Although our results are agnostic as to which level of work is a ‘better’ measure, it is clear that it does make a difference, suggesting that it is something researchers may want to consider. This may take the form of asking participants to identify how often they generally undertake the activity or the timeframe within which the work had been or was expected to be done for a particular activity.

We did see some similarities and differences between the single-item measure of the extent of vegetation management and the individual measures focused on specific vegetation management activities. The holistic measure was the only one that highlighted differences between Florida and Montana such that those from Florida reported more work towards managing vegetation for wildfire than those from Montana, which might lead us to conclude that those in Florida must be better prepared (or at least have mitigated more risk), but when we look at the models based on individual activities this effect disappears. This is emblematic of the differences that different operationalisations can have on the conclusions drawn from a study. We do note that the single-question assessment asked about activities taken specifically for wildfire, whereas the individual questions asked just about the amount of work done without being specific to wildfire. This raises the question of whether some variability in findings may reflect whether respondents are asked about activities being done specifically for wildfire or in general. In other ways, the single-item model showed broadly similar effects to the count models: the effects of gender, age and tenure were all captured in the other models as well, but the effects of education and income (discussed below) were absent.

When tallies varied based on the self-reported amount of work undertaken we saw differences in the effects of income and education. Generally, the models that allowed lower reported levels of work to count towards the tally showed no effect of income, while those that required a lot of work to count towards the tally showed a significant positive effect. That those with higher income were more likely to report doing more work for more behaviours suggested that income may facilitate fully undertaking mitigation activities. Similarly, those with less education (less than a college degree in our study) tended to show an increased tally in relation to engaging in a lot of work. There are several possible explanations for these results. Previous research found that lower income residents completed fewer mitigation activities than higher income residents (Collins 2008), but our analysis suggested this does not mean they are not active at all; rather, lower income residents may engage in some degree of work for many activities but may lack the capacity to fully manage and maintain vegetation.

The effect of education is more puzzling; one possibility is that those with lower levels of education may be more likely to be employed in professions where manual labour is more common and, as a result, may be more likely to use their own labour around their homes. In essence, combined with the results for income, we may see that those with greater incomes are using their income as the key resource to increase preparation while those with lower education are more likely to use their own labour as a key resource. These differences could shed light on how to target programs aiming to increase vegetation management activities. For instance, programs in lower income areas may want to target their efforts towards augmenting work capacity to help individuals accomplish more work given their limited resources. The FireSmart program in Idaho in the early 2000s is a good example of such an approach. The program used mitigation funds to pay local contractors in a lower income area to provide initial defensible space to homeowners, thereby overcoming individual capacity issues while also creating jobs (McCaffrey et al. 2011).

Our results also suggest that how we choose to operationalise activities by counting the full list of possible activities or only those that are applicable does matter (consistent with Dunlop et al. 2014). Using proportional measures as opposed to simple tallies brought out the effect of location: the simple tallies generally indicated that participants from California were undertaking similar numbers of activities compared with the two other locations, suggesting they were mitigating equivalently (or marginally less so). However, the proportional approach consistently showed that those from California reported engaging in a significantly smaller proportion of applicable activities, because a larger number of activities were likely applicable for their property. This may matter considerably, because simple tallies may underrepresent relative mitigation levels in areas where certain activities are less relevant or may overrepresent relative mitigation in areas with more ‘ground to cover’ in terms of applicable activities. Interestingly, we also saw proportions bringing out the effect of age, such that older participants reported doing a lot of work for a greater proportion of applicable activities, but not more activities overall. This suggests that one reason for inconsistent findings on age in previous research may be how mitigation was operationalised. For instance, Collins (2008) used a simple tally of activities and linear regression and found that younger residents performed fewer activities than older residents, whereas Fischer (2011) used a binary measure of whether any activity had been undertaken and binary logistic regression and found the opposite. Because in neither of those examples did the study account for whether these activities were applicable for the participants’ property, we cannot be sure if the extent to which the effect of age is due to the effect of age on the number of applicable actions rather than the proportion of applicable actions completed. Although other studies that used simple tallies have also found negative effects of age (i.e. Martin et al. 2009), suggesting that there are other factors beyond the tally/proportion format at play (e.g. the specific behaviours being assessed or the relative difficulty or resource-requirements of activities), it is difficult to ignore the role played by different operationalisations. The fact remains that, when risk mitigation is the sum of activities completed, inclusion of non-applicable activities add noise to assessments of mitigation.

Finally, we found few substantive differences in the interpretations of the linear models and their negative binomial counterparts, indicating that differences in findings between studies are unlikely to be due to different statistical techniques. This is not to say that the negative binomial logistic regressions are worse or less powerful models than the linear regressions. Rather, that a linear regression will often give you interpretable results even when it is not necessarily the most appropriate model to use for the type of data being examined. However, while these methods are still routinely being used, it is important to consider the potential pitfalls of using methods that are not designed to assess data in the forms in which it is available, because these results cannot necessarily be trusted when the assumptions are violated.

A key limitation of this analysis is that it was not based on a systematic process to develop standardised measures, such as Dunlop’s. Rather, we took existing data to engage in an assessment that few studies report on: how different ways of operationalising and analysing the same data might influence the findings. Given the focus of our analysis on differences in interpretation of different operationalisations within a sample, we cannot comment on limitations in extrapolating specific statistically significant findings beyond this study, although they certainly exist. In addition, the ways in which the questions in this particular survey were worded do not necessarily reflect the best wording for such questions in the future. That the single overall question specified undertaking activities specifically for wildfire, while the questions about individual vegetation activities did not, meant that it was not possible to cleanly assess, as had been originally hoped, whether a single question could act as a reasonable proxy for a long series of individual questions. But it did highlight for us the need to better understand how specifying activities being done for wildfire may or may not influence the response. Indeed, this may be a reason for different findings in other studies, some of which asked respondents whether they completed activities specifically for wildfire risk mitigation (e.g. Collins 2005; Jarrett et al. 2009; Ryan 2010) whereas others did not mention wildfire (e.g. Brenkert-Smith 2011), merely asking if participants had completed the activity.

Another common operationalisation practice that merits future consideration is that it is uncommon to weigh items based on the relative importance or efficacy of the activity in reducing fire risk. This is not minor, because having a fire-resistant roof has a much larger effect on home survival than whether the street address is prominently displayed (for fire response), yet most tallies count them equally, as did the ones we used for this study (for a notable exception see McFarlane et al. 2011). Ultimately, our intention was not to provide a guide for how to create such measures, but to provide additional insight to those of Dunlop et al. (2014) and Penman et al. (2013) on the need to engage in more careful consideration about how to conceptually operationalise the outcome variables for such studies and to underscore the potential for different results given different parameters.

Although it is common in the literature to conceptualise wildfire risk mitigation as the sum of activities completed or features present, this may not accurately represent relative exposure and wildfire risk mitigation if applicability of the activity to the resident is not assessed. Our results indicated that where not all items may be applicable to all respondents, researchers and practitioners may, as suggested by Dunlop et al. (2014), want to use a proportion of applicable items completed instead of a sum of completed activities.

Similarly, although it is common to use binary measures where a feature is either present or absent, or an activity done or not, such measures may obfuscate important information. This is particularly true for many activities, such as vegetation management, that require ongoing work. Some respondents may view answering ‘absent’ as the best choice because the work is not complete, even when they have undertaken a fair amount of work. Researchers and practitioners should consider expanding the range of response items to account for the fact that many vegetation activities are ongoing.

Finally, future studies of wildfire risk mitigation may benefit from assigning weights to items based on their effectiveness in making homes less likely to ignite from a wildfire.

The authors declare no conflicts of interest.