Digital soil mapping in Australia. Can it achieve its goals?

Keywords: communication, covariates, expert knowledge, geomorphic, mapping, pedology, pedometrics, soil survey.

Digital soil mapping (DSM) methods have been in development and use in Australia and elsewhere for more than two decades. Their purpose and use in Australia has been described in numerous articles, such as McBratney et al. (2000), McBratney et al. (2003), Holmes et al. (2015), Kidd et al. (2018) and Kidd et al. (2020). The use of DSM fits within a broader pedometric goal of providing soil related technology and data that are faster, cheaper and continually updated (McBratney et al. 2019). Conventional survey is generally viewed as soil survey via the ‘mental model’ of the soil surveyor (Hudson 1992) and is therefore regarded (in the DSM literature) as inherently flawed and biased by the ‘human factor’. The literature regarding DSM also promotes a view that DSM is an alternative to, or replacement for, ‘conventional’ soil survey (for example, see Grundy et al. 2020).

Early in the development of DSM, McBratney et al. (2003) listed seven topics that needed further research in order for DSM to achieve its intended goals. These topics were: (1) environmental covariates; (2) spatial decomposition; (3) sampling methods; (4) quantitative modelling; (5) quality assessment; (6) (re)presentation of digital soil maps; and (7) economics of digital soil mapping. Brevik et al. (2016) provided a similar list of topics for attention in their review of DSM. Authors such as Zhang et al. (2017) have described other challenges that exist for DSM, in particular those associated with the trend of DSM being applied to ever-larger areas, with an associated increase in model complexity. Grunwald et al. (2011) considered the philosophical views of mapping soils and the issues that impact upon DSM, suggesting that a new generation of DSM was needed that explicitly considers scaling across space and time, with improved linkages between the natural and human domains.

There has been significant progress in relation to some of these matters over the past two decades. For example, the TERN facility (https://www.tern.org.au/tern-observatory/tern-landscapes/) has made accessing spatial data of environmental covariates across the nation easy. Developments have also occurred in spatial decomposition (Odgers et al. 2014) and statistically based sampling strategies (Minasny and McBratney 2006; Kidd et al. 2015). The thoughts of McBratney et al. (2003); Brevik et al. (2016) and others remain valid to this day, but it is apparent that not all topics have been sufficiently raised or addressed, leading to some gaps in the understanding of the abilities of DSM to be a viable operational approach to soil survey. We suggest there are some additional matters that need attention and a significant question remains as to whether DSM is a viable alternative or replacement to conventional soil survey in a wide range of settings. The full analysis of such a topic is beyond the scope of this paper; our purpose here is to highlight the need for it.

The development and evaluation of DSM in an operational sense in Australia commenced with the Enhanced Resource Assessment (ERA) program in Queensland in the late 1990s (Slater and Grundy 1999). ERA was seen as a holistic toolkit of improvements to soil survey methods, of which DSM was only one part. Since then, there has been shift away from the holistic view of ERA, to a focus on DSM as a singular interest. There has been a continual evolution of paradigms, methods and tools (software) associated with DSM, from crisp soil surveyor driven decision trees in the 1990s, to fuzzy rule sets in the early 2000s; e.g. Grundy and Searle (1998), Brough et al. (2002), Claridge and Grundy (2003). Disaggregation (Odgers et al. 2015) became popular for a while, while the current focus is on machine learning (Taghizadeh-Mehrjardi et al. 2021). Expert knowledge was a critical feature in the early days, as the paradigm concerned capturing explicitly ‘the knowledge in the soil surveyor’s head’; i.e. a hybrid approach. A shift away from this occurred post-2000, but in more recent times, there has been a return to the concept of incorporating expert knowledge (Bui et al. 2020). The continual change in DSM methods is invariably portrayed as a positive illustration of the ongoing research, development and improvement of the approach. However, there is a corollary that from an operational sense, any use of a DSM method has invariably been a trial; i.e. the particular method had not been used in that context or landscape previously and consequently, the outcomes of its use were not predictable or certain. To a large extent, this remains the case today.

The continual evolution of DSM has led to a paradigm we call ‘chasing the new horizon’. It is perhaps best illustrated by the frequency with which the phrase ‘DSM has the potential to ….’ is used in DSM related publications and conference presentations. This is in comparison to conventional survey, in which the outputs and outcomes are well known and understood by both producers and users, before the survey is undertaken. This is in part because of long existence of conventional soil survey but also the large number of soil survey related standards published over time. In particular, the 1990s onwards saw the culmination (in print) of >60 years of evolution of standards and methods within conventional survey in Australia (Table 1). By comparison, DSM has only produced the GlobalSoilMap standard (Arrouays et al. 2014). Unfortunately, evidence from published DSM surveys within Australia suggests that adherence to standards is ad hoc at best and in some instances, poor; e.g. the publication of DSM data to a precision beyond that specified in the standard (and common sense) in the NAWRA explorer website (https://nawra-explorer.csiro.au/#soil accessed 13 February 2020) and in downloadable data for the Soil Grid of Australia (Viscarra Rossel et al. 2014).

Table 1.  Soil survey related standards developed over time in Australia, and digital soil mapping standards relevant to Australia.

Many of these standards are potentially just as relevant to DSM as conventional survey, and some, such as McKenzie et al. (2008), explicitly cover both. However, it is a reality that in the same era that soil survey in Australia has matured and documented its standards, there has also been the appearance and use of a new mapping approach lacking clear standards. It is certainly true that some standards for conventional soil survey are ill-defined, but there have historically been sufficient informal and formal standards in place that soil survey organisations generated high quality, consistent products produced to a common set of standards at that point in time. This by no means suggests that conventional soil survey products are ‘perfect’. There are many known problems of consistency between conventional surveys, within and between states. The question is whether DSM is the only solution to this.

A common criticism of conventional surveys is that they are not detailed enough, not accurate enough or did not capture the data needed today. While these are valid statements, they potentially also reflect a lack of understanding of the context of historical surveys in terms of project scope, methods and technology available at the time. For example, to expect a 1:500 000 scale land systems survey from 1970 to represent modern soil attributes at the paddock scale is entirely unrealistic. Is this a fault of the survey methodology or a lack of understanding by the user of the nature and limits of the data?

Some authors, such as Kidd et al. (2020), would argue that DSM has achieved operational stability. This is in part because DSM is seen by many as a general approach, rather than a specific method, thus frequent changes in the specific software/algorithms/data used is seen as advantageous flexibility rather than problematic. There are some texts published that attempt to provide standardised approaches, for example, Malone et al. (2017). However, these run a real risk of becoming outdated within a short period of time of being published. One indicator of operational stability is that products of the work are used beyond the purpose of their original creation. DSM products of the past 20 years have often not met this criteria compared to the ongoing use of soil survey products that are more than 60 years old.

We argue in a separate paper (the authors’ unpubl. data) that use of DSM does not represent operational stability. We also suggest the risks associated with continual operational implementation of new approaches have not been sufficiently evaluated. Whether it be from the perspective of planning and funding surveys, teaching/training or consistency between products/outputs, constantly changing methods does create operational problems. Operational stability is important for funding bodies and clients, as they have some certainty as to what they are getting and that it is consistent with other surveys. Operational certainty is also important from a technical/productivity perspective; the ability to produce survey outputs in long-term programs is hampered if effort is continually expended developing and learning new approaches. This does not mean that method development is unimportant; rather, that a balance must be struck between producing known products to recognised consistent standards vs the aspirational goal of producing ‘something new’. We suggest that there has been an imbalance and consequently there are ongoing risks.

The paradigm of ‘chasing the new horizon’ has at times led to an overly optimistic view of the capabilities of DSM as an operational tool. The future use of DSM in Australia requires a much clearer understanding of the operational limitations of the approach and the fundamental limits of the DSM paradigm. Operational limitations are a function of the operational environment at any point in time and may be resolved via changes such as the development of new tools/methods/technology or increased resources. Some of them are related to the translation of academic theory to operational use; statistically-based sampling strategies are a good example. Limits to the DSM approach are, however, inherent within the paradigm and consequently are less likely to be easily resolved. Below, we briefly discuss three examples that are highly relevant to the current implementation of DSM in Australia.

• The ability of the modelling and mapping frameworks to represent the landscape continuum and the outcomes of pedogenetic processes.

• The inherent limitations associated with covariates.

• Lack of primary (site) data.

Wilding and Drees (1983) suggested that variability in the natural landscape is inherently systematic, but the degree to which it appears non-random is influenced by the scale of observation. It is rare to find a landscape where the apparent variation in soil types or attributes will be entirely systematic (Wilding 1994), particularly at the normal scales of investigation in Australia (1:50 000 to 1:250 000). Consequently, there is always an inherent potential for the inability to fully predict all soil and landscape features within the self-imposed limits of the survey scale (this should not, however, be used as an excuse to produce poor quality survey outputs). DSM appears to be at risk of being inherently limited by the presence of non-systematic variability, as the method relies entirely upon the presumption that the analysis of covariate data will yield systematic relationships. Where does this limit sit and how does it vary with differing soil and land attributes, locations and scale? DSM has been used at the continental scale, but at the same time there is a push for it to be used at the paddock scale; e.g. Zhao et al. (2020). How these limits in relation to systematic variability express at different scales has not been explored in sufficient detail.

While DSM surveys report uncertainty statistics for individual attributes being predicted, there has been little analysis of the meaning of these results in the context of systematic vs non-systematic variability at particular scales. There is little value in using complex means to predict highly predictable attributes. Similarly, there is little value in complex attempts to predict highly unpredictable attributes. These are key questions requiring investigation, as they help set the bounds for when DSM is (or is not) appropriate, prior to a methodology choice in a project planning phase.

Another aspect of the modelling/mapping framework concerns the geomorphic construct. This is a widely used basis of most soil survey but is largely absent within DSM. Its absence creates model inefficiencies but also hampers communication of products. A key virtue of a conventional soil survey map is the ability to see differing scales within the landscape at once; simply via the use of good cartographic practise and the geomorphic construct. This ability has yet to be proven with DSM and remains a key limitation to the way in which DSM products are currently used and communicated.

A final limitation worth considering is the nature of covariates. The suite of available covariates has improved over the past 20 years in terms of distribution, resolution and diversity. There is, however, a fundamental assumption within DSM that the available suite will somehow represent the outcomes of both past and present pedogenic environments. Whether expressed in its original form as CLORPT (Jenny 1941) or in the DSM form as SCORPAN (McBratney et al. 2003), the pedogenetic model is often cited as underpinning the DSM approach. Despite this, no operational DSM projects in Australia have utilised a pedogenetic framework. Rather they have been models that sought correlation in data, with an associated assumption that the correlation will sufficiently represent pedogenetic outcomes. Correlation does not represent causation and in this regard, DSM is no different to conventional survey that relies on visual and mental correlation. The current approach within DSM is to utilise as many covariates as possible and ‘hope for/pick the best’. The poor predictive outcomes within some DSM projects suggests that this is not the best approach. There are also key limitations in relation to covariate resolution and locational accuracy that remain operationally significant and are likely to do so for a long time. Importantly, few covariates represent the below-surface environment effectively. Given many of the soil attributes of interest concern the subsoil, the ability to predict these in the absence of covariates representing that domain is likely to remain limited.

When acknowledged, the issues associated with covariates are viewed as an operational limitation rather than an inherent limit. Once again, the ‘new horizon’ comes into play, with the most common response being ‘soon we will have more/better covariates’. Given that the government programs that collect/derive the fundamental data supporting most covariates are outside of the pedological domain, the likelihood that the operational limitations of covariates will be resolved any time soon is a risk for DSM. Importantly, the inability of common covariates to represent pedogenic outcomes in the landscape are likely to remain into the future.

The operational impact of a lack of site data on the predictive capacity of DSM models is well recognised (Brevik et al. 2016; Zhang et al. 2017). It is a problem that is frequently commented upon as a constraint to outcomes achieved in DSM projects; e.g. Bui et al. (2006), Bartley et al. (2013), Thomas et al. (2018). Despite these acknowledgements, DSM has been promoted as a tool for use in the absence of site data; e.g. the disaggregation of existing polygonal data. Various strategies have been used in DSM surveys to resolve the problem of insufficient site data; for example, the inclusion of data from outside the survey area by Thomas et al. (2018). However, this potentially comes with a cost in increased model effort and complexity. More importantly, such an approach is only possible if suitable data are available. Given the known paucity of site data across much of Australia, there is an obvious question on whether the absence of site data is an inherent practical limit as much as an operational limitation. Yes, more site data can be collected to improve models, but the history of operational DSM in Australia to date suggests that DSM projects typically do not collect large quantities of site data. After all, one of the key virtues of DSM widely promoted is the ability to model without investing significantly in data collection. This is particularly the situation for large area modelling exercises such as the Soil Grid of Australia. If this is the case, then how will the issue be resolved? To answer this question, a better understanding is required of where the data gaps are.

One of the largest data gaps in Australia concerns cropping lands. The nature and history of soil survey in most cultivated lands in Australia is such that the data was either collected pre-development or there was a specific intent to describe the natural state. There is a significant absence of data representing soil attributes of cropping lands in soil survey databases. Thus, the use of data from state and territory soil survey databases for DSM will invariably lead to difficulties in depiction of cropping lands, in particular, for dynamic soil attributes such as pH, organic carbon and nutrients. There are multiple possible solutions to this, one of which is the targeted collection/collation of data in cropping lands. The National Soil Strategy has identified this opportunity and is investigating options regarding the collation of the large body of private sector data collected in cropping lands. Will this be the solution for representation of soil attributes in cropping lands via conventional or DSM means? Will it provide the feedstock for a new realm of paddock-scale DSM? There are many issues of data quality, sampling, bias, incomplete attribution etc. that make this an aspirational outcome rather than an easy, realistic one. It would be unfortunate if the very limited soil science capacity in Australia started chasing this new horizon rather than dealing with known problems with easy solutions.

Early exploratory DSM works in Australia put the soil surveyor’s knowledge at the centre of the equation with attempts to capture soil surveyor knowledge explicitly through rule-based frameworks. The need for pedological expertise in DSM has long been argued (Walter et al. 2006) and most operational DSM projects in Australia have involved traditionally trained soil surveyors. In the past 10–15 years, there has been a strong focus on machine-based approaches and reduced input of expert knowledge. The recognition of the value of pedological knowledge in DSM has, however, been reinvigorated in recent literature (Holmes et al. 2015; Thomas et al. 2018; Arrouays et al. 2020a, 2020b), although the reasons suggested for the use of that knowledge are varied. One reason may be a recognition that the machine does not have all the answers.

Rossiter (2018) warned that the combination of computing power and increasing data availability enables pedology-free inference of soil and attribute distribution. Biggs et al. (2018) provided a similar caution in their discussion of the current state of pedology in Australia, and suggested that there is a risk that the increased adoption of DSM will in fact lead to the decline of skills and knowledge within the pedological and soil survey community. There is a risk that DSM methods will lead to a perception that soil scientists are not needed and it only requires a ‘modelling expert’ to make a soil map. It is ironic that at a time when universities around Australia are struggling to maintain the teaching of soil science (Rogers et al. 2020), elements of the soil science profession have developed a methodology that may remove a potential career pathway for soil science graduates. A ‘modelling expert’ is not necessarily a soil scientist.

The renewed recognition of the role of experts in soil survey has unfortunately come too late. The retirement of the ‘baby boomer’ generation has led to a large reduction in the number of skilled pedologists/soil surveyors in the country; noting that these people gained their expertise via decades of conventional survey. Given that DSM inherently contains a paradigm of less fieldwork than conventional survey, its promotion and use in the past 20 years has led to a reduction in the pedological skills of newer generations. The supply and training of new pedologists is a larger issue than just the question of survey method, but it is indisputable that the skill of a soil surveyor/pedologist is inherently related to time spent in the field, and that DSM involves a paradigm of less fieldwork. There is consequently a real risk that in the future, soil survey (in the form of DSM) will more and more become the domain of desktop theorists with little real field experience.

In their pedometrics timeline, McBratney et al. (2019) described a number of stages in the evolution of pedometrics and DSM, and proposed that we are now in the ‘Global Mapping’ era. We posit that perhaps this should be re-titled the ‘coming down to earth’ era. Apart from the matters we have discussed above, this new era would involve acknowledgement that DSM is not the universal panacea to deriving soil data across the country. More importantly, it would involve critical analysis of the risks and potential perverse outcomes associated with the over-promotion and inappropriate use of DSM. The view promoted in the literature to date has been entirely ‘positive’ and there has been a lack of balanced evaluation.

There are some hints that this recognition is occurring; for example, both Arrouays et al. (2020a) and Ma et al. (2019) have acknowledged some of the problems associated with DSM. A more critical approach to determining the value of DSM will help identify where it fits best and where the greatest value may be obtained from the methodology in conjunction with existing methods. Recognition of the value of hybrid approaches is nothing new; it was suggested by McKenzie and Grundy (2008) and is implicit in the framework set out by Slater and Grundy (1999). Hybrid approaches do not, however, feature from an operational perspective. Divergence between academic theory and operational reality is a common thread in the DSM world. Progress would perhaps be measured by a decrease in the frequency of use of the statement ‘DSM has the potential to….’ and its replacement with ‘DSM can …’ Importantly, we would see a reduction in DSM projects that yield results that do not meet mapping standards or last the test of time.

We indicated earlier, as have authors such as Ma et al. (2019) and Kerry and Oliver (2011), that one of the key potential values of pedometric methods is the identification of relationships between soil attributes and pedogenetic factors. Knowledge of these relationships is of significant value for many purposes, irrespective of whether they are spatially depicted. Knowledge transcends scale. Investigating these relationships in greater detail should be a priority activity in Australia, as the knowledge generated is applicable at many scales and to many purposes, one of which is DSM. The work of Malone and Searle (2021) in considering relationships between field texture and particle size analysis is a recent example of this type of analysis. Importantly, such knowledge is useful at the paddock/farm scale without spatial depiction and is likely to be of value now, compared to the unrealistic goal of deriving paddock scale spatial soil attribute data.

Moving forwards, a more balanced approach to the merits and use of DSM (or not) must be embraced. The concerns of soil surveyors must be heard and understood by DSM practitioners. Arrouays et al. (2020a) commented that it is expected that conventional surveyors will oppose quantitative algorithmic approaches. Kidd et al. (2020) described ‘barriers’ to the adoption of DSM in Australia. Such a depiction of conventional soil surveyors is by no means beneficial and presumes that the questions and concerns raised by them are inconsequential, rather than valid questions of scientific rigour. Practising soil surveyors (such as the authors) are not against new methods or technologies, provided the public and scientific benefit is clearly demonstrated. This has been well illustrated by the early adoption of a variety of technologies and associated method changes in conventional soil survey in the past 30 years; e.g. GIS, GPS, digital cameras, digital field devices etc. The DSM discipline has not sufficiently and objectively demonstrated the scientific or public benefit associated with the approach, which is why there is ‘resistance’ amongst parts of the soil survey community. The soil survey community is quite rightly sceptical of blackbox approaches, which ironically suffer from many of the same problems that have been ascribed to conventional survey: the output has been derived using opaque, inefficient processes that cannot be replicated.

Authors such as Bui et al. (2020) have argued that soil surveyors must become geostatisticians. We suggest otherwise. The typical soil surveyor is not a geostatistician or mathematician and should never be. Their core skill is, and should remain, pedology. Otherwise, they cannot call themselves a soil surveyor. Their skill lies in their field craft and ability to observe and understand landscapes and importantly, communicate that knowledge to a wide range of audiences through a variety of pathways. As discussed earlier, DSM methods do not lead to extensive training of field pedologists and this problem must be recognised and dealt with. The future era must ensure that public investment in soil survey leads to the creation of soil surveyors/pedologists and primary data to service future questions, rather than the extinction of the discipline.

Over the past 25 years, considerable energy, time and money has been devoted to the research, development and operational implementation of DSM in Australia. Has it been value for money? Have the outcomes and outputs been better/greater than if the same time and money was invested in conventional survey? Will it be value for money into the future? An objective view would suggest the answer is unclear, which indicates that DSM has not yet proven its worth. More than two decades have been spent chasing the ‘new horizon’. While there are some obvious outcomes, history has shown that many of the outputs have not withstood the test of time; a clear difference and benefit compared to conventional soil survey products. It is time for a more objective analysis and review of the merits (or not) of DSM and where it is (or is not) appropriate to use in Australia. This must be coupled with a strategic review of where the pedometric toolkit can answer the most important soil-related questions and where/how the significant data gaps in the nation are most efficiently filled. The National Soil Strategy offers a potential pathway to achieve these outcomes, but to do so requires a more balanced view of technical methods and standards by our profession.

There were no specific data utilised in this study.

The authors declare no conflicts of interest.

This research did not receive any specific funding.