Proximal and remote sensing – what makes the best farm digital soil maps?

Introduction

Digital soil mapping (DSM) has been gaining in popularity over the past few decades, and this has been driven by the abundance of spatial datasets and increased computing power and data analytical techniques now available. There have been a large number of reviews written on DSM in recent years (e.g. McBratney et al. 2019; Searle et al. 2021), but it is an area of research that still has many challenges to overcome for the products to provide genuine value to a range of stakeholders. The majority of DSM studies are conducted across large areas, with many researchers often trying to map as large an area as possible (Grunwald et al. 2011; Arrouays et al. 2017). While these studies often predict at fine spatial resolutions (e.g. 30 m) over large extents, a problem is that the maps do not represent fine-scale variability well, such as the variation within an agricultural field. While many DSM studies promote the importance of the products for farmers, many farmers do not know they exist; or if they do, they may not trust the quality of the predictions. A study by Han et al. (2022) demonstrated how poor global, national, and state DSM products were at representing within-farm variability across a collection of diverse cropping fields and farms in Australia.

Creating bespoke DSM products for farmers at the field extent is less common, primarily due to the large expense of collecting samples (this varies by region/country), and the lack of skilled practitioners. Mapping soil within-fields is usually a step towards implementing management practices using precision agriculture principles. One common approach is to implement a grid sampling approach and use a simple interpolation technique such as inverse distance weighting (IDW) to create spatial maps.

However, to capture the true scale of variation in soil across space data often needs to be collected on dense grids, which can be a prohibitively expensive task (Kerry et al. 2010). Another common approach is to collect proximally-sensed data (e.g. from an electromagnetic induction sensor) and then strategically sample soil based on this information. This can be a more cost-effective approach as fewer soil samples are often required to cover the extent of variability compared to standard grid sampling (Kerry et al. 2010). When this latter approach is adopted, operators generally use simple approaches, such as using a single spatial variable to create models and maps. Nonetheless, cost can still be a limitation in these scenarios. A typical farmer in Australia can often have more than 10 fields, and having enough soil observations per field to create a simple linear model is generally a challenge. Many commercial operators in Australia work off a sampling density of one sample per 50–100 ha as this is something that can be realistically implemented (SPAA 2022).

In Australia, over half of the continent is used for agriculture. So creating useful and affordable soil maps is an important step to help farmers in improving the management of our soils. Given the issues raised above, a promising approach is to create bespoke soil maps for whole farms, as opposed to individual fields. While traditional precision agriculture focused on single fields in isolation, there has recently been a shift to combining data from multiple fields for analysis (Filippi et al. 2019a). This is due to a few reasons, such as the high cost of sampling and analysing soil, and the fact that more data can be utilised as it is from multiple fields. However this presents some challenges, for example, differences in management practices between fields (e.g. crop rotations) can result in differences in the state of soil properties (e.g. moisture), which can then impact on the data collected by proximal sensors. This has the potential to impact the value and usefulness of proximal sensing when modelling and mapping across multiple fields.

Proximal sensing is described as using a sensor in contact, or within 2 m of the soil, whereas remote sensing is described as using a sensor at least 2 m from the soil (Tilse et al. 2023). There are different components of proximal sensing, but for the purpose of this paper, we refer to sensors that are best described as on-the-go proximal sensors. This includes sensors such as electromagnetic induction (EM), or gamma radiometric sensors which are typically mounted to a ground-based vehicle and measurements are taken and recorded as the vehicle passes over fields. This is opposed to handheld, point-based proximal sensors such as visible near-infrared sensors. A more detailed description of these different types of technologies can be found in Tilse et al. (2023).

Using on-the-go proximal sensing data has been seen as the gold standard for DSM at the smaller scale. However, there is now an abundance of data collected by various remote sensors that can represent the within-field variability of soil. For example, in Australia there is now public access to airborne gamma radiometrics data (Minty et al. 2009), elevation and terrain attributes (CSIRO 2023), bare earth (BE) imagery (Roberts et al. 2019), and imagery from a range of satellites such as Landsat and Sentinel (Gorelick et al. 2017). While combining proximal data with remotely-sensed data is not a new concept, there is often a view by the industry (e.g. commercial service providers) that if proximal sensing data is present, there is limited value in the addition of remote sensing data. However, variation in soil properties is driven by a number of different factors and their interactions, and these proximal and remote sensing variables can provide surrogates that represent these.

This study aims to assess the value of different proximal and remote sensing data individually, as well as the combination of the two for mapping three different important soil properties (organic carbon, clay, and pH). This is done across multiple fields for three farms in different biogeographical locations across Australia, and in the topsoil (0–10 cm) and subsoil (30–60 cm). This could guide decisions about whether or not growers should invest in collecting proximally-sensed data. This study aims to demonstrate a robust, realistic, and cost-effective approach to mapping soil properties at the whole-farm scale for farmers and land managers which could be valuable in informing management decisions.

Materials and methods

Study sites and soil datasets

Three different farm sites were used in this study: (1) a farm in the wheatbelt of Western Australia, ‘West Farm’ (7200 ha); (2) a farm in northern NSW, ‘North Farm’ (4900 ha); and (3) a farm in southern NSW, ‘South Farm’ (2000 ha) (Fig. 1). West Farm has a mix of soil types, typically of sandier texture and gravel layers; North Farm is characterised by uniformly textured clay soils; and South Farm has a mix of duplex and gradational textured soil types. Soil cores were extracted and sub-sampled at 0–10 cm and 30–60 cm depths.

Fig. 1.

Location of the three study farms across Australia.

Soil pH (1:5 soil:H₂0) was measured by a pH meter with an ion-selective electrode. Organic carbon was measured by the combustion method as no carbonates were present in any samples. Soil carbon data was only available for the topsoil. Soil clay content was assessed by the hydrometer method. These particular soil properties were chosen as they contribute to important characteristics of soil, such as water holding capacity, chemical constraints, and nutrient availability, and are of key interest to farmers.

In terms of median pH values, North Farm had the highest topsoil pH, as well as alkaline subsoils (Fig. 2). South Farm was characterised by a more acidic topsoil and an alkaline subsoil. West Farm had consistent neutral median pH values, but with quite a bit of variation in the subsoil. Clay content was highest overall for South Farm, followed by North Farm and then West Farm. South Farm had the highest median topsoil organic carbon content, which was then followed by West Farm and then North Farm.

Fig. 2.

Summary statistics of the soil laboratory analysis for pH, clay content, and organic carbon content across the three study farms and two sampled depths.

The number of sampling sites where a soil core was extracted for each farm varied from 22 to 91, relative to the size of the farm (Table 1). In Australia, soil sampling and analysis is expensive, and realistically growers and land managers are restricted by the cost. Although a larger number of samples would likely produce more accurate predictions, the reality is that this is not economically feasible. The sampling density for this study ranges from one sample per ~80–100 ha.

Table 1.The number of sites and proximally-sensed data available at each farm.

Farm	Farm size (ha)	No. of sites	Number of fields	Proximal electromagnetic induction available	Proximal gamma available
West Farm	7200	91	52	50 cm, 150 cm	K, Th, U, TC
North Farm	4900	48	5	50 cm, 150 cm	K
South Farm	2000	22	4	50 cm, 150 cm	K

K, potassium; Th, thorium; U, uranium; TC, total count.

Proximally-sensed data

A proximal soil sensing survey was conducted to collect high-resolution apparent soil electrical conductivity (ECa) and gamma radiometrics data. The proximal soil sensing survey was conducted on 24 m swaths, and the position was recorded with differential GPS (DGPS) equipment. Continuous surface layers were obtained by kriging with local variograms onto a standard 10-m grid through the software R (R Core Team 2020).

Electromagnetic induction (EM)

Apparent soil electrical conductivity (ECa) was measured via EM using a DUALEM-21S instrument (Dualem Inc., Milton, Ontario, Canada). Information representing depths to 50 cm and 150 cm of the profile was used for all farms in this study (Fig. 3).

Fig. 3.

Maps of a subset of covariates for North Farm. PS, proximally-sensed; RS, remotely-sensed; BE, bare earth.

Gamma radiometrics

Gamma radiometric data was recorded using an RSX-1 gamma radiometric detector with a 4 L Sodium-Iodine crystal (Radiation Solutions Inc., Mississauga, Ontario, Canada). Data for potassium (K), thorium (Th), uranium (U), and total dose were available for West Farm, however, only K was available for both North Farm and South farm.

Results

Model validation statistics

Fig. 4 and Table 3 show the LCCC values from the LOSOCV procedure for all soil properties, farms, depths, and data scenarios. Overall, the average results across all three farms showed that the models using both proximal and remote sensing data resulted in the best LCCC values. This was the case for all three soil properties (organic carbon, clay, and pH) in both the topsoil and subsoil. Using remote sensing data only produced the next best model predictions on average. Based on this study, using proximally-sensed data only produced, on average, the poorest model predictions.

Fig. 4.

Distribution of Lin’s concordance correlation coefficient (LCCC) across different farms for all models using leave-one-site-out cross-validation (LOSOCV).

Table 3.Lin’s concordance correlation coefficient (LCCC) of all models using leave-one-site-out cross-validation (LOSOCV).

Soil property	Model	Depth (cm)	West Farm	North Farm	South Farm	Average
Organic carbon (%)	Proximal and remote	0–10	0.53	0.54	0.64	0.56
	Proximal only		0.16	0.29	0.13	0.20
	Remote only		0.53	0.54	0.62	0.55
Clay content (%)	Proximal and remote	0–10	0.61	0.63	0.78	0.68
	Proximal only		0.41	0.49	0.66	0.52
	Remote only		0.60	0.55	0.72	0.62
	Proximal and remote	30–60	0.38	0.34	0.53	0.42
	Proximal only		0.20	0.21	0.46	0.29
	Remote only		0.38	0.12	0.44	0.31
pH	Proximal and remote	0–10	0.32	0.37	0.65	0.45
	Proximal only		0.24	0.25	0.37	0.29
	Remote only		0.26	0.12	0.57	0.27
	Proximal and remote	30–60	0.72	0.53	0.28	0.51
	Proximal only		0.63	0.42	0.11	0.39
	Remote only		0.67	0.41	0.39	0.46

*LCCC values are ranked by colour. Green, best; Orange, middle; Yellow, worst.

When looking at individual farm results, a similar pattern in the LCCC value for the combined models being superior to the remote or proximal only models was observed on all farms (Table 3). However, in the performance assessment between remote or proximal only models. On North Farm, the proximal models for soil pH at both depths, and subsoil clay content out-performed the remote only models. On South Farm, a similar result was observed for the subsoil clay content.

Table 4 shows the corresponding RMSE values for all models. Overall, the results reveal a similar pattern to the LCCC analysis, with the average RMSE of the combined proximal and remote sensing models from the three farms always making the most accurate predictions (lowest RMSE) and the remote only models generally the next best method. Again, when looking at individual farm results, a similar pattern in the RMSE value for the combined models being superior to the remote or proximal only models was observed on all farms. The RMSE assessment between remote or proximal only models also showed that on North Farm, the proximal models for soil pH at both depths, and subsoil clay content out-performed the remote only models. On South Farm, the same result was observed for the subsoil clay content

Table 4.Root mean square error (RMSE) of all models using leave-one-site-out cross-validation.

Soil property	Model	Depth (cm)	West Farm	North Farm	South Farm	Average
Organic carbon (%)	Proximal and remote	0–10	0.29	0.11	0.28	0.23
	Proximal only		0.35	0.13	0.34	0.27
	Remote only		0.30	0.11	0.28	0.23
Clay content (%)	Proximal and remote	0–10	5.25	5.60	3.86	4.85
	Proximal only		5.96	6.18	4.81	5.65
	Remote only		5.23	5.99	4.49	5.24
	Proximal and remote	30–60	10.27	3.72	8.21	7.40
	Proximal only		10.94	3.75	8.53	7.74
	Remote only		10.31	3.97	8.94	7.73
pH	Proximal and remote	0–10	0.46	0.46	0.58	0.50
	Proximal only		0.47	0.47	0.70	0.55
	Remote only		0.47	0.51	0.66	0.55
	Proximal and remote	30–60	0.77	0.34	0.41	0.51
	Proximal only		0.86	0.37	0.44	0.56
	Remote only		0.82	0.37	0.41	0.56

*LCCC values are ranked by colour. Green, best; Orange, middle; Yellow, worst.

Discussion

It is not unexpected that using both proximal and remote sensing data for modelling soil properties was found to generally result in the best model predictions. Variation in soil is driven by a number of different factors and their interaction. This can be described by the Scorpan framework (McBratney et al. 2003), and several of these factors are represented by the covariates used in this study. For example, the NDVI variables represent crop biomass, and therefore the vegetation variable (organisms, o), the elevation data reflects the relief variable (r), and the gamma radiometrics data represents the parent material variable (p). It is therefore logical that the proximal and remote data scenario produces the best model predictions as there is a larger suite of variables that describe the variation of soil.

While proximal sensing data has been seen as the ‘gold standard’ for within-field digital soil mapping (SPAA 2022), there are some other important reasons that may explain why the proximal sensing data only scenario generally produced the poorest predictions in this study. Proximal sensing data, such as soil ECa from an EMI instrument can identify within-field soil type variability well, however, when aggregating several fields together there are some limitations. Differences in management between fields can result in considerable differences in some soil properties, such as the soil moisture levels. This can lead to stark differences in absolute ECa values between fields, which is not driven by changes in soil type. This exaggerated extent of variability across the site reduces its usefulness as a covariate for modelling other soil properties. Many remote sensing variables do not suffer from the same problems, for example, the sensing of elevation and airborne gamma radiometrics are largely unaffected by agricultural management. It would be expected that if the focus of the study was on mapping individual fields under these different data scenarios that the proximal sensing only scenario would produce improved results from those observed in the current study.

Overall, the quality of the model predictions in this study varies considerably (Fig. 3, Table 3), and this is expected due to the differences in soil properties, depths, farms, and data scenarios. Although direct comparisons to results from other studies must be done with caution, the LCCC values from the LOSOCV procedure were similar to other published studies for clay content (Zhao et al. 2022), pH (Filippi et al. 2019b), and organic carbon (Wang et al. 2022).

The proximally-sensed predictor that was commonly identified as an important predictor across various farms, soil properties, and depths was the gamma radiometrics potassium. Compared to EM instruments, ground-based gamma sensors are relatively rare. To the best of our knowledge, only a few sensors are operational in Australia. Gamma potassium is known to be related to variation in soil type, parent material, and texture (Reinhardt and Herrmann 2019), so it is logical that this was identified as an important predictor for soil pH, clay content, and organic carbon. It is also known to be less impacted by differences in soil moisture compared to EM instruments (Whelan and Taylor 2013), which could also explain why it was identified as important when mapping across multiple agricultural fields in this study. The soil ECa data was identified as important for models in the proximal and remote data scenario (seven of 15 models), and the proximal only data scenario (10 of 15 models). Soil ECa data collected from EM instruments is very common worldwide, and it is well known that this data can be correlated to soil texture, moisture content, and overall soil fertility when salinity is not a dominant issue (Whelan and Taylor 2013).

While EM and gamma radiometric proximal sensors were used in the study due to prevalence of the sensors and data availability, there are also other on-the-go proximal sensors available to use in digital soil mapping on-farm. In particular, Veris Technologies have a suite of soil sensors available that utilise electrochemical sensors for on-the-go pH measurement, and visible and near-infrared (Vis-NIR) sensors to infer carbon and organic matter content of soils (Viscarra Rossel and Lobsey 2016). In the current study, the results showed that the average predictions of topsoil carbon were poor for the proximal only data scenario (Fig. 4). It could be envisioned that including data from these on-the-go Vis-NIR sensors would improve these predictions. The inclusion of some of these other proximal sensors should be considered in future research.

The BE and NDVI variables derived from satellite imagery were the variables that stood out in terms of their inclusion in models. The different percentiles (5th, 50th, 95th) of NDVI from 2000 to 2020 used in this study represent long-term trends in crop biomass and therefore production. Variation in crop production is linked to soil variability, particularly the soil properties modelled in the current study, which are important drivers of production. The BE imagery product based on the Landsat 5, 7 and 8 satellites (Roberts et al. 2019) proved important for modelling soil pH, clay content, and organic carbon. In particular, the blue band and SWIR1 band were prevalent in many models. The SWIR1 band, which covers wavelengths 1.55–1.75 μm for Landsat 5 and 7, and 1.57–1.65 μm for Landsat 8, is useful in discriminating the moisture content, and therefore water holding capacity of topsoil (Tian and Philpot 2015). Moisture content can be related to other soil properties, such as clay content. This could explain why the SWIR1 band was included in 8 of the 12 soil clay content models for the proximal and remote, and remote only data scenarios (Tables 5 and 7). The blue band, which covers 0.45–0.52 μm for Landsat 5 and 7, and 0.45–0.51 μm for Landsat 8, is known to help distinguish soil from vegetation. However, it is likely that the blue band is picking up differences in soil colour, which can be well related to the soil properties modelled in this study (Zhang et al. 2023).

Airborne gamma is commonly used in many Australian DSM studies at the national (Viscarra Rossel et al. 2015), regional (Pozza et al. 2022), and farm level (Filippi et al. 2019b). This gamma data has proven useful in modelling the variation in several soil properties. However, this study shows that satellite-derived sensor data (from the visible and near-infrared parts of the electromagnetic spectrum) of crops/plants and soil are also highly valuable in representing within-field and farm variability of important soil properties.

The increasing quantity of remote sensing and satellite imagery presents an invaluable opportunity for digital soil assessment. The accessibility of this data is improving considerably with platforms such as Google Earth Engine (Gorelick et al. 2017), and the quality is also rising. For example, the Sentinel 2 satellites can capture imagery at 10–20 m resolution, and includes a suite of bands in crucial parts of the electromagnetic spectrum (e.g. red edge). While many DSM studies have used satellite imagery as a covariate, there can be considerable differences in the type of imagery and the processing implemented. For example, some studies use a single-day remotely-sensed satellite image (Mirzaee et al. 2016), whereas others use multitemporal satellite images (Pozza et al. 2022). Many things need to be considered when processing satellite data for use as a covariate. Some of the primary factors are the satellite system (e.g. Landsat 8 or Sentinel 2), the time period (e.g. single image or some statistic of a collection of images), and the band or spectral index to select. These decisions can be made based on the specific goals of the study, but can result in an overwhelming amount of options.

In contrast, the BE imagery (Roberts et al. 2019) is a downloadable product, and does not require the same processing and decisions as the NDVI variables used in this study. This makes it much easier to use, and also more reproducible and accessible. These freely available products present a great opportunity for creating farm-scale digital soil maps for consultants and service providers.

Conclusions

This study assessed the value of using proximal and remote sensing data for mapping three different soil properties (organic carbon, clay, and pH), at two depths, and across multiple fields for three farms in different biogeographical locations across Australia. Three different data scenarios were considered: (1) using proximal sensing data only; (2) using remote sensing data only; and (3) using a combination of proximal and remote sensing. The results showed that using a combination of both proximal and remote sensing data resulted in the best predictions. Using remote sensing data only generally led to better predictions than proximal sensing data only. One possible reason for this is that soil variation is driven by a number of factors (e.g. organisms/vegetation, relief, parent material), and there is a larger and more diverse suite of remotely-sensed variables that can be used to represent these factors. Another thing to consider is that proximal sensing data is often affected by differences in management between fields, and combining data across multiple fields as we have done in this study may impact the value of this. Nonetheless, it was found that the proximally-sensed gamma radiometrics potassium was the most widely used of all the available variables for the proximal and remote data scenario. The remote sensing variables based on satellite imagery (NDVI and BE) were the standout predictors for many of the models in the data scenarios that included remote sensing data. This demonstrates that remotely-sensed imagery of crop/plant biomass and bare soil can represent the variation in important soil properties well. In particular, the BE imagery product presents a great opportunity for improving farm-scale digital soil mapping in Australia as it is also free and easily accessible as a downloadable product. Overall, this study shows that combining freely available and fine-scale remotely-sensed products to proximally-sensed data leads to more accurate farm-scale soil property maps that could inform important management decisions.