TERN’s SoilDataFederator makes multiple, disparate sources of soil data available ‘on the fly’ in consistent, easy to use formats. The SoilDataFederator significantly eases access to soil data created by different agencies and enhances Australia’s ability to use the data for understanding and overseeing managed and natural landscapes.
Soil is a critical component of all ecosystems. Knowledge and understanding of its function and properties are vitally important in ecosystem management.
Australia is fortunate to have many soil observations and measurements publicly available for use in an ever-increasing range of applications. These data are collected and managed by a broad range of custodians across the country, including TERN and its partners. But most of the time, custodians collect the data for their own specific business purposes and make them accessible via a range of dissimilar portals.
Until now, individuals wanting to bring soil data together in a unified way had to source data from each of the individual custodians, on a case-by-case basis, and transform them to a useful form for their specific application. A challenging process for most ecosystem scientists.
There are several ways to solve this issue and unify data, from the creation of a centralised behemoth database through to the case-by-case collation of datasets. The new TERN SoilDataFederator web API uses a federation approach to data unification, where data are managed by custodians but brought together and made accessible on the fly in a consistent form.
A conceptual diagram of TERN’s SoilDataFederator web API that makes multiple, disparate sources of soil data available ‘on the fly’ in consistent, easy to use formats.
Consistent soil data quickly accessible
The SoilDataFederator is a web app (API) that uses the R programming language. The API is used to query data over the internet via a standardised set of URLs with standardised parameters. Data can be returned in a range of formats but always in a standard form optimised for delivering data on a per attribute basis.
The SoilDataFederator makes it relatively simple and efficient to develop programmatic workflows to interrogate and retrieve data from various unrelated data sources as needed. It significantly eases access to soil data and enhances our ability to use these data for understanding and managing ecosystems
“The Federator is invaluable because it resolves teething issues of collating data sets from disparate sources manually – the Federator does all this automatically. This saves a massive amount of time.
The live link to each of the various data sources also ensures we can use the most complete sets of data that are available at the time. This remarkable ability improves our work into the future to improve and update soil mapping outputs in a more systematic way.
We are immensely privileged to have an engaged community of soil scientists and organisations (governments and universities) throughout Australia that supports and enhances the Federator by the action of making available their soils datasets.
This gives us greater confidence in delivering high quality soil maps and provides us with a more complete and data informed understanding of the status of our soils.”
Dr Brendan Malone, Senior Research Scientist, Soil Processes & Function / Agriculture & Food, CSIRO
