Sentinel-1, as part of Copernicus program, provides systematic and long term SAR data serie. Processing these data could be tricky for users more focused on applications than on data pre-processing. Automatic generation of Analysis Ready Data (ARD) is required to address more and more users of Sentinel-1 data and to develop new applications.

For facilitate the use of Sentinel-1, we have developed the S1Tiling processing chain to generate automatically customized Sentinel-1 ARD temporal series. S1Tiling is used to build time series of calibrated, orthorectified, and speckle filtered Sentinel-1 data, on a user defined set of tiles, compatible with Sentinel-2 images coverage (MGRS tiles). By this way, it makes it easier the use of both Sentinel-1 and Sentinel-2 on the same application.

As inputs of the S1Tiling processing chain, the user define a request using some specific keywords such as the geographic area of interest (a list of 120x120 km MGRS tiles), the first and the last dates of the time serie, the processing to do perform to generate ARD (downloading, data masking, speckle filtering, …).
After analysing the user request, the software performs automatically all operations to generate ARD. The processing includes a set operations executed by the software:
- Data downloading: The source of data is PEPS platform (https://peps.cnes.fr), provided by CNES. Based on the user request, S1Tiling downloads all missing data on the user computer.
- Data calibration: Once the downloading is complete, the software proceeds to the data calibration (gamma0 or sigma0) and thermal noise removal.
- Orthorectification: Calibrated data are then orthorectified on 120x120 km MGRS tiles with a 10mx10m pixel size (customizable)
- Concatenation: Images from the same date and orbit are concatenated on tiles.
- Speckle filtering: A multichannel speckle filter is implemented into the processing chain, allowing users to access directly to filtered images. Other kinds of filtering can also be selected.
- Ancillary data generation: mask on data and local incidence angle map can also be generated during the processing

At the end of the processing, the user obtains a set of time series over MGRS tiles, ready to use for his own applications.

S1Tiling processing chain can be used on a large kind of platforms, from laptop to high performance computing cluster. Based on multi-level parallelism (multithreading, multiprocessing and clustering), it is able to process efficiently a large number of images, reducing the computation time for the user.

Initially developed for internal use, S1Tiling processing chain has been opened to the EO community. S1Tiling is now freely available (http://tully.ups-tlse.fr/koleckt/s1tiling) as an open-source software and can be used for a wide kind of applications. It is based on Python and CNES OrfeoToolBox (https://www.orfeo-toolbox.org/)

S1Tiling is now used by researchers working on many applications with Sentinel-1 data such as rice monitoring, forest degradation monitoring, agriculture, water management or archeology. Generated data are also directly compatible with OpenDatacube (https://www.opendatacube.org) as associated metadata can also be generated as well by the processing chain. As an example, S1Tiling is currently used to provide Sentinel-1 ARD to the Vietnam Datacube. Other examples of S1Tiling applications will be presented during the LPS 2019.

Area B - Board 190

The launch of the various Sentinel satellites by the European Space Agency (ESA) as part of the European Commission’s Copernicus Programme is generating ever-increasing interest across governments and different market sectors, each with differing user requirements. It becomes quickly apparent that it is simply not technically feasible or financially affordable to consider traditional methods of storing, handling and manipulating EO data. Local processing and data distribution methods currently exploited by industry and government are not suitable to address the challenge of scalability, increases in the size of data volumes, and the growing complexities in the preparation, handling, storage, and analysis required to meet user requirements. To allow immediate analysis of the data without additional significant user effort, these barriers need to be addressed.

To access the valuable information contained within EO data, users are required to undertake a series of complex pre-processing steps to turn the data from a ‘raw’ unprocessed format into a state that can be analysed. Unless the user has the expertise, software and infrastructure to handle and process this information, efficient exploitation of the data is not realised.

A solution to this, in part, is through the systematic and regular provision of Analysis Ready Data (ARD). According to the Committee on Earth Observation Satellites (CEOS), this is defined as ‘satellite data that have been processed to a minimum set of requirements and organised into a form that allows immediate analysis without additional user effort and interoperability with other datasets both through time and space’.

Radar data and more specifically Sentinel-1 is becoming popular, as many regions have issues with cloud cover and would like increased temporal sampling for applications such as change detection and coastal monitoring. Synthetic Aperture Radar (SAR) has the advantage of operating at wavelengths not impeded by cloud cover or a lack of illumination and can acquire data over a site during day or night time under all weather conditions.

Non-SAR experts often do not have the required experience or compute power for pre-processing SAR data or using it for the development of applications and products. The aim of this talk is to demonstrate how to add significant value by routinely producing and offering ARD Sentinel-1 and ingesting this data into data cubes, as exemplified by the IPP CommonSensing project. CommonSensing is a large collaborative project to support three Commonwealth nations (Fiji, Vanuatu and Solomon Islands) particularly exposed to climate change -whereby Sentinel-1 data will be used to monitor coastal erosion, flooding and disaster monitoring through the exploitation of an innovative ARD framework for Sentinel-1.

Area B - Board 191

Synthetic Aperture Radar (SAR) data provides different and complementary information to the more common optical remote sensing data, however they are not used as extensively or operationally as optical data. Reasons for this have included the traditionally high cost of acquisition and the relatively complex and specialised processing methods for SAR data. The release of freely available European Copernicus programme data, especially the routinely acquired global coverage of Sentinel-1 SAR data through the European Space Agency (ESA), has opened up opportunities for greater exploration and application of SAR data in Australia.

Radar backscatter response is a function of topography, land cover structure, orientation and moisture characteristics – including vegetation biomass – and the radar signal is able to penetrate clouds, providing information about the earth’s surface where optical sensors cannot. The Sentinel-1A and 1B SAR satellites have been operating since 2014 and 2016 respectively, and have been building up an archive of dual polarised C-band data, including extensive, wall-to-wall acquisitions over Australia since December 2016 at the spatial resolution of ~3m x 22m in the default acquisition mode of Interferometric Wide (IW) swath.

Digital Earth Australia (DEA) is an analysis platform for observations of all forms, but particularly those captured from satellites which have unique potential and pose particular challenges for their full exploitation. DEA uses images and information recorded by satellites orbiting our planet to detect physical changes across Australia in unprecedented detail. DEA was originally built upon the extensive Landsat archive processed into an Analysis Ready Data (ARD) product (including atmospheric correction to surface reflectance, co-registration, and associated cloud/cloud shadow masks), and is being developed to feature other satellite datasets including SAR. A research alliance between the Commonwealth Scientific and Industrial Research Organisation (CSIRO) and Geoscience Australia (GA) was established in relation to Digital Earth Australia, to develop a SAR-enabled Data Cube capability for Australia (referred to as the Australian SAR Data Cube project).

Open Data Cube (ODC) platforms enable easier access to satellite ARD, as they remove the need for the user to pre-process Earth observation datasets, and provide access to archived remotely sensed data in a format ready for use. The definition of ARD with respect to SAR data has been an area of discussion for many years and is being actively developed through the CEOS Analysis Ready Data for Land (CARD4L) framework. The Australian SAR Data Cube project has been utilising this information in developing SAR ARD products for Australia. Initial SAR ARD products include radar backscatter (gamma nought ϒ°), eigenvector-based dual-polarisation decomposition (Entropy, Anisotropy, and Alpha channels) and Interferometric coherence, all generated from the Sentinel-1 IW swath mode data available on the Australian Copernicus Data Hub.

The processing workflow for producing these SAR ARD products makes use of ESA’s free Sentinel-1 Toolbox within the Sentinels Application Platform (SNAP), as well as the proprietary software GAMMA. SNAP is an open source platform, allowing easy access and sharing of processing workflows with the capability of batch processing through its Graph Processing Tool (GPT). A similar process is run in parallel using GAMMA for comparison and assessment of the SAR ARD product quality.

This presentation will describe the SAR ARD products developed for the Australian SAR Data Cube project and the processing workflows used to produce them. Some examples of data cube applications will also be shown.

Area B - Board 192

Multi-decadal archives of satellite imagery and rapidly growing volumes of newly acquired Earth observation (EO) data provide the opportunity to monitor and assess changes to the Earth’s surface in a systematic way. However, many satellite data users need to invest substantial time and effort into data preparation that is typically done in desktop environments, which limits effective utilisation and subsequently the societal benefit that could be realised from the datasets.

There is a growing trend among satellite data providers to handle more of such data preparation work and provide Analysis Ready Data (ARD) as a standard product package for EO data to make it more accessible and useful to a wider user base. ARD are satellite data that have been processed to a minimum set of requirements and organised into a form that allows immediate analysis with a minimum of additional user effort and interoperability both through time and with other datasets. Although definitive specifications for different families of ARD products are currently being developed by Committee on Earth Observation Satellites (CEOS), through the CEOS Analysis Ready Data for Land (CARD4L) framework, the effective utilisation and interoperability of cross-provider and cross-sensor ARD products may still pose challenges.

The interoperability between ARD products will be critical for the effective user uptake of the multiple data streams. Identifying the common set of properties required for multi-sensor interoperability and assessment of the impact of parameters used in the corrections applied to derive ARD products, are important steps in achieving interoperability between ARD products. Multiple aerosol and water vapour parameters covering a range of conditions, a range of BRDF shape parameters from typical land cover types, and different solar angle normalisations covering seasonal variations have been evaluated for their impact on the derived surface reflectance, an ARD product. The results from the sensitivity analyses will be highlighted in this paper.

Area B - Board 193

The Earth Observation (EO) user community always relied on analysis ready data to feed their various applications, even before the term ARD was borne. This requires data that is discoverable, easily accessible, inter-operable, and reusable (FAIR principles). However, the EO community is exposed to rapid changes in data provision and analytical possibilities due to the shift towards Big Data delivered by satellite systems as well as data hungry artificial intelligence (AI) approaches. This poses challenges for data acquisition and analysis to the EO community, as data is provided in high velocity as well as volume by a multitude of platforms. In addition to volume and velocity, an immense variety in products in different formats that are delivered in various geospatial and temporal resolutions complicates data acquisition for the EO data user community even further.
The CEOS CARD4L initiative defines requirements for fairly provided ARD data that will facilitate the accessibility of data. Practical ARD solutions attempt to address these issues by providing digested, gridded information in the form of data cubes or similar structures. In the ESA project Earth System Data Laboratory (aka Earth System Data Cube, ESDC), for example, we provide 83 geophysical variables based on EO and dedicated to the Earth System Science community, in data cubes in a harmonised manner in three spatial resolutions: 1/12°, 1/4°, and 1°. ESDC has a continuously growing user community with currently more than 100 registered users.
ESDL is based, like the CEOS OpenDataCube, on the powerful Python framework xarray, which is designed to efficiently work on N-dimensional labelled arrays even if their size surpasses the systems memory by orders of magnitude Currently we are further developing this technology in a European H2020 project, called Data Cube Service for Copernicus, DCS4COP. The Data cube concept builds the technological backbone for a commercial service model which comprises open source software, quality labelled data layers, software development services, education and training. It aims at Intermediate Business Users and uses ARD to reveal EO data to new user communities.
The issue of uncertainties in measurements, to date, is of high importance in CalVal, and become more and more available in satellite data products generated by the space agencies. However, “uncertainties” are in a rather juvenile state in the context of ARD. We believe that it is of high importance to close this gap in order to meet user’s requirements and to ensure the acceptance of ARD also for scientifically ambitious applications. Each cell of a grid has been exposed to a sequence of algorithms in order to estimate the value of a variable for a particular grid cell. These algorithms comprise, among others, interpolation, projection, and aggregation, each of which will add to the overall uncertainty of a cell estimator. However, uncertainties are rarely indicated. In the contrary, the above mentioned CARD4L ARD product specification for instance defines “precision” as an important but not required information for ARD data due to the lack of availability of this information upstream. Knowing that analytical approaches like AI or OE benefit considerably from including uncertainties, and in view of the change in availability of uncertainties in Level 2 products, we believe this statement should be revised. Providing information about uncertainty seems a fundamental data enrichment for successful data interaction and analysis and is thus a worthwhile value-adding step. The GUM framework (Guide to Uncertainties in Measurements) is actually applicable to ARD, although it may be technologically challenging in some respects.
Combining data from disparate sources is essential for research and end user communities. Gridded EO data will often be combined with discrete in-situ measurements. At this stage, in-situ and ARD data are held in disparate data stores. The first step of combining ARD and in situ data is querying relevant information from a data cube. These operations are time as well as resource consuming. In the DCS4COP project, we will present solutions and future work that allow query operations on data cubes including select for regions using polygons that combine information from different data cubes ‘on the fly’.
“As a service” (aas) and NoSQL enter the science world. Who has not experienced the awe of adding a variable to an existing data schema including the challenge to obtain data from a often complicated data structure. NoSQL as well as small aas services allow to simplify the process of data acquisition and storage. We will demonstrate data structures and NoSQL powered micro services that significantly simplify in-situ data acquisition as well as combing these data with gridded ARD cube information.
’We will discuss here our service provider’s view on current state of the art and limitations of ARD applications, based on the practical examples of the Earth System Data Laboratory, ESDL, and the Data Cube Service for Copernicus, DCS4COP. We will conclude with an overview of current trends and give an outlook on future developments and challenges to which the ARD provider community will have to find solutions in the future to keep thriving.

Area B - Board 194

The emergence of Analysis Ready Data represents a paradigm shift in Earth Observation, which relieves the end user from the most data intensive preprocessing steps. A key feature of ARD is the provision of quality flags that indicate non-target features, which should be removed prior to any analysis. In this regard, cloud identification is a core element that needs to be addressed during ARD generation.
Sentinel-2 data offer an enormous potential for environmental monitoring due to the unprecedented spatial, temporal and spectral resolution, global coverage and free data access. However, cloud detection is severely limited by the unavailability of a thermal band that has proven to be effective for similar medium resolution data like Landsat. Spectral information alone is insufficient to separate clouds from other bright targets like built-up structures, and even the designated cirrus band is only effective for high altitude clouds. Low altitude clouds are indiscriminable as the cirrus band is located in a strong water absorption band, which only observes the upper layer of the atmosphere.
Sentinel-2A’s Multi Spectral Instrument (MSI) is a push-broom sensor with 13 spectral bands that cover the VIS, NIR and SWIR domains. The MSI is characterized by a complex sensor arrangement: for each band, twelve detectors are arranged in a staggered configuration to cover the wide FOV. As a result, a parallax exists between odd and even detectors – and although less pronounced, there is also a parallax between bands. In this regard, the NIR plateau is of special interest as there are three spectrally highly correlated bands available, which partially overlap (bands 7, 8 and 8A), and which look in different directions. While objects on the land surface are registered to the same position, objects above the land surface are projected onto slightly different locations in the focal plane, thus clouds are observed under a parallax.
For enhanced cloud detection, we have developed the Cloud Displacement Index (CDI), which makes use of the three highly correlated near infrared bands that are observed with different view angles. We compared the CDI with the currently used cloud probabilities in the Fmask algorithm. Fmask is the state-of-the-art algorithm for Landsat data, and was updated for Sentinel-2, too. Findings indicate that the CDI is superior in separating clouds from bright land surfaces as it not only relies on spectral features but makes uses of additional view geometry information. Thus, it fully compensates for a missing thermal band. Based on these results, we have proposed an integration of the CDI into the Fmask algorithm, and a global validation over 20 metropolitan areas indicates that the new approach results in improved cloud detection accuracies (Accuracy 94%), thus enabling a reliable separation of clouds from bright ground objects.

Area B - Board 195

The concept of CEOS Analysis Ready Data for Land (CARD4L) refer to products that have been processed both to a minimum set of requirements, and in a data format that facilitates user uptake in applications with minimum effort on the side of the end-users. An objective of CARD4L products are to be accessible to a wide range of users, and flexible for a wide range of applications. Each prospective CARD4L product is to be assessed by the agency responsible for production and distribution as to whether it meets CARD4L requirements.
One of the Product Family Specifications (PFS) is for surface temperature observed from multispectral sensors operating in the thermal infra-red (TIR) wavelengths. These can have spatial resolutions <100m for Landsat and ASTER; around 1 km for many of the operational sensors, such as Sentinel-3, MODIS, and AVHRR; and >3 km for instruments in geostationary orbit. The principle end-user product of relevance here is land surface temperature (LST).
Here we present a self-assessment of the operational Land Surface Temperature (LST) product from Sentinel-3 (SL_2_LST) carried out with respect to CARD4L-ST PFS Version 3.1. For each item in the PFS we assess whether the SL_2_LST is compliant, partially compliant or not compliant at both the threshold and target levels. We also identify where the PFS is not in line with community established protocols or standards for the product family in question. These self-assessment activities enable user experience to feed directly into specifications process including where data providers are able / not able to undertake the necessary changes to meet requirements. This will also help guide whether current specifications follow the appropriate approach.

Area B - Board 196

Enabling robust, defensible and integrated decision making in the Era of Big Earth Data requires the fusion of data from multiple and diverse sensor platforms and networks. While the application of standardised global grid systems provides a common spatial analytics framework that facilitates the computationally efficient and statistically valid integration and analysis of these various data sources across multiple scales, there remains the challenge of sensor equivalency; particularly when combining data from different earth observation satellite sensors (e.g. combining Landsat and Sentinel-2 observations). To realise the vision of a sensor ignorant analytics platform for earth observation we require automation of spectral matching across the available sensors. Ultimately, the aim is to remove the requirement for the user to possess any sensor knowledge in order to undertake analysis. Here we introduce the concept of spectral equivalence and propose a methodology through which equivalent bands may be sourced from a set of potential target sensors through application of equivalence metrics and thresholds. A number of parameters can be used to determine whether a pair of spectra are equivalent for the purposes of analysis. A baseline set of thresholds for these parameters and how to apply them systematically to enable relation of spectral bands amongst numerous different sensors is proposed. The base unit for comparison in this work is the relative spectral response. From this input, determination of a what may constitute equivalence can be related by a user, based on their own conceptualisation of equivalence.

Area B - Board 197