The Committee for Earth Observing Satellites (CEOS), has identified the need for satellite data to be more analysis ready - especially for land imaging satellites. The rationale behind this is to enable users to access products that have been already processed to a certain level, and to attract interest from potential new user groups currently not engaged with Earth Observation (EO) data. Such data will allow users to carry out immediate analysis on it to address a variety of applications without the extra time and cost that is associated with additional data pre-processing.
CEOS has developed a definition and a framework for Analysis Ready Data (ARD), which has been well received by both providers and the user community. The CEOS Land Surface Imaging Virtual Constellation (LSI-VC) team has been leading this initiative and working in collaboration with the CEOS Working Group for Calibration and Validation (WGCV), domain experts, EO data users and providers to implement the CEOS Analysis Ready Data for Land (CARD4L) Framework.
The CARD4L Framework provides concepts and broad specifications for EO data to be analysis ready. The Framework contains three core elements; a) Definition, b) Product Family Specification (PFS), and c) Product Alignment Assessment (PAA). The expectation is that among the range of data products currently produced and distributed by data providers small adaptations will enable these to address the CARD4L specifications. The systematic and regular provision of CARD4L will significantly reduce the load on EO data users and, as a result, is expected to boost data use.
The CEOS community is now developing procedures to support adoption and implementation of the Framework. Work has been carried out on the development of PFSs across the optical, thermal, and radar domains as well as on the Product Alignment Assessment process. The aim of the proposed presentation is to give an overview on the CARD4L Framework, to introduce its elements, the processes and steps that EO data providers must undertake to obtain a CARD4L product, as well as the steps that CEOS LSI-VC foresees to fully develop the CARD4L initiative.

The already large data volumes from current EO missions will continue to grow rapidly with the next-generation satellites around the corner, which all tend to move towards wider swaths, higher spatial resolution, systematic wall-to-wall observation strategies, and in the case for SAR missions, new polarimetric and interferometric options. In parallel on the user side, applications are moving towards dense time-series analysis over regional to national to global scales. Users and producers alike, we all risk to drown in Data Heaven!

CARD4L (CEOS Analysis-Ready Data for Land) is a joint effort by the Committee on Earth Observation Satellites (CEOS) to address this (rather pleasant) Big Data challenge (http://ceos.org/ard/). Not only to streamline data flows and enable new storage and analysis solutions (e.g. data cubes), but specifically also to broaden the Earth Observation user community by provision of data products that do not require expert knowledge to ingest and analyse. This last point is perhaps particularly relevant for Synthetic Aperture Radar (SAR), where the potential to contribute to today’s great environmental challenges with unique information is significant, but with the SAR user community remaining small and expert-oriented even after 25 years of operational SAR missions. CARD4L is an opportunity to bridge that gap.

In a coordinated effort by the CEOS Land Surface Imaging Virtual Constellation (LSI-VC) and the CEOS Working Group on Calibration and Validation (WGCV) SAR Subgroup, a number of SAR-specific CARD4L Product Family Specifications (PFS) are currently under development:
• Radar Backscatter (close to completion)
• Polarimetric Radar Decomposition
• Normalised Radar Covariance Matrix
• Geocoded Single-Look Complex
• Geocoded Interferograms (wrapped/unwrapped)
• Interferometric Coherence

This presentation aims to provide an overview of the (six) SAR specifications identified to date, and to outline how space agencies and data providers – through the CARD4L Product Alignment Assessment Process – can assess how well specific (SAR) data products comply with the metadata and calibration criteria in the corresponding product specifications.

The Copernicus Sentinel-2 mission is nowadays fully operational, with the two satellites Sentinel-2A and Sentinel-2B down-streaming several terabytes of Earth Observation (EO) data every day, freely available to users. On top of the Level-1C (i.e. Top-Of-Atmosphere and orthorectified) products, also Level-2 (i.e. Surface Reflectance) products are released to the EO user community since March 2018. With a constantly growing data volume, optimizing use and uptake of Sentinel-2 data also by users with limited resources and expertise is a priority in order to maximize the return on investment of the overall Sentinel-2 mission. To do so, specific requirements have been adopted for Sentinel-2 products metadata and for the applied radiometric, atmospheric and geometric corrections in order to facilitate access to, understanding and processing by users of the satellite data itself. In parallel, in the last years a growing number of countries and international organizations have expressed a similar need for efficient and effective access, preprocess, and use of the growing volume of space-based data for local, regional, and national applications and decision-making. In order to answer to this global need, the Committee on Earth Observation Satellites (CEOS) has started the CEOS Analysis Ready Data for Land (CARD4L) initiative, where a CARD4L product is defined as "satellite data that have been processed to a minimum set of requirements and organized into a form that allows immediate analysis with a minimum of additional user effort and interoperability both through time and with other datasets". Provision of CARD4L products by the several satellite data providers is expected to reduce the burden on global satellite data users and to allow immediate creation of Data Cubes and subsequent enhancement of the scientific value of satellite data. Therefore, in the present paper an overview of the Sentinel-2 Surface Reflectance products is provided and their compliance to the CARD4L Framework and requirements is analyzed.

A recent initiative in easing the use of Earth Observing (EO) data is the concept of Analysis Ready Data (ARD). A fundamental goal of ARD is to significantly reduce the magnitude of data pre-processing for users who currently have to download and prepare large amounts of EO data of widely varying formats for local, regional, and national decision-making. The international EO community have expressed support for the Committee on Earth Observation Satellites (CEOS) to develop the CEOS Analysis Ready Data for Land (CARD4L) product framework. CARD4L are EO data “that have been processed to a minimum set of requirements and organized into a form that allows immediate analysis with a minimum of additional user effort and interoperability both through time and with other datasets”. The CARD4L framework continues to evolve but represents a commitment by global data producers to help support informed space-based decision-making.
The U.S. Geological Survey (USGS) has been involved with the development of CARD4L since its inception and is developing its Global ARD product to adhere to this international framework. Recent efforts at the USGS Earth Resources and Observation Science (EROS) Center have been devoted towards: 1) the development and review of the CARD4L framework for Global ARD (i.e., Landsat derived surface reflectance and surface temperature), 2) evaluation of the USGS EROS Global ARD for compliance with the CARD4L framework, and 3) independent peer review of the USGS self-assessment by the CEOS Working Group on Calibration & Validation (WGCV). The purpose of this paper is to provide an update on the status of these three efforts. USGS EROS Global ARD products, including metadata, will be compared to the current CARD4L specifications to determine areas of alignment and areas that need adjusting to conform with the CARD4L framework. In addition, a review of the previous Product Family Specification (PFS) for each Global ARD product will be provided along with discussion on how suggestions from this review were incorporated into the current version of the PFS which should lead to greatly improved data interoperability.

New generations of Earth Observation (EO) satellites are creating increasingly significant volumes of free and open data with comprehensive global coverage, such that the lack of data is no longer an issue. In addition to this data, research and development activities have delivered new applications that offer significant potential to deliver great impact to important environmental, economic and social challenges, including at the local, regional and global scales. These applications highlight the value of EO, though the challenge is in providing the proper connections between data, applications and users. Even today, much of the archived EO satellite data is underutilized despite modern computing and analysis infrastructures.

Addressing this challenge is difficult for advanced economies and even more challenging for developing countries with an interest in using EO satellite data. It is simply not technically feasible or financially affordable to consider traditional local processing and data distribution methods (e.g. scene-based file download over the internet) to address this “scaling” challenge in many economies, as the size of the data and complexities in preparation, handling, storage, and analysis remain significant obstacles. Fortunately, just as satellite Earth observation technology has advanced significantly, so too has information technology. The data management and analysis challenges arising from the huge increase in free and open data volumes can be overcome with new computing technologies (e.g. distributed computing clouds) and data architectures (e.g. data cubes). Solutions such as the Open Data Cube (ODC) have a great potential to streamline data distribution and management for providers while simultaneously lowering the technical barriers for users to exploit the data to its full potential.

Many satellite data users lack the expertise, infrastructure, and internet bandwidth to efficiently and effectively access, preprocess, and utilize the growing volume of space-based data for local, regional, and national decision-making. Even sophisticated users of EO data typically invest a large proportion of their effort into data preparation. This is a major barrier to full and successful utilization of space-based data, and threatens the success of major global and regional initiatives, such as the United Nations SDG initiative and the goals of the Group of Earth Observations (GEO). As data volumes grow, this barrier is becoming more significant for all users.

Countries and international organizations have expressed a desire for support from the Committee on Earth Observation Satellites (CEOS) to facilitate access to and processing of satellite data into Analysis Ready Data (ARD) products. Systematic and regular provision of ARD will greatly reduce the burden on global satellite data users. The provision of this data is possible through many options including systematic processing and distribution, processing on hosted platforms, and processing via toolkits provided to users.

ARD products are intended to be flexible and accessible products suitable for a wide range of users for a wide variety of applications, including time series analysis and multi-sensor interoperability. They are also intended to support rapid ingestion and exploitation via high-performance computing, cloud computing and other future data architectures. The result of this effort will be improved interoperability among datasets, facilitating time series analyses and enhanced global use and scientific value of satellite data.

The call for ARD across many satellite agencies, users, and in other communities, is being driven by practical requirements to reduce the demand for limited resources and expertise in the preparation of data, and to ensure that those preparatory steps are fully accounted and understood. These drivers are becoming stronger as the community seeks to apply rapidly growing data volumes from a range of sensors in new ways, especially as time series. Effective implementation of ARD requires that the definition and product specifications are widely accepted and understood so that the ARD products deliver their intended benefit to end users and can address the SDGs as well as other local or national needs.

CEOS is currently developing the definition and requirements for three land-based ARD products. These include optical moderate resolution (e.g. Landsat) surface reflectance and surface temperature and radar moderate resolution (e.g. Sentinel-1) normalized backscatter. Though the details are left in other documents, each of these ARD products includes minimum requirements for general metadata, per-pixel metadata, radiometric correction, and geometric correction. Optical measurements require additional corrections for atmospheric absorption and scattering and solar and viewing angle corrections. Radar measurements require additional corrections for terrain topography and incidence angle.

As the world develops, so does its knowledge of, and demand for, EO satellite data. The primary problems for users are data access, data preparation, and efficient analyses to support user applications. CEOS, through its network of global connections, has determined that global users share many common needs including: minimizing data preparation time, access to free and open data and application algorithms, efficient time series analyses, use of multiple interoperable and complementary datasets together in the same framework, and flexibility for local or cloud computing options.

It is these common needs that led the CEOS organization toward the initiation of the Open Data Cube (ODC) initiative which seeks to provide a free and open data architecture solution that has value to its global users and increases the impact of EO satellite data. The ODC is based on the implementation approach used by the Australian Geoscience Data Cube (AGDC) and planned for the future Digital Earth Australia program, but is modified to allow globalization for a diverse set of users, datasets and deployment options.

The Open Data Cube (ODC) infrastructure is successfully operating in 9 countries, in development in 7 countries and being evaluated by another 36 countries. Since its inception in early 2017, the ODC has made significant progress in the advancement of open source software tools and algorithms that support the deployment and operation of data cubes on local or cloud computing systems around the world. In addition, all of the current and future data cubes depend on Analysis Ready Data (ARD) to ensure efficient time series analyses and data interoperability while drastically reducing the burden on users.

Through a partnership between the Global Partnership for Sustainable Development and Data (GPSDD), the Committee on Earth Observation Satellites (CEOS), Amazon Web Services (AWS), the Government of Kenya, Strathmore University and the Group on Earth Observations (GEO), the Africa Regional Data Cube (ARDC) was launched on May 10, 2018 to support five countries: Ghana, Kenya, Senegal, Sierra Leone, and Tanzania. Based on the Open Data Cube (ODC) infrastructure, this open source data architecture responds directly to country-level demand for improved access and use of earth observation data to address the United Nations Sustainable Development Goals (SDG) and other priorities within each country. In addition to providing this free and open data infrastructure and application algorithms, the ARDC initiative is focused on developing the capacity and community of end-users across the region to apply analysis ready earth observation data to address local and national needs.

Users within Africa have already seen significant improvements in the efficiency of generating standard products (e.g. cloud-free mosaics) compared to existing methods and are eager to build capacity to create their own unique algorithms and products. As of late 2018, the ARDC includes 17 years of Landsat data, but will soon be ingesting new data from Sentinel-1 (C-band radar), Sentinel-2 (optical), and ALOS (L-band radar). The diversity of these datasets and the ability to use them interoperably will yield many new products and have an influence on decision-making and policy in the region.

Since the release of the ARDC, there have been a number of interactions with local Africa users to identify priority applications and use-cases including: water extent (reservoirs and flooding), illegal mining, deforestation, urbanization, agriculture productivity, land degradation, water quality (sediment), burn area extent, and coastal erosion. These applications have prompted the development of algorithms including: cloud-free mosaics (several types), spectral indices (e.g. NDVI, EVI, NDBI, Fractional Cover), land classification (e.g. K-Means, Random Forest), water detection using Landsat (WOFS from Australia), water detection using radar (WASARD from NASA), water quality (TSM from Australia), land change (e.g. PyCCD from USGS, PCA from Colombia, NDVI Anomaly), coastal change, and landslide risk (SLIP from NASA).

This presentation will show several examples of time series water extent, land change and urbanization and will demonstrate the significant impact of analysis ready data in the ARDC. It is highly unlikely that users in Africa would be able to utilize the vast volumes of satellite data without removing the burden of pre-processing and allowing more time for analysis and the generation of products. In addition, these ARD datasets were compiled in time series stacks to allow interoperability among multiple datasets and efficient time series analyses. Such products would be nearly impossible, or quite difficult, to produce, using traditional scene-based analysis methods. Through the ODC, the impact of ARD is growing rapidly around the world and our ability to address local and national needs for decision-making will continue to improve.

Many land monitoring applications require more frequent observations than can be obtained from a single “Landsat-class” sensor. Examples of these applications include crop type and condition monitoring, vegetation phenology, disaster response, and surface water quality. These and numerous other applications all require near-daily imagery at medium spatial resolution. Since trying to achieve near-daily observation frequency from a single, well-calibrated satellite system may be prohibitively expensive, a reasonable alternative is to combine data from multiple international sources.

The concept of Analysis Ready Data (ARD) revolves around providing users with ready to use surface reflectance or temperature data products with radiometric and geometric pre-processing issues resolved. Such products consist of measurements generated to known precision, and gridded in fixed tiles to facilitate time series applications and change detection. While most current ARD products involve data from a single sensor, there is growing need for multi-sensor ARD efforts. An appropriate criterion for multi-sensor ARD is that the data from different sensors be interoperable – that is, observations from multiple sensors may be used interchangeably and transparently without further processing by the end user.

The Harmonized Landsat/Sentinel-2 (HLS) products from NASA GSFC are an example of multi-sensor ARD. The goal of HLS has been to create a seamless surface reflectance record from both Landsat 8 and Sentinel-2a,b, suitable for land monitoring applications that require frequent observations. Specifically, by “harmonized” we mean that the products are:
• Gridded to a common pixel resolution, projection, and spatial extent (i.e., tiled);
• Atmospherically corrected to surface reflectance using a common radiative transfer algorithm (the LaSRC algorithm, Vermote et al. [2016]).
• Normalized to a common nadir view geometry via Bi-directional Reflectance Distribution Function (BRDF) estimation (Roy et al., 2016)
• Adjusted to represent the response from a common spectral bandpass via a linear regression model.
Two primary products, S30 and L30, are generated from Sentinel-2 and Landsat 8 respectively, but resampled to 30-meter resolution grids based on the Sentinel-2/MGRS UTM tiling system. QA bits for each product include cloud, shadow, and aerosol opacity flags.

Dataset-level quality assessment (QA) activities are ongoing. We distinguish between product QA and Validation. The former is geared toward generating qualitative information on the suitability of individual pixels or granules for further analysis, while the latter provides quantitative assessment of product uncertainty. HLS Product QA relies primarily on comparison with contemporary MODIS Terra Climate Modeling Grid (MOD09CMG) reflectance products. HLS granules with more than one CMG pixel that exhibit large deviations from the contemporary MODIS CMG data are flagged and discarded from the published data set. Quantitative validation of the HLS reflectance products relies primarily on (i) validation of the LaSRC atmospheric correction approach as described in [5]; (ii) comparison with SURFAD broadband radiometric measurements; (iii) analysis of temporal stability over psudo-invariant calibration sites. Temporal stability metrics indicate relative uncertainty of ~3-4% over short periods.

Version 1.4 of the HLS data set was released in November 2018, and includes data for ~25% of the global land area, including all of North America, national coverage for Germany, France, Ukraine, and South Africa, and a distributed collection of additional global test sites. Temporal coverage includes Landsat 8 data from 2013 to present, and Sentinel-2a,b data from 2015 to present. New products are routinely processed and made available with latency <7 days. Products may be downloaded from the HLS website: https://hls.gsfc.nasa.gov .

This presentation will discuss the status and evolution of the HLS project in the context of multi-sensor ARD. Two key points will be emphasized: (1) the dependence of multi-sensor ARD on rigorous sensor-level characterization and cross-calibration activities, including in-situ measurements of reflectance; (2) the ongoing need to quantify uncertainty in derived harmonized products. In addition, we will review recent applications of HLS products for land monitoring, including agriculture and forestry.