The importance of wake detection on SAR images is twofold: to estimate vessel velocity by exploiting the Doppler anomaly (a ship and its wake show different Doppler frequency due to the different speed) and to detect ships with faint scattering (very small boats, or small and very fast boats). When visible, wakes are quite evident and their detection by mean of photo‑interpretation is a quite easy task even when they are not straight lines or have irregular shapes. But automatic algorithms have always shown their limits.

Historically this problem has been addressed by looking for peaks in the Radon transform of the SAR image. Such approach suffers from three issues:
- It assumes rectilinear wakes
- Speckle and the non‑uniform appearance of wakes can decrease their detectability
- In case of wake detection without previous ship detection, the choice of the size of image clip to process is critical, because the Radon assumes segments limited only by the clip itself: too large clips, therefore, lead to missed detections, whilst clips too small lead to many false alarms. In any case, it is also very expensive to run it on whole images or very large clips.

Highly twisting routes and irregular sea features like low wind areas and natural slicks are just few examples of how the task of wake detection, which is well accomplished by photo‑interpretation, can be actually a challenging task for fully automatic algorithms.

In addition to the difficulties to extract wakes from a complex sea scene, speckle and existence of different types and aspects of wakes in different weather and sea conditions and in different SAR modes and resolutions make the automatic wake detection task even harder.

This is a typical problem where machine-learning techniques can be very effective. In this work, we build a manually labelled dataset of SAR clips with and without wakes and we show how deep Convolutional Neural Networks (CNN) can be successfully exploited to detect wakes on SAR images. To our knowledge, this is the first time such networks are used to address this task. The advantage is clear: machine‑learning techniques can naturally deal with non‑rectilinear wakes and irregular patterns simply because the network has been trained with real life examples and it does not depend upon the hard assumptions needed by the radon‑based techniques. Moreover, although the data collection and training phases may last long, the operational phase is very fast and hence well suited for processing whole images in near real time applications.
Experiments showed very promising results, demonstrating the applicability of the technique to such task also in near‑real time scenarios.

SUMMARY
Recent implementations of machine learning (ML) tools running on large scale data have been enabled by vast improvements in computing capability and parallelization, theoretical advances in ML models (in particular in the field of deep learning), and most importantly, an explosion of available data – the fuel of any ML engine.
Copernicus' unparalleled volume and quality of Earth observation (EO) data has been paramount in driving recent improvements; however, the EO field has not yet seen significant uptake of ML methods due to several shortcomings, which we will address in the presentation:

•There are only a few, high-quality sources of annotated data currently available for training supervised algorithms on satellite imagery. This data is critical to the advancement of ML research. We will present a tool for crowd-sourced classification and provide a means to integrate with existing open datasets (OpenStreetMap, SpaceNet, etc.), thus creating a large, new corpus of data for researchers to make use of.
•Most available ML frameworks require special approaches to accept EO data. The most challenging aspect is handling time-series data, which is critical for the classification of vegetation. We will present an integration of various tools (eo-learn, sat-utils and label maker) to leverage popular open source libraries like TensorFlow and MXNet EO ML.
•Many ML efforts in the EO field are limited to a very specific geographic area. To scale results on a global scale, we are relying on AWS infrastructure, including its SageMaker platform, and Sentinel Hub satellite imagery processing services.

All tools and datasets developed are released under open-source licenses. This provides an opportunity to open Copernicus data for further research and ML applications by providing annotated data and software tools. Due to the temporal and spatial resolution of these products, the data is especially well suited for creating new and novel ML algorithms and EO applications.

MACHINE LEARNING
With the availability of massive volumes of data, ML has become an important tool for analysis, ranging from various random forest algorithms to complex convolutional neural networks. Satellite imagery displays characteristic issues and artefacts: clouds, atmospheric effects and inaccurate geolocation are distorting the data, missing or cloudy scenes create gaps, etc., making it difficult to exploit well-known frameworks such as TensorFlow, MXNet, and others. Furthermore, lack of ground truth for training and validation is one of the major challenges preventing from efficient use of ML tools.

We try to address these elements within the Query Planet project, running within ESA's Φ-Lab, which is developing the following open-source tools:

•eo-learn, package acting as a bridge between EO data and existing ML and computer vision tools
•classification application and label maker enhancements
•repository for ground truth data

Importantly, the above-mentioned elements are integrated and demonstrated on a number of use-cases, mainly related to land cover classification and global water monitoring. These use-cases are available in an open-source manner, including training sets and networks, which makes it extremely convenient for anyone to use, modify, and extend.

EO-LEARN
eo-learn is a collection of open source Python packages that have been developed to seamlessly access and process spatio-temporal image sequences acquired by any satellite fleet in a timely and automatic manner. eo-learn is easy to use, its design is modular, and encourages collaboration – sharing and reusing of specific tasks in a typical EO-value-extraction workflows, such as cloud masking, image co-registration, feature extraction, classification, etc. Everyone is free to use any of the available tasks and is encouraged to improve on them, develop new ones and share them with the rest of the community.

eo-learn makes extraction of valuable information from satellite imagery as easy as defining a sequence of operations to be performed. Figure 1 illustrates a processing chain that executes automatic classification of land cover in a user specified region of interest.

The eo-learn library acts as a bridge between the EO/Remote sensing field and the Python ecosystem for data science and ML. The library uses NumPy arrays and Shapely polygons to store and handle remote sensing data. Its aim is to make entry easier for non-experts to the field of remote sensing on one hand, and bring state-of-the-art tools for computer vision, ML, and deep learning existing in Python ecosystem to remote sensing experts.

The design of the eo-learn library follows the dataflow programing paradigm and consists of three building blocks:

•EOPatch - common data-object for spatio-temporal EO and non-EO data, and their derivatives; it contains multi-temporal remotely sensed data of a single patch (area) of Earth’s surface, typically defined by a bounding box in a specific coordinate reference system, both in raster and vector format. EOPatch is completely sensor-agnostic, meaning that imagery from different sensors (satellites) or sensor types (optical, synthetic-aperture radar, etc.) can be added to an EOPatch.
•EOTask - a single, well-defined operation being performed on input EOPatch(es), which returns a modified EOPatch. EOTasks are the heart of the eo-learn library. They define in what way the available satellite imagery can be manipulated in order to extract valuable information out of it. Typical users will most often be interested in what kind of tasks are already implemented but can also write custom EOTasks, if their desired functionality doesn’t yet exist.
•EOWorkflow is a collection of EOTasks that together represent an EO-value-adding-processing chain or EO-value-extraction pipeline by chaining or connecting sequence of EOTasks. The EOWorkflow takes care that the EOTasks are executed in the correct order and with correct parameters. Under the hood the EOWorkflow builds a directed acyclic graph. There’s no limitation on the number of nodes or the graph’s topology, as long as the graph is acyclic.

There are several existing packages, covering common EO analysis steps:
•eo-learn-core, the main subpackage which implements basic building blocks, commonly used functionalities, and logging/reporting.
•eo-learn-coregistration, dealing with image co-registration to correct gelocation errors.
•eo-learn-features is a collection of utilities for extracting data properties and feature manipulation.
•eo-learn-geometry is used for geometric transformation and conversion between vector and raster data.
•eo-learn-io, input/output subpackage that deals with obtaining data from various data source services or saving and loading data locally. It provides seamless access to global archive of Sentinel-1 GRD, Sentinel-2 (L1C and L2A), Sentinel-3 OLCI, Sentinel-5P, Landsat-8, MODIS, Envisat MERIS and ESA archive of Landsat-5 and -7 through the Sentinel Hub services. Open-source libraries sat-utils are used to work with locally stored or remotely accessible GeoTiff files and OpenStreetMap data.
•eo-learn-mask, used for masking of data and calculation of cloud masks.
•eo-learn-ml-tools - set of tools that can be used before or after the ML process.

The eo-learn package can be easily integrated with other Python packages, e.g. within EOTask node. We will demonstrate how it works with TensorFlow and other ML libraries. Jupyter Notebook is used as the main "IDE".

GROUND TRUTH LABELS
We are addressing lack of ground truth data required for training and analysis in two ways - by identifying openly available regional and global datasets of proper quality, which can be used as an input, and by creating a classification app, which can be used by experts or crowds to collect missing inputs. OpenStreetMap, SpaceNet, Corine land-cover, various official register datasets (buildings, roads, farm parcels) and similar can be efficiently used to create training data. These sources will be freely available to use on the Geopedia cloud-based GIS.

The classification app is a web-based tool, which requires user authentication, in order to associate individual records with a specific user for labelling quality assessment. Users can define their own labelling campaigns, and make them public or private. Campaigns define which satellite images will be annotated (e.g. Sentinel-2 or other data-source, based on the use-case), the size of the area (e.g. 512x512 px), and the sampling method of areas selected (e.g. random selection, mis-classified area by ML model). Completeness of the labelling over the whole area is required in cases where one wants to avoid vaguely defined data while completeness is not enforced in other cases, where we are looking only for specific elements (e.g. built-up areas). Users are able to explore the area around the dedicated tile, and visualise various band combinations (e.g. NDVI, false colour, NDWI, custom option). Campaigns allow to address various use-cases (label options, area limitations, satellite imagery sources, supporting datasets). The open-source nature of the tool allows further customization. Classified data can be exported using a dedicated API (integrated with eo-learn) or exported in standard formats (e.g. SHP, GeoTiff, GeoJSON).

CONCLUSION
Volume, availability and quality of open EO data have reached the level where big data methods are not just a meaningful option, but a necessity. However, there are not yet many established options freely available and we believe that Query Planet is addressing these needs. We will demonstrate its usability in two start-to-end use-cases – land cover classification on country scale and global monitoring of water reservoirs.

This announcement, part way through the project, is meant to call for cooperation with other researchers in the field, so that we can produce results fitting their requirements if possible.

Currently, what exists in the field of data fusion is a collection of routines/algorithms that can be linked and embedded for various applications. A very well-known open-source toolbox is Orfeo which provides a large number of state-of-the-art algorithms to process SAR and multispectral images for different applications. Another one is Google Engine that includes a large image database and a number of algorithms (or you can add your algorithms) that can be used for image processing.

An innovative system is CANDELA, an H2020 research and innovation project under grant agreement No. 776193, which has as one of its objectives the fusion of radar and multispectral images at semantic level but also at feature/descriptor level. The first results will be presented during the Living Planet Symposium.

For this case, we propose to recognize different target area details in overlapping SAR and multispectral images complementing each other with rapid succession. For doing this, we already selected Sentinel-1 and Sentinel-2 images that can be rectified and co-aligned by publicly available toolbox routines offered by ESA allowing a straightforward image comparison.
In addition, we propose data fusion, and interpretation. The most important aspects to be considered are:

• The Sentinel-1 C-band constellation consisting of two radar satellites following each other along the same orbit delivers Earth surface images from their side-looking radar instruments taken during day or night. The images have a spatial resolution of typically 20x20 m. One can easily discriminate bright reflectors (e.g., edges of buildings) from dark surfaces (e.g., windless water surfaces). Brightness differences of Sentinel-1 pixels can be due to surface roughness or smoothness, edges of buildings, farming practices, etc. The main advantage of Sentinel-1 is that it offers virtually cloud-free images.

• The Sentinel-2 twin satellite constellation is delivering Earth surface images from their multicolour nadir-looking optical imagers taken during daylight. These images have a spatial resolution of typically 10x10 m. Brightness differences among the colour bands can be analysed to discover land cover characteristics such as urbanisation, vegetation properties, or current sea surface dynamics. Infrared images can be used primarily for vegetation properties. In contrast, Sentinel-1, Sentinel-2 images are affected by clouds and other weather conditions.

• The fusion of Sentinel-1 and Sentinel-2 data shall allow a joint interpretation of radar backscatter and optical reflectance data. As ESA has undertaken considerable effort in the absolute calibration of all Sentinel instruments, we expect the discovery of many hitherto unseen phenomena with well-defined confidence levels.

While we are accustomed to image fusion as a radiometric combination of multispectral images, a comparably mature level of semantic fusion of SAR images has not been reached yet. In order to remedy the situation, we propose a semantic fusion concept for SAR images, where we will combine the semantic image content of two data sets with different characteristics (e.g., TerraSAR-X and Sentinel-1). In our case, we observed several coastal areas in Europe by a high- and a mid-resolution spaceborne instrument and combined their information content. By exploiting the specific imaging details and the retrievable semantic categories of the two image types, we obtained semantically-fused image classification maps that allow us to differentiate several coastal surface categories. In order to verify the classification results, we will compare SAR images to multispectral satellite images and in-situ data.

The simulations of microwave radiometry (MW) observations of the atmosphere at kilometer scale have a wide range of applications, especially over ocean.

The future SWOT mission for ocean topography will offer sea surface height measurements over a 1-km grid. The wet tropospheric correction (WTC, proportional to the water vapour) being a major source of error, MW simulations are used to assess the performance of the on-board microwave radiometer which is dedicated to the WTC retrieval.

The retrieval of large wind speeds over hurricanes is a promising application of SAR observations and is expected to improve the forecast of their trajectories. A better understanding of the impact of the atmosphere (ice, rain, clouds) on the SAR backscattering coefficient through MW simulations would allow a better accuracy of the wind speed retrieval.

MW simulations are based on the use of a radiative transfer model (RTM) applied on numerical weather prediction model (NWP). Each NWP grid cell is described by its surface state (temperature, wind speed, dew point temperature) and its atmospheric state (temperature, humidity and liquid water path) over a given number of pressure levels (typically 60 levels).

Considering the complexity of the RTM, the simulation of observations at a single wavelength over a typical scene of 50 km x 50 km takes more than 1 hour using a serial approach and several wavelength are required for a proper description of the atmosphere (humidity, temperature, clouds, rain).

During the past couple of years, a new computational environment has emerged which allows a large reduction of the time consumption and paves the way to new applications for MW simulations.

The approach proposed here benefits from the xarray/dask python environment for the computation: the dask library allows parallel computing using the common python library API for scientific computation as xarray and scikit-learn, including neural network training and prediction.

The Apache parquet improves the I/O performances and is used for the storage of the NWP inputs and the simulated outputs.

The time consuming level-by-level RTM simulations step is replaced by a neural network trained over a statistically representative dataset.

The presentation will focus on the scientific validation over typical atmospheric situations and on the time consumption benefits. Illustration are provided for application of the same code on a personal laptop and on a hundred-node cluster.

Environmental status assessment often relies on experts’ ill-defined knowledge about the physical interaction of the electromagnetic radiation with the environmen-tal phenomenon of interest. It consists in the integration of multi-band information within a multi spectral image at a given timestamp. Namely, Spectral Indexes (SI) are defined on a real domain to integrate a large multi spectral image to a more synthetic form that can highlight some aspects of the environmental status: by applying a function combining the bands’ signals, SI maps can be generated and then segment-ed to identify vegetation vigor, water bodies, bare soil areas and so on. Nevertheless, to describe complex phenomena such as flooded areas and burned areas single SI are often ineffective for several reasons:
• many environmental phenomena have a different appearance when changing the geographic context and observation conditions (specific land covers, presence of clouds, shadows, etc.): a single SI may be not sufficient to capture all aspects of a given phenomenon. For example, to identify all types of water such as shal-low water, deep water, wetlands, rivers and inland water bodies, rice flooded fields may distinct SI have been defined.
• One needs a calibration phase for determining the proper threshold on the SI val-ues that allows segmenting the phenomenon footprint, i.e., the spatial extent of the phenomenon, with an acceptable accuracy. Often one must engage with sev-eral trials and error phases to find the best threshold(s) to segment the SI maps for detecting the areas interested by the phenomenon and affected by the lowest pos-sible commission and omission errors.
• Besides the SIs, other contributing factors may constrain and influence the envi-ronmental phenomenon footprint. For example, floods generally occur in mostly flat regions while cannot occur in area characterized by steep aspect. Expert’s and heuristic knowledge of an environmental phenomenon is often expressed by mul-tiple criteria which contribute to distinct extent to determine or influence its occur-rence.

To overcome the limitations listed above we have conceived a fuzzy approach for defining Environmental Status Indicators (ESI) [1] with the following properties:
• the ESI is defined in [0,1] and can be computed at pixel level or at Region Of Interest (ROI) level: for each individual pixel (or ROI) a single ESI value can be computed; in this paper we consider the pixel level ESI;
• the ESI has a stable performance with respect to its ability to detect the phe-nomenon: when increasing (decreasing) the threshold on the ESI values in [0,1] the accuracy of the detected phenomenon mapping is high and stable.
• The ESI can exploit ill-defined knowledge of the experts, i.e., it allows the in-tegration of information derived from any contributing factor, i.e., physical variable which is available and deemed by experts as relevant to influence an aspect of the environmental phenomenon of interest.
• The ESI is adaptive to the specific local context and observation conditions and its computation is feasible in a distributed processing framework on big geo data so as to map the ESI in near-real time to support decision makers in critical situations.
• The ESI computation can model distinct criteria:
• the distinct credit of a contributing factors;
• the possibility to take into account the local agreement of contributing factors to determine their influence on the result;
• the possibility to model the attitude to risks of the decision maker in de-fining the ESI;
• the possibility to optimize the ESI in presence of ground truth.

This approach stems from the way in which synthetic maps are created within a traditional GIS, in which first several layers of thematic information are loaded, then from each layer constraints are defined to perform selections of features, and finally all features are aggregated in a synthetic map by applying a Boolean operator, namely intersection or union. This GIS operative modality suffers for the defects derived by the rigidity of both the constraints and the aggregation op-erators which are crisp and thus admit only Boolean satisfaction degrees.
To overcome such deficiencies, we generalized this traditional GIS operative mo-dality by proposing an approach that allows the propagation of imprecision to the end of the process by allowing the specification of soft constraints, i.e., soft selec-tion conditions admitting degrees of satisfaction, and fuzzy aggregation operators with behaviour that can be tuned in between that of the intersection and union. The approach heavily relies on the approximate knowledge of the expert to iden-tify the thematic maps, i.e., the contributing factors, and to define both the soft constraints and the aggregation operator to generate the synthetic status map. It is performed in two phases:
• in the first phase contributing factors, physical variables in the form of the-matic maps that can influence the phenomenon, are identified by experts; a statistic analysis of the values of the contributing factors is performed on the training classified data in order to define soft constraints which better dis-criminate the class of interest from the others; soft constraints admit degrees of satisfaction in [0,1], which are interpreted as degrees of partial evidence of the phenomenon due to a specific contributing factor; in this phase, also an importance for each factor can be computed proportional to the degree of separability between the classes achieved by applying the soft constraint on the training data set; alternatively a degree of reliability or trust can be deemed for each factor depending on the knowledge of the phenomenon;
• in the second phase, a linguistic quantifier, such as most, is specified to de-rive a quantifier guided aggregation operator, an Ordered Weighted Averag-ing (OWA) operator [2] modeling a desired blending of AND and OR aggre-gation. The OWA is applied to aggregate the partial evidence maps to gener-ate the synthetic ESI map. OWA operators allows to define fusion strategies with a mean like semantics whose decision attitude can be represented by a values in [0,1], where 1 is fully optimistic and 0 fully pessimistic.
A weakness of the approach is the fact that while experts can identify the contrib-uting factors and define the soft constraints for deriving the partial evidence maps, they have generally no a priori knowledge for specifying the most appropriate linguis-tic quantifier defining the OWA operator: they proceed by applying some linguistic quantifiers, namely at least 1, all and average, and by evaluating their results on a test set. The OWA achieving the best accuracy is then applied to compute the ESI maps in all areas. Clearly, with a few trials one cannot be sure to have identified the best OWA.
In the present work we propose a machine learning approach exploiting Volunteered Geographic Information (VGI), assumed as ground truth, to learn the best OWA op-erator for a given ROI. Indeed VGI is created by the use of smart applications by operators surveying an ROI. The workflow of the overall process for generating an ESI map is depicted in figure 1.
Notice that the process is in two phases which can be implemented in a map reduce fashion. Machine learning takes place in the second phase. It explores the space of OWA operators in order to identify the best ESI mapping in accordance with the available VGI in each given ROI. Such a flexibility enables the possibility of adapting the learning of the OWA operator to a ROI selected by the expert based on a given spatial stratification: for example, ROIs can be identified as homogeneous area with respect to given land covers and a specific OWA operator can be learned inde-pendently from each ROI. Then the obtained OWA operator for a given ROI can be applied to any area having similar land covers of the ROI.
Moreover, learning the aggregation operator can make results less sensible to slight different definitions of both the contributing factors and the soft constraints. In facts, when defining distinct contributing factors and soft constraints the first phase com-putes different partial evidence maps; thus the process learns the best OWA operator given the current evidence maps for each ROI in which VGI is available.
By discussing a study case of flooded area mapping we show that the proposed ap-proach can adapt to distinct ill-defined knowledge of the experts and to local condi-tions achieving stability and good accuracy, with an average F-score above 0,88 for any threshold in (0,1] obtained by a 10-fold cross-validation in three distinct ROIs using as training set for the learning, i.e., ground truth, only 10% of the available data, and the remaining 90% for the evaluation (see Figure 2): this unfavorable cross-validation condition was done to test the robustness of the approach when the train-ing data are a small amount with respect to the area to classify, as it occurs in real situations. Finally, we show how the learned OWA operator provides added knowledge on the relationships of the contributing factors and on the decision atti-tude to risk modeled by the ESI.

References
1. Bordogna, G., Pagani, M., Pasi, G., A Flexible Decision support approach to model ill-defined knowledge in GIS, in Geographic Uncertainty in Environmental Security, A. Morris and S. Kokhan eds , Springer, ISBN 978-1-4020-6436-4, 2007.
2. Yager, R.R., 1996, Quantifier guided aggregation using OWA operators. International Jour-nal of Intelligent Systems, 11, pp. 49–73.

A better understanding of the infrastructure across Mongolia is required in order to manage the country’s livestock, which supports approximately 30% of the population. Mongolia is a large country with a relatively small population, which makes surveying its vast territory time consuming and expensive using ground-based techniques. Satellite Earth Observation can therefore provide a cost-effective alternative for providing frequent updates on the locations of livestock shelters, wells, tracks and gers, which are the traditional Mongolian portable dwellings used by nomadic communities. However, Mongolia’s large size also presents a data handling and image processing challenge.
This paper describes how machine learning based techniques – particularly Deep Learning have been used for detecting and monitoring Mongolian infrastructure, to provide both a static and dynamic view of human and livestock distribution across the landscape. Certain features like gers are very distinctive, whereas other features such as shelters are challenging for both classification and object-based methods to extract a sufficiently detectable signature.
This paper will present the key successes and challenges of this application and highlights the project’s collaboration with Mongolian stakeholders. In particular, the Administration for Inter-Aimag Otor Pastureland Use & Coordination, which sits within the Ministry of Food, Agriculture and Light Industry (MoFALI) and has a remit to create, develop and manage reserve areas which are only made accessible to herders during extreme weather conditions, when they are unable to find food and shelter for their livestock.
For the initial ger detection, a self-designed model for image segmentation (based on U-net architecture) has been applied. For the future purposes of the project, different Convolutional Neural Network (CNN) architectures will be tested, including object detection models (for example Fast R-CNN) and models that combine both segmentation and object detection (Mask R-CNN). A tiered approach using Sentinel 2 data and Worldview is implemented, in order to assess which features can be detected at which resolution. Manually digitized data is used for validation, because traditional vector datasets (such as Open Street Map) are not available for this type of infrastructure.
The SIBELIUs project is supported by the UK Space Agency’s International Partnership Programme (IPP).