Image classification is a well-known technique used in remote sensing to map land cover types and to monitor changes at the Earth’s surface over time. Most image processing software applications include multiple classifiers, along with the ability to display results and to evaluate the accuracy of the classification. In this study, a visual modelling tool was used to build multiple classification workflows using machine learning techniques such as Softmax Regression (SR) and Support Vector Machine (SVM). The modelling tool simplifies the process of building custom classification workflows without requiring any knowledge of a specific application programming interface (API).

The study area chosen for this research is in eastern Texas, USA, along the Brazos River. The site was chosen because it contains a mix of land cover classes that are predominantly vegetative, including wetlands, forest, agriculture, and rangeland.

A Sentinel-2A Level-1C image of the study area was used for image classification. Additional processing steps were taken to prepare the image for analysis: (1) Layer stacking the visible and near-infrared bands, while resampling the 20-meter bands to 10 meters, (2) defining a spatial subset around a specific area of interest, and (3) masking all non-vegetation features such as roads, water, and buildings. A first-order entropy texture image was also created from the red band of the Sentinel-2 image.

Training data consisted of samples of Sentinel-2 image pixels that represent the following feature types: Nonforested wetland, Rangeland, Forest, Grass/Pasture, Cropland. Distinguishing between these land-use types using remote sensing imagery can be challenging, especially since many areas contain a mix of land-use types. In seldom cases the categories are defined by abrupt boundaries, except for possibly cropland.

Several models were built to execute and compare the results of multiple classifiers. Supervised classification based on machine learning typically requires extra steps such as normalizing and randomizing the input data, training the classifier, and minimizing loss. The modelling tool incorporates these elements into the classification process. Users only need to define the trainer (such as gradient descent or iterative) and the classifier (such as SR and SVM) that will be used.

Each model was run using 10 spectral bands as input to the classification, and again with an entropy texture layer as additional input, to determine if texture improved classification accuracy.

This study demonstrates that models facilitate the comparison of different classifiers while allowing an analyst to experiment with different input features. Building a model with a visual programming tool has the benefit that inputs, outputs, data management operations, and processing tasks can be linked with a drag-and-drop user interface instead of learning API code. Models can be packaged and deployed to desktop and cloud-computing environments for reuse and further customization. Model files can also be incorporated into larger image-processing models.

Area C - Board 305

Many vision applications require high-accuracy dense disparity maps, and a large number of algorithms for stereo matching have been developed. However, some issues still remain and the contribution of neural networks has not yet been investigated in the context of space imaging. Indeed, while neural newtorks have recently been very successful in computer vision tasks, relatively little work has been done on stereo matching. To our knowledge, there is no study comparing neural networks with current methods on satellite images.

We compare two neural networks architectures : MC-CNN [1] and CRL [2]. The first method MC-CNN, approach the problem by learning a similarity measure on small image patches by using a siamese network. CRL network, learns the transformation between the two images, it's an end-to-end approach. Our first contribution has been to adapt both networks to our needs since MC-CNN is not suited for satellite images and no open-source CRL version can be found.

Both neural methods were then trained on WorldView satellite images, resulting from the IARPA challenge. As ground truth disparity is not available for this dataset, we compared four classic matching algorithm MEDICIS [3], MSMW [4], SGM [5] and MGM [6], and we used the one that perform better to create a ground truth disparity maps. In our study, MGM outperforms other classic approaches. Both neural networks were therefore trained on the disparity maps produced by MGM, and were evaluated on an region of interest covered by a lidar ground truth by measuring the percentage of pixels where the true elevation and the predicted elevation differ by more than one meter.

Our experiments show that, MC-CNN with an error rate of 42.6%, provides state-of-the art performance on satellite images, similar to MGM with an error rate of 40.18%, and has the ability to generalize his training to other types of landscapes. CRL with an error rate of 66.24% produces poor results, and does not have the ability to generalize his training. CRL's results are probably biased by the the way learning has been achieved. The next study will therefore aim at creating a ground truth simulator for satellite images based on 3D models, in collaboration with the CNES. A CRL analysis on direct learning of the altitude map is in progress, since the CNES 3D simulator currently provides ground truth elevation maps.

[1] Zbontar, J., LeCun, Y. Stereo matching by training a convolutional neural network to compare image patches. Journal of Machine Learning Research (JMLR) 17 (2016) 1-32.
[2] Pang, J., Sun, W., Ren, J. S., Yang, C., & Yan, Q. Cascade residual learning: A two-stage convolutional neural network for stereo matching. In International Conf. on Computer Vision-Workshop on Geometry Meets Deep Learning (ICCVW 2017) (Vol. 3, No. 9), (2017, August)
[3] Cournet, M., et al. "2D Sub-Pixel Disparity Measurement Using QPEC/Medicis." International Archives of the Photogrammetry, Remote Sensing & Spatial Information Sciences 41 (2016).
[4] Buades, Antoni, and Gabriele Facciolo. "Reliable multiscale and multiwindow stereo matching." SIAM Journal on Imaging Sciences 8.2 (2015): 888-915.
[5] Hirschmuller, Heiko. "Stereo processing by semiglobal matching and mutual information." IEEE Transactions on pattern analysis and machine intelligence 30.2 (2008): 328-341.
[6] Facciolo, Gabriele, Carlo De Franchis, and Enric Meinhardt. "MGM: A Significantly More Global Matching for Stereovision." BMVC. 2015.

Area C - Board 306

With the launch of new generation satellite missions the need to access historical Earth Observation (EO) data series has strongly increased, particularly for long-term science and environmental monitoring applications. This trend is driving users to request time-series of data spanning 20 years and more, likely to increase even more in the future, in particular regarding the growing interest on global change monitoring and policy makers decisions on the observed changes particularly in the climate system (atmosphere, ocean, cryosphere, carbon and other biogeochemical cycles, sea levels).

The content of EO data archives is extending from a few years to decades and therefore, their value as a scientific time-series is continuously increasing. The advances in satellite sensor characteristics (spatial resolution, temporal frequency, spectral sensors) as well as in all related technical aspects (data and metadata format, storage, infrastructures) underline the strong need to preserve the EO space data without time constraints and to keep them accessible and exploitable, as they constitute a humankind asset. This paper describes the ESA Heritage Mission Programme (LTDP+) activities aimed at the generation of long time series of coherent data through valorisation and curation of heritage data and for providing continuity with current and future missions.

To ensure, enhance and facilitate discovery and access of ESA space science heritage data (and associated information holdings) by a broader community of users the LTDP+ Programme supports pilot initiatives to assess innovative tools and services. The Heritage Mission Data Cube is based on a new paradigm aiming at pulling out the full potential of EO data providing access to large spatio-temporal data stored in remote archives and enabling on-the-fly analysis-ready form. Systematic and regular provision of Analysis Ready Data (ARD) significantly reduces the burden on EO data users. This paper will therefore describe the Heritage Mission Data Cube platform and relevant functionalities put in place to demonstrate the easiness of data exploration and delivery of satellite heritage data in its native context of a virtual globe via any standard Internet browser.

Area C - Board 307

The Observation to Archive (O2A) concept of the Alfred-Wegener-Institute (AWI) is a data-flow framework developed by AWIs computer-center to build a generic and cost-effective virtual research infrastructure to enable the flow of data from sensor observations to archives.

O2A is comprised of several extensible and exchangeable components as well as various interoperability services and is meant to offer practical solutions towards supporting the typical scientific workflow ranging from data acquisition activities until the very last data publication activities.

Examples of O2A components are: (1) the SENSOR and STREAM components designed to provide metadata on platforms, instruments and sensors along with near real-time data transfer solutions, (2) the DASHBOARD component offering dashboard-oriented monitoring monitoring solutions which includes graphing and mapping widgets among others, (3) the VIEWER offering map-based visualization and analysis solutions, (4) the repositories PANGAEA and EPIC for data and publicationsrespectively, and (5) the DATA portal as a one-stop shop web interface for disseminating various types of scientific content associated with research platforms and thematically grouped data and data products.

On this poster we like to introduce the integration of satellite data flows of ESAs Sentinel missions into the O2A framework for scientific usage in a quite user friendly environment.

Area C - Board 308

The GeoImageNet project is a unique collaborative initiative between remote sensing researchers, web platform developers, artificial intelligence experts and satellite imagery providers. Through the GeoImageNet platform that will be operational in June 2019, this project will help establish links with other Canadian researchers in academia, public research centres and industry. Very High Resolution (VHR) satellite images are useful in several areas of remote sensing research and the improvement of algorithms for using these images is an area of interest for many of Earth Observation (EO) researchers. The main objective of the project is to implement a web platform that allows users to make annotations on VHR images (e.g.: Pleiades 50 cm; WorldView-2 40 cm; WorldView-3 30 cm). The resulting training and test sets can be then used to train and benchmark deep learning models on VHR images.

This work is inspired by the well known Computer Vision project known as Imagenet [1,2,3,4]. This seminal initiative provided the Computer Vision community with a large annotated dataset that fostered the development of new state-of-the-art models for years. One of the keys to its success is both the large size of the corpus and the extent of its taxonomy (WordNet).


The necessity of developing the GeoImageNet platform today stems from the need to exploit new remote sensing data, such as very high spatial resolution or very high temporal frequency satellite images, with algorithms better suited to the automated extraction of information on these data. Nowadays, deep learning algorithms have demonstrated extraordinary performance in several areas, including image processing with convolutional neural networks (CNNs). Recent works applying CNNs on VHR satellite imagery reports much higher success rates than other state-of-the-art approaches. However, these works highlighted the necessity of building a reference dataset of annotated images large enough to train the CNNs. In addition, the specificities of satellite data require more research on transfer learning techniques or the adaptation of existing CNN architectures [5],[6].


The platform uses a GeoServer for the visualization and storage of the VHR images in 8 bits format and 4-bands (RGB-NIR) that are enhanced (stretching radiometric dynamics) and orthorectified with sufficient precision. The images are displayed and can be annotated through a taxonomy browser that supports a taxonomy for land covers and another for objects. The GeoImageNet components resort to multiple APIs. This includes standard WPS process servers, middleware (proxy, permissions, authentication, application registries), and dataset handlers. Sets of patches extracted from the dataset will be available for download by invited users. A deep learning model API will also allow these users to upload their models and benchmark their results. Validated annotations (with their corresponding georeferences) and models will be available for public download.

By providing both data and benchmarking services, GeoImageNet aims to reduce the technical barrier to scientific discovery in remote sensing and artificial intelligence applications. The vision behind the platform is to create an innovation pipeline, where researchers develop new algorithms and models, which are then integrated into GeoImageNet and can be quickly used by non-specialists for their own applications, thus facilitating transdisciplinary collaborations.

References:
[1] Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A. and Fei-Fei,L. (2015) ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision.
[2] Deng, J., Russakovsky, O., Krause,J., Bernstein, M., Berg, A., Fei-Fei, L. (2014) Scalable multi-label annotation. ACM conference on human factors in computing (CHI).
[3] Russakovsky, O. and Fei-Fei,L. (2010) Attribute Learning in Large-scale Datasets. Proceedings of the 12th European Conference of Computer Vision (ECCV), 1st International Workshop on Parts and Attributes.
[4] Deng, J., Dong, W.,Socher, R., Li,L.-J., Li, K. and Fei-Fei,L. (2009) ImageNet: A Large-Scale Hierarchical Image Database. IEEE Computer Vision and Pattern Recognition (CVPR).
[5] Van Etten, A. (2018), You Only Look Twice: Rapide Multi-Scale Object Detection in Satellite Imagery, https://arxiv.org/pdf/1805.09512.pdf
[6] Wang K., Zhang, D., Li, Y., Zhang,R. and Lin, L. (2017). Cost-Effective Active Learning for Deep Image Classification.IEEE Transactions on Circuits and Systems for Video Technology ( Volume: 27 , Issue: 12).

Area C - Board 309

The geometric documentation of a cultural heritage site is demanding task due to the complex morphology of the objects as well as the need for large scale performance. The last years, more and more researchers have used new laser technologies and sophisticated computer vision techniques, mainly for terrestrial applications, with a complementary use of land surveying measurements. Although such technologies and processes provide efficiency, they are time-consuming with high cost for big covered areas with difficult accessibility. Aerial laser scanning (ALS/LIDAR) can contribute to the extraction of high-level information from huge cultural heritage sites for both structural and historical purposes.
Compared to traditional photogrammetry, GPS/GNSS measurements and land surveying, ALS/LIDAR provide: 1) homogenous accuracy, 2) fast acquisition of massive data (point clouds) and processing, 3) minimum human dependence, 4) weather/light independence, 5) canopy and vegetation penetration through LIDAR pulses, 6) higher data density (very dense point clouds), 7) additional information such intensity, multiple returns, etc., and 8) cost-effectiveness for big covered areas. However, several research approaches have been proposed the recent years that fuse ALS/LIDAR and high resolution aerial imagery to exploit both, the good geometric quality of ALS/LIDAR and the spectral image information from aerial imagery, for the calculation of the volume of excavations, object detection, feature extraction, etc.
The wider area of the Sanctuary of Zeus at Mountain Lykaion is considered, among others, as one of the most important cultural heritage sites in Greece. This study aims to the aerial coverage of this area, that is, 117. 85 km2, through ALS/LIDAR and RGB high resolution imagery. At first step, the ALS/LIDAR point clouds and the RGB imagery were simultaneously acquired over the area of interest. The adjustment of neighboring ALS/LIDAR strips was firstly performed through strip alignment and then a georeference process using ground control points (GCPs) and check points (CPs) was carried out afterwards. The ALS/LIDAR point clouds were classified to extract bare-earth points. The bare-earth points were used to extract the corresponding Digital Terrain Model (DTM) of the area of interest. Also a fully automatic aerial triangulation of the aerial imagery was conducted using GPS/INS data. Finally, orthoimages were generated using the LIDAR/DTM. Also, the corresponding coloured ALS/LIDAR were created using the generated orthoimages. Point density varied considerably over the whole block depending on the ALS/LIDAR strip overlap, i.e., 5 points/m2 and 40 points/m2 for regions covered by only one strip and more than one strip respectively. Multiple echoes and intensities were also recorded. The ground sample distance (GSD) of the orthoimages was 10 cm. The results demonstrate the utility and the functionality of ALS/LIDAR point clouds fused by high resolution aerial imagery for high-level 3D geometric documentation of huge cultural heritage sites

Area C - Board 310

We present an innovative healthcare-oriented technology based on Earth Observation satellites for the real-time monitoring of solar radiation, operating in different spectral bands (UV-A, UV-B, visible and infrared) and enabling mobile apps that support personal photoprotection, vitamin D sufficiency and skin photoageing prevention by considering also dermocosmetic products application. The presented technology relies on the simultaneous elaboration of several geostationary and LEO Earth Observation satellites, deeply using cloud platforms and Big Data Analytics techniques to ensure the elaboration in near real-time both of the ground-level multi-spectral solar radiation and of its effects on the skin of each end-user. The system is based on a reliable atmospheric radiative transfer modelling already validated thanks to the inter-comparison with respect to high-accuracy spectroradiometers at ground.

The presented integrated digital system is aimed to support people in maximizing benefits (e.g. vitamin D synthesis) and minimising the risks associated to solar radiation exposure (e.g. erythema and skin cancer, as well as skin ageing).

Area C - Board 311

Land use and land cover change (LULCC) are found to be two of the most important influencing factors for local and regional climate and can even affect large-scale atmospheric processes. Since the Earth’s surface is still undergoing major modifications, it is of paramount interest in regional climate modelling to address this complex research topic in a sufficient way. In order to capture the impact of land cover on climate, land surface models (LSMs) in regional climate models (RCMs) are representing surface processes such as surface energy balance, water balance, and carbon exchange which further affect various other components of the climate system. However, the land surface representations vary significantly among currently used LSMs which results from differently processed and entered land cover information.

The overall objective of the LUCAS (Land Use and Climate Across Scales) project is to investigate the biophysical impacts of LULCC on regional climate in Europe. LUCAS is an endorsed Flagship Pilot Study initiated by the European branch of the regional climate CORDEX community, operating under the auspices of the World Climate Research Program (WCRP). In LUCAS, for the first time an ensemble of RCMs is used in coordinated experiments including land use changes. This requires dynamic land use information in appropriate high spatial resolution and consistent over long time periods. In this study, we investigate the potential of machine learning algorithms for providing dynamic land use information for RCMs. As recent studies have demonstrated, machine learning algorithms in combination with GIS and remote sensing satellite data make a powerful tool in terms of resolving the nearly static properties of currently used land cover datasets.

The setup of our workflow and first results of how to generate a dynamically changing land use dataset will be presented, following recent approaches where machine learning algorithms are getting increasingly important in the field of modelling land cover dynamics. A central feature of the method is the combination of a cellular automaton with an artificial neural network (ANN) to create the transition rules for the different land cover and land use classes. Furthermore, the potential of integrating the combined model with a system dynamics approach is addressed. We choose Eastern Europe, which underwent significant land use changes in the recent past, as a first case study region. The high resolution satellite data and added value products provided by the Copernicus Data and Information Access Services (DIAS) can be utilized to train the ANN and further to validate the results of the conducted modelling.

It is expected that the application of the presented method will provide an important interface between latest advancements in land cover observations, new land use modelling approaches and regional climate modelling, and that the resulting products will increase accuracy and reliability of RCMs and improve the understanding of the impacts of LULCC on regional climate

Area C - Board 312

Polarimetric synthetic aperture radar (PolSAR) can obtain much more information about the land covers and targets than single polarized SAR systems, and it is increasingly used in various remote sensing applications, such as change detection, urban planning, biomass estimation, etc. Among them, land use/land cover (LULC) classification is of the key importance, thus many supervised and unsupervised classification methods have been developed. Feature extraction and classification techniques are the two essential elements which determine the classification result. In this paper, we focus on the feature factor since that some suitable features may result in a correct classification, even if only a simple classifier is used; in contrast, it could be quite difficult to achieve a satisfactory result without well-selected features, even though a more advanced classifier is employed.

In the past few decades, many physical features which reveal specific scattering mechanisms have been obtained based on various polarimetric decomposition methods, including H/A/alpha, Freeman-Durden, Yamaguchi, Neumann, Huynen, and so on. However, a single feature or feature set may cause misclassifications in cases where different land covers have the same scattering mechanism and where the same class of land cover incur different scattering mechanisms. For example, the buildings not aligned orthogonal to radar line of sight (namely oriented buildings) also cause cross-polarized scattering, which is mainly generated by vegetation. In the original decomposition methods, these two different land covers cannot be separated, leading to serious confusion in the classification maps. Some newly developed decomposition methods can overcome the problem of volume scattering overestimation in oriented building areas to some extent, thereby distinguishing the oriented buildings from natural areas. However, the orthometric and oriented buildings are usually assigned to only one class in the classification result. In fact, it is also necessary and meaningful to separate these two kinds of buildings for urban planning or other applications. Since orthometric and oriented buildings have different scattering characteristics, some kind of new scattering features derived from polarimetric decomposition are needed for more detailed and accurate classification results.

It is shown that the PolSAR classification performance can be improved by combining multiple features together. However, various features have information redundancies, which may impact the performance of the classifier. In addition, simple combination of the features can increase the computation complexity. Therefore, there is a need for dimension reduction which can reduce the redundancy among various features and can also preserve the discriminative information for subsequent classification. Generally, there are two ways for dimension reduction, i.e., feature selection and feature extraction. Feature selection means selecting an optimal feature subset from all the features, and the common techniques include random forest, genetic algorithm, etc. In this paper, we investigate feature extraction which transforms the original feature space to a lower dimensional subspace, with most of the salient information preserved. There are many linear and nonlinear techniques for feature extraction and dimension reduction, including principal component analysis (PCA), independent component analysis (ICA), linear discriminated analysis (LDA), locally linear embedding (LLE), Laplacian eigenmaps (LEs), etc. However, these methods only deal with matrix, therefore, the original three-dimensional feature cubic (image row × image column × feature dimension) has to be rearranged into a matrix (pixel number × feature dimension). That is to say, the image is vectorized, then the spatial relationship among neighboring pixels is neglected when feature extraction is implemented.

In recent years, tensor which can represent high-order data has drawn much attention in various areas, and it has been applied in hyperspectral image analysis, computer vision, and so on. Based on multilinear algebra, many tensor subspace learning algorithms have been developed, and some studies have constructed tensor-based frameworks for dimension reduction. In the field of PolSAR image interpretation, Tao Mingliang incorporates the tensor decomposition techniques and ICA to obtain an intrinsic feature set from the original feature tensor data. This method does not unfold the tensor into a 2-D matrix and thus preserves the spatial information of the PolSAR data when executing feature extraction. Based on the extracted feature set, some simple classifier is applied to obtain the final result. However, in this algorithm, the tensor-based feature extraction is an unsupervised procedure, that is to say, it assumes no prior information on the input data. In fact, the discriminative information from the given training samples should be considered in tensor decomposition, so that the distance between the extracted feature set of different land covers can be maximized, leading to more accurate classification results.

To make the best of the information of training samples while extracting the intrinsic feature set, this paper proposes a supervised manifold-learning algorithm for PolSAR feature extraction, using tensor discriminative locality alignment (TDLA). In addition, to enhance the separability of orthometric and oriented buildings, we adopt an extended four-component model-based decomposition method to obtain different scattering features, as a part of the original feature set. This decomposition method proposes a new cross scattering model for urban areas, which is concentrated on the cross scattering from oriented buildings, making it can be separated from orthometric buildings and natural areas. The entire procedure of the proposed PolSAR classification method can be listed as follows:
1. A wide range of informative polarimetric features are obtained by various decomposition methods or directly from the covariance matrix, forming a third-order feature tensor. In total, 48 features are obtained, including 5 scattering features generated from the new decomposition with cross scattering.
2. The input data of the algorithm is a set of training samples and their class labels. For each training sample, the features of the current pixel and its four direct neighboring pixels are used. The feature vectors of 5 pixels are combined together to constitute a 48×5 feature matrix.
3. Based on the training data, the TDLA algorithm is employed to obtain a multilinear transformation from the original 48×5 feature space to the reduced feature space, i.e., a 1×3 feature vector.
4. For each pixel of the PolSAR image, the extracted feature in reduced feature space (1×3 vector) is achieved by the optimized multilinear transformation.
5. Using the extracted feature, the SVM classifier is applied to get the final classification result.

In the proposed method, the tensor representation helps to represent the pixel’s spectral-spatial feature, so it can better preserve the spatial information of the PolSAR data. In addition, the discriminability of classes for classification is preserved. In the experiment, we use the Radarsat-2 PolSAR data over San Francisco for assessment. It shows that the classification accuracy of the proposed method is at least 6% higher than that of the methods which apply traditional feature extraction and dimension reduction frameworks.

Area C - Board 313

The Soil Moisture and Ocean Salinity (SMOS) satellite is an interferometric passive radiometer launched by ESA in 2009. After three year acquiring data, in 2012, the ESA Support to Science Element « SMOS + Neural Network » project started to study the use of machine learning techniques, in particular neural networks (NNs), to process SMOS data to estimate soil moisture. Since then, several applications have been studied and successfully implemented operationally in the framework of different subsequent projects. All of these projects have in common the use of neural networks to exploit the synergies from multi-sources data bases containing data from passive microwave radiometers at different frequencies, active microwave scatterometers, optical/infrared reflectivities, numerical weather prediction models, in situ measurements and static maps of soil and vegetation properties.

Some of the applications that will be discussed in this presentation are:

(i) The use of NNs to take advantage of the synergy of SMOS and other sensors such as the Advanced Scatterometer (ASCAT, active microwaves) and MODIS (visible and infra red) to improve the estimations of soil moisture content (Rodriguez-Fernandez et al. 2015).
(ii) NNs have also been successfully used to construct long time series of soil moisture from the Advanced Microwave Scanning Radiometer - Earth Observing System (AMSR-E) and SMOS. A NN with input data from ASMR-E observations and SMOS soil moisture as reference for the training was used to construct a dataset sharing a similar climatology and without a significant bias with respect to SMOS soil moisture (Rodriguez-Fernandez et al. 2016).
(iii) Using data from SMOS and soil temperature from model simulations by the European Centre for Medium-Range Weather Forecasts (ECMWF) it has been possible to implement a Near-Real-Time soil moisture product that is distributed by ESA and EUMETSAT (Rodriguez-Fernandez et al. 2017a).
(iv) Simulated soil moisture fields by ECMWF have also been used as training dataset to compute a specific soil moisture dataset for data assimilation into numerical weather prediction models ECMWF forecasts system (Rodriguez-Fernandez et al. 2017b).

-Rodriguez-Fernandez, NJ, Aires F., Richaume, P., Kerr, Y. Prigent, C,, Kolassa, J., Cabot F.m Jimenez, C., Mahmoodi, A., Drusch, M. (2015) Soil moisture retrieval using neural networks: application to SMOS, IEEE Transactions on Geoscience and Remote Sensing, 53(11), 5991-6007
-Rodríguez-Fernández, N. J., Kerr, Y. H., van der Schalie, R., Al-Yaari, A., Wigneron, J. P., de Jeu, R. & Drusch, M. (2016). Long Term Global Surface Soil Moisture Fields Using an SMOS-Trained Neural Network Applied to AMSR-E Data. Remote Sensing, 8(11), 959
-Rodríguez-Fernández, N. J., Muñoz-Sabater, J.; Richaume, P.; Albergel, C.; de Rosnay, P. & Kerr, Y. H. Evaluation of the SMOS Near-Real-Time soil moisture, Hydrology and Earth System Sciences, 2017a, 21, 5201-5216
-Rodriguez-Fernandez, NJ, de Rosnay, P., Albergel, C, Aires, F., Prigent, C., Richaume, P., Kerr, Y., Muñoz-Sabater, J., SMOS Neural Network Soil Moisture Data Assimilation, 2017b, ECMWF report for ESA/ESRIN contract 4000101703/10/NL/FF/fk.

Area C - Board 314

Satellite optical imagery can provide significant amount of information about marine ecosystems. The recorded imagery are commonly associated with geophysical data products, such as near-surface concentrations of chlorophyll-a (Chl-a). This data not only can be incorporated into a multitude of environmental research streams, but also enable us to build automated monitoring tools to assess the state of aquatic environments. In this work, we aim to build an anomaly detection model capable of identifying anomalous water-quality patterns. We envision such a model can be integrated into a variety of environmental and remote sensing applications. This effort is contemporary with recent data analytics and image processing breakthroughs empowered by modern Artificial Neural Networks (ANNs) commonly known as Deep Learning (DL).

We take advantage of a class of ANNs namely, Autoencoders (AE). At the very high level an AE is a data encoder and decoder cascaded back to back. The encoder receives input data namely atmospherically corrected Sentinel-2 and Landsat-8 products and projects the data onto a lower dimensional space through a series of nonlinear transformations. This is followed by a decoder componenet that obtains the encoder output and attempts to reconstruct back the input. Once the AE is trained on data with no anomalies, it learns nonlinear bases that account for the maximum variations in the normal data. As a result, it is expected that a properly trained AE should be able to compress and reconstruct back the non-anomalous data accurately. In contrast input data containing anomalies should suffer from high reconstruction errors. We implement our AE with the Convolutional Neural Network (CNN) encoder and decoder. In a CNN layer multiple convolution kernels inspect the input for presence or absence of spatial features. These filters are learned through a training process with the goal of finding features that are optimized to accomplish a particular task. The latter lifts the burden of requiring human expertise for feature engineering, which is costly, biased, and often inaccurate.

Here, we first train multiple Convolutional AEs (CAEs) using multispectral images of Sentinel-2/Landsat 8 images of San Francisco Bay area. Our input images consist of a subset of available Remote Sensing Reflectance (Rrs) products. Then, during the inference stage, we feed our trained AE with both anomalous and normal images. We expect AE to return large errors at the locations of potential anomalies. Our preliminary results corroborate this hypothesis by demonstrating a correlation between the reconstruction errors and geophysical anomaly indicators such as image-derived and measured Chl-a levels. For a more comprehensive understanding of the performance of our proposed CAE, we plan to broaden the scope of this work to other sites. In addition, the performance of neural networks draws largely on the architecture and tuning of the hyper-parameters. Therefore, we envision one avenue of further research would include exploring the design space of CAEs for the task at hand.

Area C - Board 315

Keywords: Sentinel – 1 SAR, target detection, maritime surveilance, VTMIS, radar, AIS, data fusion

Abstract:
This paper describes some experimental results of sea target detection by Sentinel – 1 SAR imagery and in-situ validation by using terrestrial sensors (Radar and AIS) of the Vessel Traffic Management and Information System (VTMIS) of Bulgaria. Images of Western Black Sea region, provided by ESA Sentinels Scientific Data Hub for the period of April 1-st till May 30, 2018 are processed for automatic target detection. Detected targets are then compared with Radar and AIS data, collected by the Bulgarian port infrastructure company and its VTMIS. Database include lists of integrated (Radar and AIS) targets, Radar targets only (with no AIS transponder, small non-SOLAS vessels) and AIS targets only (outside coverage area of Radar surveillance subsystem).

Image processing
First the images are pre-processed to reduce level of noise and to improve signal-to-clutter ratio. One of the most common approaches for image processing is implemented by using so called “Moving (sliding) window”. Moving (sliding) window image processing is the kernel window operation used as image filter. Two types of moving window image processing are used: Neighbourhood-averaging filters and Median filters.
Neighbourhood-averaging filters replace the value of current pixel by a weighted-average of the pixels covered by the window (the weights are non-negative with the highest weight on the current pixel). If all the weights are equal then this is a mean, or linear filter;
Median filters replace the value of current pixel by the median of its neighbours. This usually takes time to implement due to the need for sorting of the values. However, this method successfully removes the speckle noise while preserving edges of the image.
The size of the window depends on the purpose, how many minimum samples the kernel function needs and the object resolution of the image. It is possible however to use variations of the above mentioned filters with 5X5, 7X7 or 9X9 pixel areas, iteration filtration procedure or differential procedure where image and noise are filtered separately.
Fig. 1. shows images of a Sentinel – 1 target, processed with linear and median filters with 5X5 and 7X7 pixel areas. Type of processing and size of pixel areas influent on the shape and size of the image. Median filter for example keeps the shape better than linear one which practically smears it more or less depending on the size of the pixel area.

Fig. 1. Sentinel-1 SAR Image Processing

Fig. 2. Sentinel – 1 SAR Target Detection

Target detection
To make appropriate target detection after the pre-processing, the the CFAR detector is applied to adapt the threshold constantly to the mean value of the clutter return. The threshold is derived from the mean level, modified by a multiplier. A guard interval is included as well in order to avoid targets that are close each other to affect noise estimation. The Neighbourhood averaging CFAR (NA-CFAR) detector sets the threshold value to provide near optimum clutter suppression by following the average level of radar signal and thus reducing the clutter to a level, close to the noise response.
Fig. 2 represents results of the CFAR detector performance combined with linear and median filtration respectively. It is also possible to see some parameters of detected targets as the length and orientation.
Validation
To validate Sentinel – 1 SAR target detection the system data from radar and AIS sensors of the Vessel Traffic Management and Information System of Bulgaria (VTMIS) is used. The data from all sensors is recorded on two servers installed in Traffic control centers. Servers keep the archive for a period of 3 months. It consists of AIS data provided by all AIS base stations, raw radar pictures, provided by coastal surveillance radars as well as system information, including GMDSS communications, meteo data, etc. The archive can be replayed by using a specialized software, developed by the provider. Fig. 4 illustrates part of the Transas Navi Harbour NH4.55 VTMIS player, representing the traffic picture on May 14-th 2018.

Fig. 3. NH4.55 VTMIS Player

Fig. 4. Sentinel – 1 SAR Targets validated

By using the player a list of all targets observed in the coverage area of the VTMIS can be extracted in an appropriate format for in-situ validation of Sentinel-1 SAR data. For 14-th of May 2018 for example Sentinel-1 SAR image combined with AIS and Radar targets data from Bulgarian VTMIS is shown on fig. 3. Within the 25 seconds duration of Sentinel-1 swat 5 consecutive records with 5 seconds time interval are selected and visualized from the VTMIS database. Results are shown on fig. 4.
Images of three validated targets are shown on fig. 5 – at anchor on the left, moving to North and to South respectively. Differences in positions provided by Sentinel-1 SAR and VTMIS could help calculation of their velocity vectors (i.e. course and speed).

Figure 5. Sentinel – 1 SAR Targets validated

Acknowledgement
This work was supported by the Research Project “EXpanded TRAffic Surveillance and Deep Sea Oil Pollution Observation – ExTra S-D-S-O-P-O” under the ESA contract No:4000122601/17/NL/SC with Nikola Vaptsarov Naval Academy – Varna, Bulgaria.

Area C - Board 316

Data mining techniques allow scientists to extract and evaluate efficiently tendencies from large databases. In that context, the purpose of this study was to explore the potential of Data Mining and Machine Learning (ML) methods to assess the validity of altimetry measurements over ocean and compare their performances with the historical editing criteria.
Currently, the detection of spurious data in radar altimetry measurements relies on a legacy data editing method consisting in checking whether the value of several altimetric parameters is outside a validity domain defined by minimum and maximum thresholds. This historical editing method is described in the data user manuals and in the CALVAL reports of altimetric missions. It has been developed and used by the community of experts over the last 20 years.
Our study considered mainly clustering and classification techniques to assess the validity of 1 Hz SLA (Sea Level Anomalies) from 1 cycle of standard JASON-3 GDR data. A representative composite repeat cycle of data was created from data regularly sampled over one year, and the entire data set was manually annotated to produce a training and validation dataset .Unsupervised and supervised learning techniques were tested to compare their respective performances: PCA (Principal Component Analysis), Decision Trees, Random Forests, Logistic Regression, Support Vector Machines (SVM), and Naïve Bayes.
Filtering, standardization, principal component analysis and segmentation where applied to select discriminating parameters and to build reliable classifiers.
Finally, measurements validity was determined from their classification in specific groups. Confusion matrices, ROC curves and other performance indicators such as precision, recall, F-score were produced for validation purposes in order to compare the performances of the standard “editing” criterion with the ML methods. The first conclusions of our work highlight a correct classification with unsupervised learning models as well as the excellent performances of the supervised models. The best performance was obtained with the Random Forests, with 99.3% of data correctly classified.

Area C - Board 317

Assuming that an intensity SAR image is modeled by a self similar random process [1] we propose a possible rather simple analysis scheme for the detection of oils spills in open sea. Moreover as this is essentially a smoothness or complexity analysis technique of the image it can be potentially applied to the recognition and classification of other patterns.
To explode notions related to self similarity and fractals related to SAR image processing (and optical images) is not completely new. In e.g. [3,4] computing an approximate Box/Hausdorff fractal dimension of the neighborhood of each pixel some possible algorithms for oil spill detection are proposed. Indeed, since "fractal" is a more or less vague term, several underlying models with varying results and computational complexity can be proposed. As suggested in [3,4] box counting techniques have a rather good performance in solving this detection problem. However this may be not the case from the the point of view of the required computational time to perform this task. In contrast, Fourier analysis methods or similar as wavelet analysis [1] are potentially much simpler and fast since there exist several optimal FFT algorithms. Here, among all possible models of self similar random fields to model the image we propose the class of the stationary (or locally stationary) 1/f processes [1]. These fields were introduced by A. Kolmogrov in a first statistical study of turbulent flows. An X={X(p),p} random field of these class, with p in IR², has an spectral density of the form (or asymptotically near the frequency ω=0) [5]:

Sx(ω)= C/ ||ω||^α (1) .

The exponent α gives a quantitative measure of the the correlation of the random field. For greater α, X is more correlated and in contrast, for α ->0, X is nearly white noise. The parameter α is linearly related to the so called Hurst parameter [1], which measures the self similarity of the random field . Finally, at least in the Gaussian case, α is also related to the Hausdorff dimension of the graph of X and the roughness of it .
In this context, we shall assume that the portion of image containing the oil spill fluid will be approximately behaving as a 1/f random field with a certain parameter α. Waves are supposed to be attenuated in contact in the presence of oil, which under suitable conditions, will traduce in a smoother and highly correlated surface in comparison to non contaminated open sea water. The proposed detection scheme is basically to test if the possible oil spill area follows model (1) for some α. To do this, given any finite energy filter h₁(p) with a Fourier transform H₁(ω) we consider a scaled version of it h₂(p):=h₁(p/2) and if X is our image/field we obtain two filtered (convoluted) versions of it, namely:
Y₁(p)=(h₁*X)(p) and Y₂(p)=(h₂*X)(p).

The corresponding spectral densities then are given by :

Sy₁(ω)=|H₁(ω)|^2 Sx(ω) and

Sy₂(ω)=16|H₁(2ω)|^2 Sx(ω).

Assuming stationarity [5] and after some Fourier domain calculations, one gets that if X has indeed an spectral density as (1) then the variances σ₁ and σ₂ of Y₁ and Y₂ respectively, are related by:

σ₂²=2^(α+2) σ₁² (2).

Equation (2) is the key to obtain estimates for α. But furthermore, we can prove a kind of reciprocal of this result: If an equation like (2) holds for some fixed α and any finite energy filter then the spectral density Sx has to be as (1). In practice, α is unknown and one can only test with a finite number of filters. However, based in these calculations, in this work we propose that for a finite suitable parametrized family of filters h(s,p) , s=s₁...sn and corresponding estimates of α(s) we can design a rather simple first warning detection scheme which in a first stage heavily relies only on linear, matrix and spectral numerical operations. Here we present some practical examples with real images. We note that α(s,p) defines in some way a transformation or map of the original X. To illustrate the idea, if for certain fixed values s₁, s₂, s₃ we generate an RGB auxiliary image assigning to each color component a value linearly related to the corresponding estimates of α(s) for each filter. In the zones of the image where these values are, ideally, equal the related RGB intensities must be equal as well generating a nearly dark grey flat area where α(s) equals a constant. Obviously if more filters (n≥3) are involved more accurate the following decision will become. The second stage (classification and decision) takes the transformed image α(s,p) as an input into a combination of a neural network and a classical statistical hypothesis [2] testing previously trained with several images samples and its respective transforms α(s,p). In this work we present some promising experimental results with this technique.

References:
[1] Cohen S., Istas J., Fractional Fields and Applications, Springer, 2013.

[2] Duda R., Hart P., and Stork D. Pattern Classification, Wiley, 2nd. edition. 2000.

[3] Marghany M., Hashim M., "Fractal dimension algorithm for detection Oil Spills using RADARSAT_1 SAR", in Computational Science and its applications ICCSA 2007. Lecture Notes in Computer Science, Vol. 4705. Springer, 2007.

[4] Marghany M., Hashim M. "Discrimination between oil spill and look alike using Fractal dimension algorithm form RADARSAT_1 SAR and AIRSAR/POLSAR data", International Journal of Physical Sciences , 6(7), pp. 1711-1719, 2011.

[5] Rosenblatt M., Stationary Sequences and Random Fields, Birkhäuser, 1985.

Area C - Board 318

Abstract: In single-pass InSAR system, the water bodies and road areas often have relatively low coherence because of their scattering characteristics. The coherence of shadow areas is also very low for receiving no radar echos. Layover areas do not necessarily have low coherence, but their phase measurement might abnormal because of signals overlying from many targets with the same range. In conventional InSAR processing for generating DEM, phase unwrapping over low coherence areas and layover areas can take a considerate amount of time and memory, and the unwrapped results are generally unreliable and may be with lots of errors. Hence, there is a need to detect water bodies, shadow areas, roads and layover areas. Once these areas are detected and classified, the phase unwrapping can carried out by avoiding these areas to reduce unwrapped phase errors largely. Then, different strategies can be chosen to deal with these areas. Therefore, DEM accuracy can be improved in a large extent. For this purpose, we present a new algorithm to realize intelligent classification for these areas. It includes three parts: multiple features extraction and fusion (MFEF), a new deep learning network with multi-path refinement and global information, and a post-processing to reduce mis-classification pixels and improve the continuity.  
Firstly, we present a MFEF method to extract multiple features from SAR images and fuse them with weights. In this stage, we adopt Gabor transform for SAR images with Gabor convolution filter to extract local features, GLGCM (Gray Level Gradient Co-occurrence Matrix) for SAR images using Sobel operator with four directions to extract texture features, HOG analysis to extract spatial characteristics, and shape filter to extract geometry features of different targets. Then, we set the weights for four features according to the given rules [What are they?]to realize feature fusion.  
   To realize end-to-end classification, we introduce deep learning, which shows better performance in classification and detection for optical image and video image than the tradition methods. In this paper, we present a new deep learning structure. It is based on RefineNet network which is one of the most popular semantic segmentation deep learning net structures. The RefineNet adopts a multi-path refinement network that explicitly exploits all the information available along down-sampling process to enable high-resolution prediction using residual connections. It achieves much better results in the several public data sets than the majority of existing algorithms. However, we discover its problems of misclassification for the same kind and lack of enough use [Not clear what you meant here]for global information. Therefore, we introduce the spatial pyramid pooling to utilize more context information, and we adopt the visual attention mechanism to use the global information fully to resolve the misclassification in the same kind. In addition, we improve the loss function to simplify the network and generate the good generalization capability. Finally, we present the new deep learning network structure. 
  In order to restrain the speckle noise of SAR images and improve the continuity of the classification results, we introduce the method of spatial regularization with graph-cut. In the paper, the strategy for graph-cut energy minimization is established, and the energy function is given. The energy function contains two parts: the first term is a penalty for the sample corresponding to the label, the second term is a smoothing penalty between a sample and its adjacent pixels to measure spatial coherency. Through graph-cut-based spatial regularization, the connectivity can be improved, and the misclassified points can be reduced. 
  To validate the algorithm, the millimeter wave airborne millimeter wave single-pass InSAR data are tested. In this experiment, 2200 SAR images with size of 500×500, the corresponding coherence maps and InSAR phases are all utilized for the training of the proposed algorithm. 400 SAR images without being trained are tested. According to the classification results, the accuracy for shadow areas and water bodies is about 85%, the accuracy for roads and layover areas is about 81% and 78% respectively. All of these suggest the effectiveness of the proposed algorithm for single-pass InSAR classification. Furthermore, the algorithm can also be used to realize intelligent classification of these low coherence areas and layover areas for TanDEM-X system.

Area C - Board 319

The Satellite Missions scenario, from Earth Observation to Science, to planetary rovers, is enormously changing both for the number of the deployed assets and for the acquisition capability of the embarked sensors. In fact, sensors evolve towards increasing acquisition capabilities, in terms of bands (hyperspectral in optical and full polarization in SAR), resolutions and duty cycles. The result is a considerably increased data volume to manage on-board and consequently to download to ground stations.
By using software technologies such as AI and BLOCKCHAIN and hardware improvements such as LPGPUs and DSPs, the asset can accomplish its tasks in autonomy, like detecting fires, floods, or any other anomalous unforeseen event, and react to it with consequent tasks. For example, if a fire is detected, the satellite could re-task its activities to follow the fire and to send alarm to the ground, where the small amount of required information can be sent by different communication bands. Moreover, the satellite can ask other satellites to follow the fire after its passage or it can ask more data/information to improve its knowledge and understanding, completely in compliance with the “virtual sensor” approach.
We propose an integrated system in which hardware devices, both in space and on the edge, are structured to host a pluggable software neural network (typically specialized in processing images for classification, segmentation and masking) with related runtime environment (an inference engine). Moreover, a group of application stores (App-Stores) allows one to choose software AI algorithms based on their functionalities and deploy them to specific hardware devices (be them satellites, the cloud, or edge devices) and then retrieve results by accessing online permissioned data repositories. The overall system is secured and made safe by typical blockchain technologies (public key cryptography, peer-to-peer networking, distributed consensus management) when dealing with data integrity and freshness, information accuracy and quality (improved by peer reviewing and a rating mechanism), general identity management and account security. This system requires a specialized design of each device from a hardware and software point of view, since we foresee also a “federated” working mode, in which devices interact one another: an inverse hierarchical protection domain (also known as protection rings) shall be implemented so that the pluggable neural network is segregated inside a composition of rings, each responsible of implementing a functionality: system management at ring-0, federation management at ring-1, business rules (smart contracts) at ring-2 and application (the actual AI component) at ring-3.

Area C - Board 320

Rainfall-triggered landslides are catastrophic geo-hazards that often take place after long-duration and/or intense precipitation. The catastrophic consequences of these landslides are aggravated in highly-populated areas and locations where critical infrastructures such as roads and pipelines traverse. Therefore, a system that can predict landslides and issues early warning is very appealing. Besides rain as the triggering factor, various controlling factors make a natural or engineered slope susceptible to landslides. These factors are linked with, among others, topography, geology and geomorphology of slopes, soil type, ambient temperature, surface radiation, vegetation, soil moisture, land use and land cover. In this study, we present the results of a practical research study that has been carried out at Deltares, The Netherlands for forecasting rainfall induced landslides. To that end, we have set up a framework that combines geo-engineering, remote sensing, hydrology with machine learning to predict the onset of landslides under the effect of precipitation (Figure 1).

Figure 1. Landslide forecasting framework of this study.

In this data-driven approach, Machine Learning (ML) methods are used to predict landslides by exploiting multiple Earth observation datasets, including rainfall data (e.g. TRMM 3B42 and PERSIANN-CDR) and DEMs (e.g. SRTM1 and ALOS) and the Global Landslide Catalogue. A detailed inventory of 10,988 landslides (Figure 2) at a global level is built out of which cases with better location accuracy (~ 4500 landslide cases) in addition to the same number of non-landslide cases (in total ~ 9000 cases) are used to train supervised machine learning classification algorithms.

Figure 2. Landslide points used to create the landslide database.

At this stage, binary classification methods such as logistic regression were used to distinguish landslides and non-landslide cases. To train the ML model, 11 example sets (E0 to E10) with different combination of triggering and controlling factors (model features) were built. Figure 3 shows the combination of controlling and triggering features (x0 to x5) used in this study.

Figure 3. Features used for training the machine learning algorithms.

The example sets were split into training (67%) and test (33%) sets which then were used for training and assessing the logistic regression model. The accuracy of the logistic regression model in form of Receiver Operating Characteristic (ROC) curves and the associated Area Under Cut (AUC) is illustrated in Figure 4.

Figure 4. Accuracy of logistic regression model in classifying landslides and non-landslides.

From the results of logistic regression model, it can be suggested that the rainfall-induced landslides could be distinguished from non-landslides cases with an accuracy higher than 88%. Although the database and forecasting framework that were reported in this study is at its initial stage, the results of the study shows that such a framework, with enhanced datasets, can be used for forecasting rainfall-induced landslides and landslide early warning systems.

Area C - Board 321