Climate and air quality prediction are sources of environmental information necessary for adaptation to and mitigation of environmental hazards. Process-based global models initialized with the best initial-condition information are the main tool used to generate these predictions. Intensive research is performed to improve these systems, including the merge with approaches that use artificial intelligence to make the most of these models and the increasing number of environmental observations. This research makes a broad use of high-performance computers, which need to be used efficiently in a fast-changing scene of heterogeneous machine architectures. It also requires addressing big data problems as the data sources are varied, complex, and require fast and trustworthy access channels. Both, process-based modelling and efficient use of computers and data, require multidisciplinary teams that bring together the required expertise to make adequate progress in a such a complex scene where scientists have traditionally worked in niches with little and difficult communication. However, the scene becomes even more complex when the service aspect is taking into account. The process that takes large-scale predictions and converts them into action-oriented information includes stages that go from the understanding of the relevant processes for their efficient communication to vulnerable communities, the evaluation of the forecast system, the quality control of the data and services provided, and the adaptation of the information to the specific sector, which might be local in some instances, among many others. Only an appropriate user-engagement framework and the continuous connection between scientists and communication experts guarantee that the information can be displayed in a form that can be employed in real-life decision making. Some of these aspects will be addressed in this talk, using examples of the research performed at the Barcelona Supercomputing Center, focusing on how current trans-disciplinary research provided by physical, engineering and social science experts can and should be transferred to operational environments to develop action-oriented services.

Remote sensors are being developed at an enormous speed, and acquisition of hyperspectral image data (HSI), with up to hundreds of spectral bands over a given spatial area, is much more affordable nowadays. However, we still struggle with the lack of ground-truth data which is used for training supervised HSI classification engines. Therefore, applying machine learning-powered techniques for segmenting HSI (i.e., for assigning a class label to each pixel in an input hyperspectral data) is extremely challenging in practice, where we need to deal with emerging use cases and real-life applications. In such scenarios, we rarely possess a representative training set which is balanced, high-quality and sufficiently large to effectively train high-capacity learners, especially in the context of deep neural networks. Therefore, although hyperspectral images have been already shown useful in accurate identification of a variety of materials, efficient analysis and segmentation of such imagery became a big issue in practical applications and it is currently being faced by the machine-learning and image-processing communities worldwide. In this talk – which will be divided into two main parts – we will present two approaches for dealing with limited, not representative, or even non-existing training data in the context of hyperspectral image segmentation. First, we will present various data augmentation techniques which are specifically designed for hyperspectral data. Data augmentation is a popular technique which helps improve generalization capabilities of deep neural networks, and can be perceived as implicit regularization. It plays a pivotal role in scenarios in which the amount of high-quality ground-truth data is limited, and acquiring new examples is costly or impossible. As already mentioned, this is a fairly common problem in remote sensing; especially affecting multi- and hyper-spectral imaging, where manual annotation of image data is very difficult, time-consuming, and prone to human errors, because it usually requires additional sources of reliable information about the scanned region. Therefore, automated approaches for elaborating artificial (but realistic) training examples are widely researched in the literature. In this talk, we will present data augmentation policies for generating hyperspectral training examples which are underpinned with various underlying data augmentation techniques (e.g., injecting Gaussian noise into HSI or generating HSI examples using generative adversarial networks). Our experimental study performed over the HSI benchmark datasets (Salinas Valley, Pavia University, and Indian Pines) showed that: (i) augmenting training sets (in the Monte-Carlo cross-validation setting) using our approaches helps boost the classification performance of convolutional neural networks (and other conventional supervised learners) trained using the augmented sets without sacrificing their inference speed (our deep networks operate in real time, and they are competitive with other state-of-the-art HSI segmentation techniques in terms of the segmentation accuracy), and (ii) our augmentation policies can be used to effectively balance HSI datasets which commonly include very under-represented classes. The experiments (over benchmark hyperspectral scenes – Indian Pines, Salinas Valley and Pavia University, acquired using ROSIS and AVIRIS sensors) are coupled with various visualizations and statistical tests which help understand the process of generating new training examples (e.g., in the case of generative adversarial networks, we investigate the generated data distribution and compare it with the original data distribution during the training), and verify the statistical importance of the obtained results (i.e., whether the classification performance of deep models trained using original and augmented datasets differs in the statistical sense). Importantly, we will also present how data augmentation techniques can be applied in the offline scenario (before training of deep neural nets, which is a standard procedure in deep learning) and in the novel online scenario (additional data examples are generated during the inference and the incoming sample is classified in a voting scheme of a classification ensemble). Our experiments showed that the combined data augmentations (online and offline) effectively improve generalization abilities of spectral deep networks for very small training sets – the differences are statistically important at p<0.001 (two-tailed Wilcoxon tests); we obtained the average accuracy of 95% for Salinas Valley (SV) balanced dataset, 94% for SV imbalanced, 83% for Indian Pines (IP) balanced, 76% IP imbalanced, 91% for Pavia University (PU) balanced, and 88% for PU imbalanced in the rigorous cross-validation setting. In the second part of our talk, we will show how to employ machine-learning approaches for segmenting hyperspectral imagery in emerging real-life use-cases, where the ground-truth data does not exist. Therefore, we will present various unsupervised methods which are aimed at dividing an incoming HSI into concise regions which encompass pixels of similar spectral characteristics. Our review will include not only conventional clustering algorithms (which require extracting features from HSI) but also emerging deep clustering methods which either extract deep features automatically (they are later used for clustering) or provide an end-to-end processing pipeline for deep clustering (e.g., using auto-encoders coupled with convolutional networks and clustering loss functions). Finally, we will show how to quantitatively assess the quality of such unsupervised segmentation in the wild (when the ground-truth data is missing). Our talk will be concluded with a brief discussion on the open-issues in hyperspectral image segmentation and analysis which still need to be tackled by the remote-sensing and machine-learning communities, especially in the cases where the segmentation engines are to be deployed on-board of a satellite.

Currently in its demonstration phase, the BIGMIG project, supported by the ESA Integrated Applications Promotion Programme , which focuses on the use of space-based big data to support aid and development organisations facing the challenge of preventing and managing forced migration. Artificial Intelligence based methods are used to extract previously untapped information from coupled Earth Observation and GPS data sources and form it into geospatial products which can be used to make a real difference to those unfortunate to be affected.

BIGMIG data products will empower organisations to better carry out their decision-making processes and goals by equipping them with actionable information more accurate and frequent than their usual. By providing this information in a format that meets the needs of those which can enact upon it, the BIGMIG project targets the two main driving mechanisms of forced migration: hunger and poverty. In this way, the work carried out as part of the BIGMIG project aligns with UN Sustainable Development Goals: zero hunger (SDG-1) and no poverty (SDG-2).

The BIGMIG demonstration scenario presented here aims to reduce forced migration rates in northern Mozambique by increasing the resilience of the Mozambican population through an enhancement of food security. Working with Ayuda en Acción (AeA), an organisation concerned with emergency aid and poverty reduction, and the Mundukide Foundation, a local partner implementing Ayuda en Acción’s programme, the authors are creating a system capable of automatic generation of relevant LULC and crop-related information.

While the traditional approach, based exclusively on field campaigns, has already proven short and limited for the scope to AeA’s goals in Mozambique, the BIGMIG System will make a significant difference in AeA capability through providing support in efforts to expand their region of operation, and for the design, implementation and monitoring of the development initiatives in the country. The intention is for the BIGMIG processing and classification chain to be deployed to the cloud, providing end users with access to updated information products as the raw data becomes available. End user interest in the service is strong: so far, there has been substantial investment in the form of office and fieldwork hours to ensure the service becomes operational; and, if necessary a commitment towards the cost of acquiring of very high resolution satellite data.

Two products and their underlying generation techniques are being developed during the demo phase. Both are discussed here. The first being a baseline Land Use Landcover Map (LULC) (forest, cropland, grassland and water), the second a detailed crop type map. Both products make use of Copernicus data: Sentinel-2 time series in conjunction with agricultural land parcel data recorded by the Mundukide Foundation. The agricultural land parcel data constitutes a variety of crops such as mung bean, pigeon pea, sesame and vegetable gardens recorded over several years (2015-2018).

The LULC map makes use of more traditional machine learning methods such as Random Forest (RF) and Support Vector Machines (SVMs), increasingly readily available due to their incorporation in accessible geospatial analysis platforms. For the second product, the detailed crop type map, in order to achieve sufficiently accurate predictions a different method was required.

From a remote sensing perspective classification of small-holder agricultural fields is particularly difficult in part due to their small size and spatiotemporal diversity in the perceived spectral signature. In this specific case, the resolution of the available satellite data was not sufficient to conduct an accurate classification from spatial variations alone. In addition, during the growing season when crops exhibit the largest temporal gradient in their spectral signature, the satellite data was often obscured by clouds.

The workflow used to generate the detailed crop type map makes use of powerful state-of-the-art deep learning methods capable of learning and inferring from complex, noisy, multi-dimensional datasets. This work builds on recent research in the area which uses an ensemble of convolutional and recurrent neural network layers to ingest a multi-spectral multi-temporal data stack to carry out automatic land use semantic segmentation.

The deep network architectures presented here are those which have been found to produce the most accurate crop classifications within the confines of the dataset in the area of interest (northern Mozambique). Details of the deep learning training and validation process are given and performance against various semantic segmentation accuracy metrics, on the test dataset, are compared.

The authors would like to acknowledge the support Beñat Arzadum, from Fundación Mundukide.

Data augmentation is a technique which creates new training samples through simple transformation and manipulation steps. It aims at enlarging the dataset in order to address gaps in data representation and to minimize the problem of over-fitting. This leads to improved model performance and prevents imbalanced learning. In the image classification domain, both satellite and non-satellite data have benefited from several augmentation techniques. In satellite image classification, clipping, rotating, flipping, shifting and translating are the most commonly used image transformations, whereas in the context of satellite image super-resolution, materials weren't available to form a strong opinion on which augmentation techniques were more useful.

In this abstract we evaluate the usefulness of traditional data augmentation techniques in satellite image super resolution tasks in addition to proposing a novel fusion-based data augmentation technique which can boost the accuracy of the reconstructed images. We categorized the augmentation methodologies in two groups as instance based and channel based augmentation. The former refers to the traditional augmentation techniques which increases the number of training instances while the latter is a set of additional bands to be fused with training samples to increase the dimensionality. Instance based techniques included flip, rotate, shift, zoom and translate transformations. Channel based aims to increase the dimensionality of the training sample by stacking derived channels to the visual bands such as contrast enhancement, edge detection and unsharp mask. We believe that the most relevant features to denoising, sharpening and resolution enhancement tasks might be useful to the model especially at the early stage of training.

In order to evaluate the performance, state of the art super resolution algorithms such as SRCNN and msiSRCNN are adopted as baseline to evaluate the efficacy of the proposed augmentation methodology compared with traditional augmentation techniques. Following the rule of thumb, the accuracy of the super resolution using both algorithms is quantitatively measured using Peak Signal to Noise Ratio (PSNR) metric. PSNR measures the reconstruction accuracy by comparing super-resolved image against the original ground truth high resolution image. We prove that fusion/channel based augmentation methodology outperformed traditional instance based augmentation techniques. The accuracy achieved opens a wide field of research with deep learning and data augmentation in the satellite image super resolution domain.

The European Union Satellite Centre (SatCen) is an Agency of the Council of the European Union (EU) whose mission is to support the decision making and actions of the EU in the field of Common Foreign and Security Policy by providing products and services resulting from the exploitation of relevant space assets and collateral data. SatCen is a key institution linking Space and Security as well as a primary user of satellite data with great expertise on managing and disseminating products derived by the exploitation of Earth Observation (EO) data coming from commercial providers and from governmental systems as well.
To maintain the SatCen capabilities at the state-of-the-art, appropriate Research and Innovation (R&I) activities are implemented. In particular, the SatCen RTDI (Research, Technology Development and Innovation) Unit covers different work streams within the EO data value chain (e.g. new data sources, emerging technologies, innovative EO solutions and cooperative activities with ESA and GEO). Such activities are performed in close cooperation with SatCen technical and operational teams in order to ensure that results are aligned with actual needs.
The Space and Security domain is more and more relying on effective processing of big geospatial data to provide value added products for decision-making. The exploitation of Very High Resolution (VHR) images is traditionally performed through visual analysis and annotation by expert Image Analysts but the sheer volume of available data to be processed now makes this approach impractical. The sharp growth of Artificial Intelligence (AI) based solutions for EO applications is opening the opportunity for a change of paradigm in the adoption of new disruptive technologies and analysis approach.
Artificial Intelligence, when scientifically interpreted together with the underlying physical principles, has shown its capabilities to enable the efficient exploitation of data as well as to bring new insights and predictive capabilities. The application of Deep Learning tools can support the extraction of information from huge amounts of heterogeneous data in a timely manner.
This paper presents the SatCen approach to exploit Artificial Intelligence techniques in the Space and Security domain by implementing a number of pilot services using VHR optical data.
In particular, three use cases are described:
• Detecting specific man-made facilities at country scale;
• Processing damage assessments in small areas;
• Supporting the fight against illegal fishing by automatically scanning the Mediterranean Sea.