The European Space Agency (ESA) has contracted cosine Remote Sensing to highly integrate Thermal Infrared (TIR), hyperspectral imaging and artificial intelligence technologies into a miniaturized imager able to fit a volume of less than two liters. The combination of hyperspectral VNIR, TIR and AI in such a compact envelope is unprecedented. The imager is named HyperScout-2 and as it will use the HyperScout-1 platform, that has flight heritage, as starting point to further integrate spectral channels.
HyperScout-2 will be used as in-orbit test-bed to benchmark the performance of such a miniaturized class of systems as well as to perform hands-on investigations to forecast the benefits of combining frequent measurements in the VNIR and TIR from nanosatellites, with less frequent but very accurate measurements performed by institutional satellites such as the Copernicus fleet.
HyperScout-2 will enable experimental programs to investigate the use of Artificial Intelligence (AI) for a variety of applications in the field of object detection and data inference, potentially beneficial also to the larger class satellites the Agency is developing as part of the Copernicus expansion element.
The first enabled product is the fusion of TIR imaging with hyperspectral data fully co-registered. Additional applications that will be investigated are: 1. high resolution by data fusion between hyperspectral and thermal, 2. evapotranspiration and agriculture applications, 3. thermal inertia and soil moisture, 4. Urban Heat Island, 5. Assisting search and rescue for pyroclastic flow/surge deposits, 6. Oil Spill, 7. Fire monitoring, 8. Underground fires and 9. Water quality.
The disruptive development makes use agile project management and risk mitigation approaches in order to catch the challenging schedule dictated by the main mission program. The project is targeting a flight window in the second quarter 2019, as part of the FSSCAT mission expected to complement the Copernicus Land and Marine Environment.
This contribution reports on a general description of HyperScout-2 as well as of the fast track program in which is implemented. We will also highlight the way the space asset will be exploited especially regarding the understanding of the potential of small systems deployed within small satellites constellation if integrated into ecosystems made of small and large systems.

The EO community is progressively realizing that data can be really considered as a commodity, while the real value has shifted to the capability to perform smart and effective data selection, extract their valuable part, i.e. their information content, and manage it with a prompt exploitation. Therefore, advanced functionalities, based on detection, extraction and exploitation of data information content will be the core value points in this, envisaged, paradigm shift moving focus beyond “raw” data.
Most of these new functionalities will have to manage huge amounts of raw data, implement structured and automated information refinement processes, able to react in real or near real time to single pieces of information, to touch off new tasking for improved EO acquisitions specifically targeting the detected needs. This is the way toward autonomous decision and tasking capability leveraging a synergic use of several assets in a possible future global view of a large (federated) EO system of systems.
The process of designing, implementing and demonstrating the technologies these new functionalities will be based on, is running just now and the need for practical, quick and effective tools, able to ease and support this process, is rising. The current scenario already demonstrates a relevant increase of in orbit demonstration activities in general and of structured opportunities managed by public (ESA and national agencies) and private entities. However, in practice, this new emerging market is fully intended to the technological demonstration of single HW devices, while no attempt has been made to approach the emerging needs within a comprehensive view covering the full chain from HW and SW technologies to their application into actual use cases and applicative scenarios.
The AI Express project aims at the creation of a re-usable technological base, providing a common set of In-orbit Smart Services for Small Satellites, able to satisfy all these needs, bridging an existing gap in demonstration and also defining a way to smartly evaluate novel mission concepts. It is completely intended to the demonstration of the benefits and the impacts of new HW and SW based technologies, and specifically of Artificial Intelligence based ones. Both the new HW and SW technologies will contribute to the definition of next generation of smart satellites, identifying and validating the novel possible building blocks that will contribute to the expected paradigm shift, able to fit to big data availability, systems responsivity and autonomous, or even collaborative operations.
Autonomous operations and Artificial Intelligence will impact over the whole space Value Chain: from the user-service interaction to satellite platform, from service performances to ground segments, from resources management to mission planning. Main expected impacts will affect reactivity, responsiveness and latency.
The AI Express testbed will implement a “framework” intended to the on-board execution of AI based algorithms for payload data processing and actionable information extraction. It is made of a comprehensive development and test framework, composed of a set of layers, each including HW and SW modules, ready to be proven on-board on a demonstration platform in orbit, as a standalone nano-satellite or as a part of a larger demonstration mission (e.g. it perfectly fits to the In Orbit Now service). Users will be able to configure a custom mission on-demand, combining available “standard” payloads, HW and SW resources already available and flying, as well as ask for custom configurations based on their own payloads to be embarked as host. Also custom configurations of AI processing devices (e.g. Movidius Myriad VPU) can be embarked as stand-alone “hosted payloads” providing specific capabilities in a heterogeneous processing environment as a service.
Thanks to the integration in the D-ORBIT’s ION platform, Ai Express services will be available on-demand virtually in real time, by configuring the SW modules and HW elements/ resources already available in-flight from the catalogue. AI algorithms too, can be selected from catalogues (also external ones) or provided by customers.
It will provide services fitting to the emerging future payload technologies and the “NewSpace” missions scenario. In few words, it makes configurable AI available on-board, directly integrated to payloads’ data and satellite’s bus services and makes also available configurable missions on-demand (out of the already flying elements). Thus users will be able to start from their own application, select sensors, task acquisitions, apply pre-processing, select AI algorithms and define what actionable information they want to use or downlink.
Innovations have been targeted at multiple levels:
• high level functionalities on-board: Novelty Detection, Deep Neural Networks, Artificial Intelligence and adaptive algorithms (with on-the-fly customization and active learning) inside a ring based HW/SW safety infrastructure;
• definition of a framework of common (shared and re-usable) SW modules providing basic functionalities (with standard interfaces and protocols): HW abstraction, attitude fine determination, time management, data flow, data filtering and co-registration;
• on-board architectures’ computational capabilities: testing effectiveness in AI technologies of promising generic processing units supported by parallelization architectures (heterogeneous computing platforms and many-cores systems), of novel memory organization and sizing.
• sensor and its management: assessing how current sensors’ capabilities can be better exploited with a smart use of fine near-real time knowledge of the acquired scene with different acquisition modes and improved pointing capabilities;
• the “NewSpace” enabling technologies, based on blockchain.
AI Express aims to be a unique practical implementation entirely devoted to a new class of applications, able to investigate all possible advantages and to identify limits. In this sense, the possibilities provided by such a system will allow the quick development and demonstration of the key (both HW and SW) technologies will characterize next generation of satellite missions, bringing a considerable advantage to systems designers and developers.

Edge computing is becoming more important in the artificial intelligence landscape. It was pushed by the rise of GPUs, which due to massive parallelisation enable cost efficient deep learning. With the introduction of Intel’s Myriad VPU, and the slimmed down versions of Nvidia’s Volta architecture, more efficient GPUs were introduced to further promote edge computing. Answering this trend Google developed a custom Application-specific integrated circuit (ASIC) specifically for neural network inference, called Tensor Processing Unit (TPU).

The TPU is a relatively low powered, small and very capable computation device when compared to older CPU or GPU architectures. It’s on average about 10-15x faster than CPU/GPU and reaches about 30-80 times the amount of TOPS/Watt. The heart of the TPU architecture lies in its 256x256x8 Matrix Multiply Unit. Recently, Google has implemented this TPU architecture to fit a smaller package for inference on the edge: the edge TPU.

The edge TPU offers a host of advantages towards applications in space. It is packed with a development board based on the MX8M SOC, which can run a modified version of Linux, allowing for quick development. It’s capable to run computer vision models like mobilenet at 100+ frames per second, consuming significantly less power than its CPU or GPU equivalents. It also has a very small footprint measuring only 40 x 48 x 5.11 mm. The combination of high processing speed, low power consumption and small footprint make it an ideal candidate to be fitted under demanding circumstances, including spacecrafts, satellites, extraterrestrial rovers, or under remote terrestrial conditions. Exemplarily, edge TPUs could provide very useful analytical capabilities on-site, when bandwidth is limited. This includes inference on micro satellites to circumvent downlink limitations or areas in general with insufficient internet connectivity.

As a company active in machine learning, ML6 has found this technology very useful for different use cases. We exclusively obtained several units along with comparable GPU based devices from other manufacturers, and are actively investigating capabilities with regard to vision applications. Our first investigations particularly highlight the edge TPUs capabilities for live video object detection, and its capabilities to achieve state of the art accuracies even when utilizing its inherent 8-bit quantization, allowing for even more efficient computation.

In the last decades, the NewSpace paradigm shift ran over all field of space let affecting both commercial and scientific opportunities. A plethora of new actors faced the space engineering world creating not only business opportunities, but also scientific opportunities.

The European Space Agency (ESA) has always looked with an interested eye at what was developing in the NewSpace world with particular interest in the technology developments, scientific applications and demonstrations of innovative sensing techniques.

This paper will present the first scientific and technology CubeSat initiative of the Earth Observation Programmes directorate in the NewSpace field, which combines the FSSCat mission (Federated Satellites System on Cat) - the winner concept of the 2017 Sentinel Small Satellite (S3) Challenge under the Copernicus Masters project – with the idea to use onboard Artificial Intelligence (AI) so as to dramatically improve the mission and/or system capabilities. The latter concept, applicable on a wider scale than just this mission or indeed just small satellites, is referred to with the generic name of “Φsat”. Therefore the Φsat-1 experiment onboard FSSCat is the first embodiment of this innovative concept.

The FSSCat mission will feature among others improved hyperspectral optical capabilities, to which AI technology will be applied. The mission has a twofold objective:
1. It will demonstrate innovative remote sensing techniques based on a combined GNSS-Reflectometer (GNSS-R) and L-band microwave radiometer, enabling ice extent and ice thickness measure, together with hyperspectral optical (VNIR and TIR bands) measurements to allow pixel decomposition for soil moisture retrieval and for other land and water applications.
2. It will embark a series of technology package to be demonstrated in flight such as:
o An AI chip, integrated into the HyperScout®-2 hyperspectral imager, to perform innovative cloud detection and masking on the hypercube data
o An optical inter-satellite link to demonstrate the enabling building blocks for upcoming satellite federations, and
o A proof-of-concept of a Federated Satellite System protocol using a Radio Frequency Inter Satellite Link.


It is planned that at the time of the conference Φsat-1 will have been launched in the Vega SSMS PoC flight.

DarkCarb is a pioneering Earth observation (EO) satellite, under development at SSTL, designed to acquire high resolution Mid Wave Infrared (MWIR) imagery and video from low Earth orbit. The mission will set a precedent in IR performance from a small and capable satellite platform while maintaining the SSTL cost effective approach thereby enabling a spacecraft price which makes building constellations, capable of delivering rapid re-visit and wide area coverage, an attractive and worthwhile commercial investment.

The DarkCarb satellite features an innovative low mass and volume MWIR imager which, when combined with the implementation of novel image enhancement algorithms, will achieve high quality sub-4m GSD imagery. The instrument is assembled using COTS devices and components fabricated using standard industry processes and optimised for accommodation on SSTL’s small Carbonite LEO satellite platform.
The DarkCarb satellite product will therefore enhance the current 1 m GSD visible imaging and video capability of SSTL’s Carbonite series of satellites demonstrated through Carbonite-1 and 2, launched 2015 and 2018 respectively.

The high spatial resolution MWIR imagery DarkCarb will deliver provides several key and complementary differentiators to visible imagery and therefore has the potential to become a high value data product for the EO market. MWIR imagery provides the capability to differentiate between objects and surfaces of different temperature and emissivity. As the detectable signal is only dependent on the temperature of the scene, DarkCarb also has the ability to extend imaging opportunities into night time.

The primary application domain for DarkCarb is defence and security with particular utility when used in conjunction with optical assets for change detection during night time. The high resolution and small footprint make it ideal for acquiring more detailed information from known areas of interest allowing the detection of land based vehicles and can support the recognition of larger objects such as aircraft and ships, allowing airfields and known maritime time areas of interest to be monitored. In addition, as the imager can detect temperature differences it should be possible to derive information on recent activity in a scene not normally discernible in Visible-NIR imagery, such as recently used or moved vehicles, and ship wakes.

The video capability allows information on highly dynamic features in scenes to be provided and will be of key interest for applications relating to human activity. DarkCarb also has the potential to assist with disaster support activities for wildfires, volcanic eruptions and flooding, for example, and may also provide utility in the mapping of heat islands in urban areas to support environmental activities through the detection of buildings or installations emitting a significant level of heat. Other potential applications include providing an indication of the operational status of industrial installations via the detection of heat from the operation of equipment, and detecting waste and pollution spills or discharge from sewage plants and power plants.

The DarkCarb mission is therefore a highly innovative development which has the potential to seriously disrupt the status quo of the commercial satellite imagery market by providing affordable high quality and high resolution MWIR data which will address a range of applications.

A freemium consumer model for Earth Observation? It’s time to democratize space for the benefit of all humanity.

Summary
Our ever-changing world faces many big challenges. Looking at Earth from space improves our understanding of how our planet is changing. New space technologies have enabled the Earth Observation market to grow, with a broad spectrum of data types and a greater choice of temporal and spatial resolutions, addressing a range of markets, from agriculture to maritime, from security to mapping and from energy to financial intelligence. But there is no service for the individuals. Earth Observation data is not accessible by, or suitable for, consumers. With billions of people now empowered with smart phones, the time has come to share information about our ever-changing world and sustainable future in space to the people.

Timely video from space has unique attributes of movement and story-telling power that can help humanity change the outcome of both rapidly evolving disasters and crises, and the longer-term picture of our future on Earth and in space. A new consumer model that democratizes space is necessary to ensure these unique video perspectives are accessible—and benefit—all humanity.

Our ever-changing world. The human world changes every day through the actions of a growing population.

The natural world changes every day.

The interaction of these worlds, of humans and nature, ensures life on Earth is constantly changing.

On a planetary scale, we face many big challenges—poverty, access to food and clean drinking water, health, climate change and humanitarian crises.

For example, in the decade 2007-2016 there were on average 354 reported natural disasters each year, 68,273 lives lost, 210 million people affected and $141B of economic loss suffered.

Either through natural disasters or political conflict, there nearly 70 million displaced people in the world now, struggling for basic human rights of shelter, sustenance, medicine and education—survival.

We also face big opportunities—to become a space-faring civilisation with settlements on new worlds for a sustainable multi-world future—opportunities that require people to be inspired and informed as a foundation of democratic consent.

Ever more people must face these challenges and opportunities, as the population of Earth grows exponentially.

Solutions are needed more than ever.

We should build solutions that can reach billions of people, empower billions of people to make informed decisions, solutions designed for the benefit all humanity.

Earth Observation has a vital function in addressing the world’s big challenges.

Satellites provide a bird’s eye perspective—a unique perspective—of life on Earth.

New space has seen more satellites launched to observe Earth, with more types of data, more viewpoints and more analytics to inform us of what’s happening. But there is a long way to go in this ‘new space’ revolution.

The use of satellites for monitoring Earth is only at the beginning. By analogy, compare the mobile phone in the early 2000s and now. The current Earth Observation market is where the mobile phone was in the early 2000s. The satellites, like the phone, are getting smaller and more powerful. Yet, in this analogy, the smart phone and the app store are still to come—and that’s where Sen fits in.

There are two limitations with current solutions:

1/ there is nothing for the people—there is no accessible, useful or independent media about our ever-changing world and future in space for the people of this world.

2/ there is no ‘Earth live’

In respect of the above problems:

Current EO content is optimised for the uses and markets where it has been proven that the data is important and adds value—scientists, governments and businesses, who rightly benefit from the value of the data.

The content is not designed for individuals.

The content is not accessible to billions of people with smart phones.

We need to expand the offering to engage the people of this world.

The solution is streaming real-time videos from space to smart phones.

Why?

Because video has unique attributes of movement and story-telling power which, if given to the people, can change the outcome.

Video can change the outcome of both rapidly evolving disasters and crises, and of the longer-term management of our planet.

Let’s consider rapidly evolving events.

Current solutions are pictures which show the before and after, between revisits of the satellite. Whilst revisit rates are improving, there is still a gap in terms of timely, actionable information about rapidly evolving events, such as storms, fires and other natural disasters.

Imagine if, instead of looking at the before and after, the difference between picture A and picture B, you could see real-time video of the journey from A to B—imagine if you could use real-time video to change what the final picture looks like, to change the outcome?

Video has the unique attribute of movement. From movement it is possible to derive speed and direction.

For example, what is the speed and direction of that flood?

From speed and direction you can then derive actionable alerts: people can be alerted and plans for evacuation and containment can be enhanced:

…so real-time videos can change the outcome.

Let’s consider the longer-term governance of our planet:

Video has another unique attribute: story-telling power

Story-telling power.
Think of the power of movies to tell a story and change the way people think, feel and act … movies move people

Take climate change. As well as treaties and goals, ultimately we need the people of the world to act, to change behaviour, to make the difference. Seeing Earth from space can help change the way we think, feel and act.

Another example is our exploration of space beyond Earth. Video is needed to tell this story, to provide informed consent and inspiration for a sustainable and democratic future.

There are 3 key elements to Sen’s approach:

1/ Timely video, because it can change the outcome

2/ Content focus for humanity, because the greatest stories involve people.

3/ Distribution. A freemium model: (a) We need to give to the people—with a free version direct to smart phones to ensure it is as accessible to as many people as possible. (b) We need an open source platform for analytics developers to accelerate a global ecosystem of information built on EO data that can reach ever more people and markets. Imagine an ‘app store’ with a wealth of apps built on analytics of EO data.

I pay tribute here to ESA and NASA have made their EO data open source and proven that when they have done so, demand follows. But New Space has not yet found a way to make such a model work commercially. Sen’s approach, designing a service optimised for a direct-to-consumer model and open source ‘app store’ approach, will create demand for the data on which Sen can build a sustainable business model.

The global infrastructure for the distribution of information exists and is growing, with many plans that will ensure people in all countries of the world of world can enjoy access to high speed broadband.

Over 3 billion people access social media. Over 4 billion people have access to the Internet. Over 5 billion people have a mobile phone.

The distribution platform for information to reach an increasing number of the world’s population is therefore well underway.

Democratizing space is necessary for people to fulfil their responsibilities to each other and the future of the planet.

Democratizing space means empowering people with information about our ever-changing world and future in space; democratizing space means empowering people to change the outcome of our present and future.

It is Sen’s vision is to democratize space with unique perspectives of our ever-changing planet and our future in space, accessible by everyone, to inform, educate, inspire and benefit all humanity.

Sen will achieve this by its mission of streaming real-time videos from space to billions of people.

ENDS