Introduction
In recent years, the number of artificial satellites has dramatically increased, making space-based services more accessible. Satellites are already integrated into our daily lives, from smartphone location services to weather forecasts. However, many challenges remain in leveraging satellite data. For instance, obtaining data for specific locations at specific times can be difficult, the time from data acquisition to usability can be too long, or the costs can be excessive. Fujitsu Limited, as an ICT (Information and Communication Technology) company, is advancing research and development to overcome these challenges and make space data more user-friendly and valuable. This article will explain Fujitsu's "Space Data On-Demand" concept and the three innovative technologies enabling it, in an easy-to-understand manner for those less familiar with the technology. We will particularly focus on "Satellite Edge Computing," which performs AI processing on satellites, introducing its mechanisms and potential.
What is "Space Data On-Demand"?
What if you could photograph any desired location on Earth from space at any given time? This would create immeasurable value across various fields: rapid disaster response, monitoring environmental changes, precision agriculture for detailed crop management, and infrastructure management for roads and bridges. The ability of artificial satellites to capture vast areas at once further enhances this convenience.
Challenges of Conventional Satellite Data
In reality, it's not that simple. Artificial satellites orbit Earth at incredible speeds, approximately 8 km/s (about 28,800 km/h, roughly 100 times faster than a Shinkansen). They can circumnavigate Earth in about 90 minutes. Changing the planned trajectory of a satellite moving at this speed is far more difficult than a car traveling at 300 km/h on a highway abruptly changing lanes. While a car can brake, a satellite has no brakes.
Furthermore, launching and operating satellites incurs enormous costs. Consequently, conventional satellite data has always presented the following challenges:
- "Desired data is unavailable": It's challenging to photograph specific locations at precise times, making it difficult to obtain needed data when and where it's required.
- "Data is expensive": High satellite operating costs translate to expensive data acquisition.
- "Information is not timely": Significant time elapses between satellite data capture and its delivery to the ground, processing, analysis, and conversion into usable information.
"Space Data On-Demand" as a Solution
To address these challenges and make space data more accessible and valuable, Fujitsu proposes the "Space Data On-Demand" concept. "On-Demand" means "upon request," aiming to provide users with the space data they need, when they need it, in the required format, similar to video streaming services.
Specifically, AI on the satellite automatically analyzes images and immediately provides analysis results and insights (meanings and discoveries derived from data) tailored to customer needs. This aims to realize the ideal of obtaining "the results of analyzed images" of any desired location on Earth, at any time.
For example, consider a large-scale farmer who wants to know the condition of their fields. Traditionally, they would have to purchase satellite images and either analyze them themselves or hire specialists. However, with "Space Data On-Demand," on-satellite AI automatically analyzes the farmland and delivers specific advice to their smartphone or PC when needed, such as: "Crop growth in this section is good. However, this section shows signs of water shortage. Early irrigation is recommended." This envisions a future where an expert overseeing farmland from space constantly provides advice.
Overview of Three Technologies
Realizing "Space Data On-Demand" requires innovative technologies. Fujitsu is advancing research and development by combining the following three technologies.
These technologies can be easily understood through a cooking analogy. They form three pillars that create a workflow: quickly sort necessary ingredients (Satellite Edge Computing) → carefully prepare them to enhance quality (Observation Data Precision Enhancement Technology) → combine various ingredients to cook a delicious meal (Large-scale Geospatial Data Processing Platform).
1. Satellite Edge Computing
This technology performs AI processing on satellites and transmits only essential information to the ground. This significantly reduces communication volume and delivers information quickly. Analogously, it's like a local reporter summarizing important news and sending only that to headquarters.
2. Observation Data Precision Enhancement Technology
This technology generates high-precision, detailed information from coarse (low-resolution) or incomplete data. It's similar to sharpening a blurry photo or filling in missing parts. It extracts maximum value from limited data.
3. Large-scale Geospatial Data Processing Platform
This technology integrates diverse geospatial information and industrial data to reconstruct industrial activities on Earth within a digital space. This enables global prediction and optimization of logistics and economic activities. It's akin to a simulation game replicating global business flows.
These three technologies are complementary. (1) efficiently extracts information, (2) enhances its quality and precision, and (3) integrates various information for analysis and decision-making support. Only by coordinating these technologies can the ideal of "Space Data On-Demand" be realized.
Satellite Edge Computing
Why is on-satellite data processing necessary?
The slow pace of increased adoption of space services is due to significant technical hurdles. These include limitations in bandwidth for data transmission from satellite to ground (the amount of data that can be communicated at once) and the excessive time required for ground-based data processing.
Conventionally, satellites would transmit all vast amounts of raw data (unprocessed image data) to the ground for processing. However, this method has limitations because as satellite observation capabilities improve, the volume of data continuously increases.
It's like trying to transport a large volume of cargo through a narrow tunnel (communication link). The more cargo (data) there is, the more congested the tunnel becomes, delaying cargo delivery. Moreover, widening the tunnel (communication link) incurs enormous costs.
"Satellite Edge Computing" as a Solution
The key here is "Satellite Edge Computing." "Edge" means "at the periphery" or "nearby," referring to the location where data originates (the satellite). Therefore, satellite edge computing is the technology of "processing data directly on the satellite, immediately after acquisition."
Specifically, it involves pre-processing, extraction, and summarization on the satellite, significantly reducing the volume of data sent to the ground (downlink) while shortening the overall time to decision-making.
For example, on-satellite AI analyzes images and transmits only truly necessary information (detected anomalies, identified objects), resolving communication bottlenecks. This enables rapid alerts for time-critical situations like disaster monitoring.
"Oh, is that all?" you might think. However, since user needs vary, processing must be done with programs tailored to individual users. Traditional artificial satellites didn't support this. Thus, "to allow everyone to use the acquired data freely," all data was sent to the ground and processed there. But Fujitsu believes this will be necessary in the future. As a first step towards a system where users can obtain only the data they need, Fujitsu is focusing on establishing design technology that simultaneously satisfies the following three elements:
Soft Error Tolerance to Space Radiation: Space is exposed to significantly stronger radiation than Earth. This radiation can cause malfunctions when it strikes computer electronic circuits (known as soft errors). Therefore, a mechanism resistant to radiation and capable of processing without issues even when errors occur is necessary.
Programming Environment: Addressing soft errors requires both hardware and software solutions. Traditionally, programs for artificial satellites and aircraft were developed by highly trained specialists to be soft error tolerant. Moving forward, at least for user-defined processing, a software environment that allows users to program themselves is essential.
High-Performance AI Processing under Power Constraints: Satellites can only use limited power generated by solar panels. Therefore, advanced AI processing must be performed with minimal power. Even smartphones can now run impressive 3D games, but they generate significant heat. Artificial satellites typically cannot be air-cooled, so power consumption must be suppressed for this reason as well.
Further details on these topics will be explained later in this article.
Observation Data Precision Enhancement Technology
In parallel with Satellite Edge Computing, Observation Data Precision Enhancement Technology plays a crucial role. In a nutshell, this technology is like the "art of sharpening a blurry photo."
Why is this technology needed?
When filtering data on a satellite, resolution may be lowered or some information omitted to reduce communication volume. Additionally, the ground may be obscured by clouds, or only coarse data might be obtainable due to sensor limitations. The role of this technology is to extract precise and detailed information as much as possible from such "incomplete data."
Example of a Specific Technology: High-precision Precipitation Estimation Technology (Enhancing Precipitation Estimation Accuracy)
Let's explain this with High-precision Precipitation Estimation Technology, a technology Fujitsu is developing.
This technology combines two cutting-edge AI techniques: diffusion models and residual learning:
Diffusion Model: This technology is analogous to gradually clearing a fogged-up photograph to make it clear. It progressively sharpens blurry images.
Residual Learning: This method learns not the "correct answer itself," but only the "difference (deviation) between the current estimate and the correct answer." This allows AI to learn efficiently and improve accuracy.
What are the results?
This technology enables the creation of detailed precipitation distribution maps at 1 km square resolution from coarse satellite data, which typically treats 10 km square areas as single grids (a resolution improvement of approximately 10 times). Furthermore, it automatically corrects systematic deviations inherent in satellite data, such as a consistent overestimation compared to actual values.
As a result, it becomes possible to accurately estimate "where and how much rain is falling," even in areas with few rain gauges like oceans or mountainous regions. This is valuable in various scenarios, including early disaster warnings, planning for agricultural water, and water resource management.
Large-scale Geospatial Data Processing Platform
The ultimate goal is to integrate the technologies described so far and construct the Large-scale Geospatial Data Processing Platform, which leverages information related to Earth's industrial activities as "intelligence."
What is Large-scale Geospatial Data Processing Platform?
In a word, Large-scale Geospatial Data Processing Platform is the "technology to recreate the entire Earth as a simulation game." It reproduces global industrial activities such as logistics, manufacturing, and agriculture within a computer, allowing us to digitally verify scenarios like "What if this port becomes congested?" or "What if a disaster occurs?" before they actually happen.
Key Technology Points: Hexagonal Grids and Spatiotemporal AI
Large-scale Geospatial Data Processing Platform divides the entire Earth into hexagonal grids (H3 grid system). Hexagons are used because the distance from the center to each corner is equal, enabling accurate calculation of logistics distances and costs. Additionally, smaller hexagons can be efficiently nested within larger ones, allowing for flexible zoom-in and zoom-out from global to regional scales.
Each hexagon stores weather data from satellites, and real-time information such as ship positions and traffic volumes from ground sensors. These connect to form a massive "geospatial graph."
On this graph, Spatiotemporal Graph Neural Networks (STGNNs) play a vital role. The remarkable aspect of this AI is its ability to simultaneously understand both "connections between locations" and "the flow of time." For example, it learns patterns such as how congestion at one port can spread to an adjacent port (spatial), causing delivery delays tomorrow, and accumulating into overall delays next week (temporal).
What can it achieve?
- Early detection of problems: Noticing that "this congestion pattern is similar to past events where large-scale delays occurred."
- Concrete future prediction: Predicting, "There is an 80% chance of material shortage in this region in 3 days."
- Simulation of countermeasures: Testing "how much overall delay would be improved if this port's processing capacity were increased by 20%."
By doing so, Large-scale Geospatial Data Processing Platform contributes to solving industrial-scale challenges, such as finding optimal routes during disasters and predicting and avoiding supply chain issues, aiming to realize a sustainable and prosperous society.
Data Analysis in Space! The Challenge of "Satellite Edge Computing" to Eliminate Time Lag
When you hear ICT (Information and Communication Technology) in space, many might think of Space Exploration Technologies Corp. (SpaceX)'s "Starlink". With thousands of small satellites deployed in low Earth orbit at an altitude of 550km, it allows for internet access as long as the sky is visible, positioning it as a complement to communication infrastructure that has primarily developed in densely populated areas.
"Satellite constellations," which organically connect a large number of small satellites, are beginning to be utilized not only for internet connectivity but also for civil and military applications such as vegetation, weather, and resource surveys on Earth using numerous cameras and radars.
Low Earth orbit satellites orbit Earth at a high speed of 8 km/s. The acquired image data is obtained as long ribbon-like data, containing distortions arising from changes in angle and distance to the ground due to movement. For practical use, it is necessary to extract the desired data ("when, where, and what data is needed") from this enormous volume of data. Raw data acquired by satellites undergoes multi-stage processing – compression/decompression for transmission to the ground (L0 processing), extraction by specifying required location and time, distortion removal and conversion into a topographic map (L1 processing), and association of various physical data with the topographic map (L2 processing) – to become value-added data suitable for practical use. However, conventionally, the computational capacity of small satellites was limited to data acquisition. Recently, imaging (L1 processing) has become possible, but actual analysis beyond L2 processing is performed after transferring vast amounts of data to the ground via communication.
This inevitably leads to significant time lags. For example, if applied to disaster countermeasures like earthquakes or typhoons, it would be reactive. If data analysis were possible in satellite orbit, and required image data could be obtained in a few minutes, quasi-real-time countermeasures would become feasible, greatly expanding the range of applications.
Against this background, Fujitsu, with the cooperation of Professor Nagai of Yamaguchi University, an authority in space utilization engineering, began developing deployable edge computing technology for satellites.
Operating edge computers in satellite orbit requires addressing multiple challenges such as the harsh space environment (vacuum, radiation), and limited size, weight, power, and heat dissipation. By leveraging the high-reliability, miniaturization, and energy-saving technologies cultivated through handling ICT equipment from large computers to laptops and smartphones, we propose a technology to process satellite data "quickly" and "accurately" in space. This technology is introduced below.
Real-time Analysis in Space! "In-Satellite L2 & Higher-Order Processing Technology"
The first is In-Satellite L2 & Higher-Order Processing Technology. By combining energy-saving technology with speed enhancement through parallel processing using GPUs, we have validated that it's possible to create topographic images with value-added information (L2 processing), such as wind speed and wave height distributions, within the time required for quasi-real-time processing (under 10 minutes), using the limited power (20W) available for small satellites. Traditionally, L2 processing involved transferring gigabytes (Gbytes) of data to ground stations and leveraging ground computational power. Performing this L2 processing on the satellite significantly reduces data transfer volume to the ground and enables high-speed processing.
Facing the Harsh Environment of Space! "Fujitsu Research Soft error Radiation Armor" High-Speed, High-Reliability Technology
Another technology is "Fujitsu Research Soft error Radiation Armor," a high-speed, high-reliability technology. It applies software high-reliability technologies required for mainframe computers and other systems. In space, stronger radiation is constantly encountered compared to Earth, making malfunctions due to its effects inevitable. Traditionally, when a malfunction occurred, the entire system would be reset and reprocessed. However, by dividing computations into multiple parts and re-executing from the necessary sections, the time required for retries is minimized, enabling highly reliable computation in a short time. We call it "SOft error Radiation Armor" because it is a software-based radiation protection technology. Enabling multiple retries within the time required for quasi-real-time processing allows complex and high-difficulty computations to be executed on satellites.
These technologies are implemented on commercial CPU and GPU modules. By choosing an environment that can leverage a large amount of open-source assets instead of dedicated hardware, users can quickly run their desired applications in this environment, contributing to minimized development time and cost.
