Hello. This is Suzuki from the Converging Technologies Laboratory and Ijuin from the Human Digital Twin Division. Our research group is aiming to realize "Ocean Digital Twin" and, as part of this, we are developing an automatic classification technology for multiple seaweed species using image AI. Recently, we presented our research results at the 31st Ocean Engineering Symposium. Here in this post, we will introduce the content of our presentation.
Background and Purpose of the Research
We are aiming to realize "Ocean Digital Twin" by digitizing all aspects of the ocean with high precision. This digital twin will enable the formulation and pre-verification of measures according to the purpose. (Further details are available in press release)
As the first step towards realizing Ocean Digital Twin, we are working on the development for blue carbon credits.
Blue carbon credits are a system for certifying the amount of CO2 absorbed and stored by blue carbon ecosystems such as seaweed beds, seagrass beds, mangrove forests, salt marshes, and tidal flats as "credits."
Through blue carbon ecosystem conservation and regeneration projects, companies and organizations can receive credits in proportion to the amount of CO2 absorbed from the atmosphere.
These credits can be used to offset their own emissions or sold to other companies and organizations.
In order to convert CO2 absorption into credits, accurately measuring the type and area of seaweed is essential. Conventionally, this measurement has been done visually by divers. However, this places a heavy burden on divers and requires a significant amount of time for measurement.
Therefore, we aim to solve this problem by introducing automatic collection and analysis of marine data using Ocean Digital Twin. In particular, the technology for automatically classifying multiple seaweed species from automatically collected seaweed images is very important for marine data analysis. In this article, we will introduce this automatic classification technology in detail.
As a previous effort towards realizing Ocean Digital Twin, we have published "Demonstration of Ocean Digital Twin Technology at Nationally Certified Sustainably Managed Natural Site in Ishigaki Island" in a past blog article. Please take a look at that as well!
Related Technologies
There are two main methods for automatically classifying multiple seaweed species from seaweed images:
- Semi-manual classification technology: This method involves manually specifying the location and type of seaweed species within an image, then segmenting and classifying the image using the maximum likelihood method. However, it takes as much as 6 minutes of processing time per frame, and it is difficult to specify when the water is turbid.
- Classification technology using machine learning: This method trains AI using a large dataset of seaweed images to automatically classify different seaweed species. However, it requires as much as 10,000 training data images per seaweed species, and the classification performance decreases when the water is turbid.
In light of these issues, we proposed an automatic classification technology for multiple seaweed species that is robust to underwater turbidity, requires only a small amount of data, and offers rapid processing times.
Proposed Technology
Our technology consists of two phases: a class-specific feature map creation phase and an automatic classification phase.
- Class-specific feature map creation phase: Using clear images taken at close range (within 0.5 m) from the subject, we create a general-purpose feature map that serves as the basis for automatic classification. This allows us to extract highly discriminating features that are independent of underwater turbidity and appearance, enabling automatic classification with a small amount of data.
- Automatic classification phase: We apply an image enhancement process to seaweed images affected by underwater turbidity and automatically classify multiple seaweed species. The enhancement process enables classification that is robust to underwater turbidity even with a small amount of data. Furthermore, by comparing the original features of the seaweed that are independent of underwater turbidity and appearance, we achieve classification efficiently.
The details of the two phases are explained below.
Class-specific Feature Map Creation Phase
First, we prepare one image for each seaweed species, taken from a close distance that is less affected by underwater turbidity.
Next, we specify the area of the seaweed species we want to classify and create a class-specific reference image.
This class-specific reference image is divided into multiple patches, and each patch is compressed into a two-dimensional feature using a dimensionality reduction method called UMAP (Uniform Manifold Approximation and Projection).
By using UMAP, we can automatically extract two-dimensional features that emphasize the differences between seaweed species.
These automatically extracted two-dimensional features become the class-specific feature map and serve as the comparison source for automatic classification of seaweed species.
As a result, we can create a highly discriminating class-specific feature map that is independent of underwater turbidity and appearance from a small number of images, one per seaweed species.
Automatic Classification Phase
First, we apply an image enhancement process to the seaweed images affected by underwater turbidity to make them comparable to the reference images.
We have previously developed a technology to restore the color cast and blur of images caused by underwater turbidity.
Further details on the enhancement technology are available in this press release.
This enhancement technology enables classification that is robust to underwater turbidity.
Next, we perform the same patch division and feature extraction using UMAP as in the class-specific feature map creation phase on the enhanced image.
The extracted features are compared with the class-specific feature map created in the class-specific feature map creation phase using kNN (k-Nearest Neighbor method), and the seaweed species are automatically classified.
This allows us to automatically classify multiple seaweed species efficiently.
Experiment and Results
We verified the effectiveness of the proposed technology using red and green algae data with underwater turbidity taken in Shimoda City, Shizuoka Prefecture.
As a result, the proposed technology achieved an average classification accuracy of 80.5% compared to the conventional technology (correct image), achieving accuracy equivalent to visual inspection by divers.
In addition, the processing speed was significantly reduced from 6 minutes/frame for conventional semi-manual classification to 21.2 seconds/frame with the proposed technology. This reduces the working time per hectare from approximately 3.5 days for conventional semi-manual classification to approximately 5 hours as well. Furthermore, while conventional machine learning required 10,000 data images per species, the proposed technology enables automatic classification with only 1 data image per species.
Summary
We have proposed an automatic classification technology for multiple seaweed species that is robust to underwater turbidity, operates efficiently with minimal data, and have confirmed that it achieves accuracy equivalent to visual inspection by divers. In the future, we will expand the seaweed species targeted for automatic classification. We will also expand the target to organisms other than seaweed and promote development not only for blue carbon but also for biodiversity conservation. And, in fiscal 2030, we plan to establish a global-scale marine data platform and realize a sustainable ocean where humans and the ocean coexist!
Finally, we would like to share the state of the Marine Engineering Symposium that we recently attended.
The Marine Engineering Symposium is a place to share the latest information in a wide range of fields of marine engineering in a workshop format.
This was the 31st time the symposium was held, and it was held at the Surugadai Campus of the College of Science and Technology, Nihon University, with the core theme of "The Sea Opening Up the Future - Two Days to Learn About the Diversity of the Sea."
Approximately 100 participants gathered to discuss a wide range of topics such as blue carbon credits, marine surveys and development, and tsunami disaster prevention, and held enthusiastic discussions. Through our participation and presentation at this symposium, we hope to make more people aware of the connection between Fujitsu and the marine field, and to spread the image of "Fujitsu = Ocean."
