TOKYO – Fujitsu today announced details of a technology that uses LiDAR and AI to acquire high-resolution 3D data from organisms and other natural features like coral reefs, as well as man-made structures like wind turbine infrastructure in the sea with autonomous underwater vehicles (AUV) as part of its R&D efforts to create ocean digital twin. The digital twins created with this technology will allow researchers to study highly accurate reproductions of underwater ecosystems to predict changes in the environment, as well as simulate the possible effects of conservation measures.
The technology relies on a real-time measurement technique originally developed by Fujitsu for its “Judging Support System,” developed in collaboration with the International Gymnastics Federation (1) to assist in gymnastics judging, which enables scanning from mobile AUVs even when rough currents and waves make conditions difficult. The technology also leverages Fujitsu’s AI for image enhancement to correct color and clarify images, allowing for the accurate identification and measurement of targets down to several centimeters even in murky waters.
To verify the technology, Fujitsu conducted a field trial in the waters near Ishigaki Island, Okinawa prefecture in Japan, alongside the National Maritime Research Institute, National Institute of Maritime, Port and Aviation Technology (NMRI) (2), successfully acquiring precise 3D data for mapping coral reefs.
Moving forward, Fujitsu aims to expand the measurement targets of this technology to include seaweed, which absorbs a large amount of blue carbon (3), creating an ocean digital twin for seaweed beds by FY 2026. This will support companies and local governments in planning measures including the estimation of carbon stored in seaweed beds, measures to conserve and cultivate new seaweed beds, and initiatives to promote and conserve biodiversity in coral reefs.
Fujitsu plans to present detailed results of the field trial at the 2024 Spring Meetings of the Japanese Society of Fisheries Science, which will be held from March 27, 2024 (Wednesday) to March 30, 2024 (Saturday).
Figure 1. Concept of ocean digital twin technology in relation to achieving SX
Leveraging digital technology and AI to address urgent threats to our world’s oceans and biodiversity
To support global efforts to combat the effects of climate change and preserve the biodiversity of our oceans, which cover approximately 70% of the Earth’s surface, Fujitsu is working on the development of ocean digital twin technology that enables advanced verification of marine-related measures. The concept of the ocean digital twin technology is to use AUVs and satellites to collect data relating to features of the ecosystem including vegetation distribution and 3D shapes of underwater life like seaweed and coral reefs, as well as man-made structures that affect aquatic resources and the marine environment. Fujitsu collects this data to build a model that quantifies and simulates changes in the environment and growth of organisms that make up the marine ecosystem and using it for preliminary verification of marine-related measures. For example, to pre-verify measures related to blue carbon, 3D data allows researchers to determine the biomass of plant life in a given area and calculate the amount of carbon dioxide absorbed by it. Biological is also incorporated to perform time-series simulations of the growth of underwater organisms. This makes it possible to simulate changes in carbon dioxide absorption when a certain marine ecosystem conservation measure is taken, including the growth status of underwater organisms, and to verify the effectiveness of global warming countermeasures in advance.
To conserve marine ecosystems and understand the amount of carbon dioxide absorption, it is necessary to acquire high-resolution 3D shape data with a resolution (4) within several centimeters, and to identify underwater organisms and estimate their volume. Existing technologies like acoustic sonar, for example, prove insufficient for this task, however, with resolution limited to about 10cm due to issues such as beam width limitations.
About the developed technology
1. Image enhancement AI technology that restores the color and contour of underwater subjects
Fujitsu developed an image enhancement AI technology that performs deep learning optimized for underwater subjects to create high-resolution 3D images of underwater organisms and structures, even from images taken from murky water where the color is distorted and the contours of objects are blurred. This technology consists of two types of AI—one that achieves turbidity removal and contour restoration, correcting the original color of the subject and improving the blurred contours of target objects to generate enhanced images before 3D conversion. This prevents errors during 3D processing and subject recognition, making it possible to measure the shape of each object.
Figure 2. Precise 3D shape data of coral reefs using image enhancement AI technology
2. Underwater 3D measurement technology that allows real-time 3D measurement from a moving AUV
To measure in 3D in real time underwater, Fujitsu adapted a high-speed sampling technology that uses short-cycle laser emission and high-speed scanning, which it developed for its “Judging Support System,” developed in collaboration with the International Gymnastics Federation to assist in gymnastics judging. Furthermore, Fujitsu introduced an underwater Light Detection And Ranging (LiDAR) (5) device that allows selection of the suitable wavelength for measurement according to sea conditions from three laser wavelengths. This will not only enable three-dimensional measurement from a moving autonomous unmanned submarine, but also make it possible to measure moving objects by developing technology to follow the movement of objects.
Figure 3. 3D measurement results using underwater LiDAR
To confirm the effectiveness of this technology, Fujitsu, in collaboration with NMRI, conducted a field trial (6) to automatically collect underwater data in real time. Fujitsu equipped an underwater sensor that combines a camera and LiDAR for real-time 3D measurement, integrating it into the “AUV-ASV connected system” (7) developed by NMRI. From this experiment, high-resolution 3D data of centimeter-scale objects including pipes installed underwater and coral reefs, were successfully acquired in real time.
Figure 4. AUV-ASV connected system and data acquisition
Future Plans
In the future, Fujitsu will continue to develop technologies that can reliably acquire data even in challenging environments (e.g. strong currents and uneven seabed topography) and participate in measures that are expected to contribute to decarbonization from promoting growth of carbon-absorbing seaweed. Measurement targets will be expanded to include inspections and accumulate use cases. Based on the measured 3D data, Fujitsu will develop ocean digital twin technology that performs simulations incorporating knowledge from biology, environmental science, and other fields. Fujitsu aims to build partnerships with companies, local governments, and organizations working on measures towards carbon neutrality, and to support customers in planning measures using ocean digital twin technology.
[1] International Gymnastics Federation :Headquarters: Lausanne, SwitzerlandPresident: Morinari Watanabe
[2] National Maritime Research Institute, National Institute of Maritime, Port and Aviation Technology (NMRI) :Location: Mitaka City, TokyoDirector General: Minemoto Takemasa
[3] Blue carbon :Refers to the carbon that is absorbed and stored in coastal and marine ecosystems.
[4] Resolution :The ability of a measuring instrument to the like to identify physical quantities. Expressed as the minimum distance between two discernible points of the measurement object.
[5] LiDAR :A technology that measures the distance to an object and the shape of the object by emitting a laser at the object and capturing the reflected light with a light sensor.
[6] Field trial :Conducted in the waters near Ishigaki Island, Okinawa Prefecture, from Monday, January 22 to Thursday, January 25, 2024.
[7] AUV-ASV connected system :An underwater research system developed by NMRI that transmits sea-bottom images taken by an autonomous underwater vehicle (AUV) to an autonomous surface vehicle (ASV) via a tether cable, and then transmits the images to a research vessel via Wi-Fi.
