Tokyo – NEC Corporation has developed a first-of-its-kind technology (*1) that uses proprietary AI to rapidly generate highly detailed 3D models in as little as one minute, solely from footage captured with general-purpose cameras such as those on smartphones, while automatically removing unnecessary subjects. This technology was developed in collaboration with the Keio AI Research Center (*2).
The technology can generate 3D models in as little as one minute that precisely replicate on-site conditions without the need for costly specialized equipment and without interrupting operations, even in frequently changing environments such as electric power infrastructure facilities and construction sites. Additionally, the 3D models can be easily viewed on ordinary computers or tablets, allowing users to instantly assess on-site conditions. This facilitates remote facility inspections and enables rapid decision-making in the event of anomalies.
With this technology, NEC aims to promote the adoption of digital twin technology among infrastructure operators and the construction industry, helping to alleviate labor shortages and enhance facility safety. NEC plans to commercialize this technology within fiscal year 2027.
Background
In recent years, infrastructure operators and worksites in the construction industry have been facing labor shortages and rising travel costs associated with facility inspections. This has resulted in expanded efforts to share on-site footage with supervisors in remote locations to facilitate remote decision-making and guidance. However, relying solely on video feeds from sites makes it difficult to quickly identify specific scenes or viewpoints, posing challenges in accurately assessing on-site situations.
In this context, as one method for creating digital twins—which replicate real-world on-site conditions in virtual spaces—there are high expectations for the use of 3D models that allow users to view the areas they wish to inspect from any viewpoint. Meanwhile, hurdles to adoption remain, including the need for costly specialized equipment, the halting of on-site operations for filming, and the considerable time required to generate 3D models after filming.
To address these challenges, NEC combined Gaussian Splatting (*4), a technique that has recently attracted attention for its growing use in generating backgrounds for movies and animation, with its proprietary AI, achieving the rapid generation of highly detailed 3D models.
Value provided by this technology
This newly developed technology eliminates the need for costly specialized equipment and enables the rapid generation of highly detailed 3D models that precisely replicate on-site conditions without disrupting operations. Furthermore, the 3D models can be viewed on ordinary computers or tablets, allowing users to instantly assess on-site conditions. In addition to enabling remote facility inspections and facilitating rapid decision-making in the event of anomalies, this technology allows for the optimal allocation of limited personnel, which supports more efficient and advanced on-site operations.
Features of this technology
(1) Rapid generation of 3D models using efficient, optimized placement of points
The technology automatically analyzes the visual complexity of on-site video footage. In complex areas, the points forming the 3D model are densely clustered. In contrast, in more uniform areas, these points are more spaced out, optimizing the quantity and density of the points. This enables the generation of highly detailed 3D models while minimizing the total number of points required, all without sacrificing visual detail. Compared to conventional methods for generating Gaussian Splatting, this approach reduces the generation time by 90%, enabling rapid generation in as little as one minute.
(2) High reproducibility of on-site conditions through automatic removal of unnecessary subjects
During the 3D model generation process, the technology automatically detects and removes temporary subjects that are visible in on-site footage. Additionally, since the background of the areas removed can be inferred and filled in based on the surrounding footage, accurate 3D models of on-site conditions that are free from unnecessary subjects can be generated even at sites in operation.
Going forward, NEC remains committed to driving the adoption of digital twin technology among infrastructure operators, the construction industry, and beyond to alleviate personnel shortages, facilitate business transformation, and enhance facility safety.
