🚀 AI-Powered Rapid 3D Modeling Technology from NEC and Keio University!

[Erkan Teskancan]

💡 3D Model from Smartphone Videos in One Minute!​

NEC Corporation and Keio AI Research Center have developed a groundbreaking technology that creates highly detailed 3D models from videos shot with smartphones in just one minute. This system automatically removes transient objects, creating digital twins that accurately reflect real-world site conditions. And all this without the need for expensive specialized hardware!

🏗️ A New Era for Digital Twins​

Targeting digital twin applications for the infrastructure, public utilities, and construction sectors, NEC plans to commercialize this technology within fiscal year 2027. This innovation will facilitate remote inspection, reduce inspection costs, and accelerate decision-making processes.

🎥 No More Video Limitations!​

Traditional video recordings made it difficult for remote teams to quickly view specific perspectives or dynamically adjust angles. Existing 3D modeling methods, on the other hand, were fraught with obstacles such as expensive sensors, the need to halt operations, and long processing times.

🧠 The Power of AI and Gaussian Splatting​

To overcome these challenges, NEC and Keio University combined the Gaussian Splatting technique with specialized neural network processing. This made fast and high-quality spatial reconstruction possible with mobile device cameras, without the need to interrupt ongoing operations.

✨ Two Key Innovations: Optimization and Object Removal​

• Visual Complexity Analysis and Adaptive Particle Distribution: The system analyzes the visual complexity of the video, concentrating 3D Gaussian particles in complex areas while thinning them out in flat regions. This adaptive optimization reduces processing load by 90% while preserving visual detail.
  
• Transient Object Removal and Background Infilling: Artificial intelligence automatically detects and filters out transient objects such as moving personnel and vehicles. It then infills missing background structures based on environmental data, creating a static, unobstructed 3D model that represents the permanent layout of the facility.
  

The resulting 3D models are compatible with ordinary computers or tablets. This allows field engineers and remote coordinators to instantly assess site conditions, conduct virtual inspections, and accelerate decision-making during operational anomalies.

🚀 Technical Details: The Secret to Speed​

While traditional 3D reconstruction methods require long processing times to generate dense polygon meshes, Gaussian Splatting speeds up the process by representing the 3D space as continuous, translucent 3D ellipsoids (Gaussians). The NEC and Keio platform further enhances this technique by integrating an active visual complexity analyzer. This allows the system to place initial Gaussian seeds in an optimized manner and focus GPU processing power only on complex surfaces, compressing the entire rendering process into one minute.