🤖 Low-Latency Sensor Fusion in Humanoid Robots: AEON's New Architecture! 🚀

[Erkan Teskancan]

Innovative Solution to Perception Challenges in Humanoid Robots​

In industrial automation and complex operating environments, general-purpose humanoid robots require extensive spatial intelligence to map facilities, manipulate components, and create real-time digital twins. Hexagon Robotics is addressing these demands by leveraging the NVIDIA Holoscan edge AI framework in its AEON humanoid robot. This integration replaces the resource-intensive legacy sensor pipeline with a centralized, pre-processed architecture designed to boost perception throughput without overloading primary compute modules.

Overcoming Mechanical and Computational Barriers​

General-purpose robotic operations rely on real-time sensor fusion, combining diverse streams including RGB video feeds, depth data, and inertial measurement unit (IMU) readings. Hexagon Robotics' previous designs utilized dual compute nodes interconnected via ROS 2 to manage simultaneous localization and mapping (SLAM) and AI-based feature detection. Image transformations, neural network inference, and classical computer vision algorithms ran via GStreamer pipelines using NVIDIA DeepStream plugins.

This distributed arrangement created operational and physical limitations. Connecting sensors via standard MIPI interfaces required 12 fixed ports split across both compute units, necessitating inter-unit data transfers that introduced latency. Furthermore, routing multi-conductor cable harnesses through the robot's rotating neck mechanism imposed severe mechanical routing constraints, increasing wear and limiting rotational mobility.

FPGA Bridge and High-Throughput Optical Transmission​

To resolve physical interconnection limitations and latency overhead, Hexagon Robotics developed a unified hardware-software sensor bridge. This architecture includes a custom data processing board paired with configured FPGA logic to synchronize frame capture across cameras and depth sensors. This ensures precise temporal alignment across all sensing modalities.

NVIDIA Holoscan functions as the central runtime environment, managing sensor pipeline execution and integrating Holoscan Sensor Bridge modules to handle IMU telemetry and deterministic data transmission. Instead of routing individual copper MIPI lanes through articulated joints, synchronized sensor payloads are streamed over an optical fiber using an NVIDIA IP core.

The optical link terminates at an NVIDIA Jetson Thor compute module via a dedicated receiver board. Incoming traffic is processed directly through the platform's hardware-accelerated multi-gigabit Ethernet interface. This arrangement supports peak network bandwidths of up to 100 Gbit/s while eliminating unnecessary buffer copy operations in the measured transmission path.

Computational Resource Optimization for Spatial Intelligence​

Freeing central processors from low-level data ingestion yields measurable performance gains in AEON's onboard architecture. Offloading sensor ingestion to dedicated interface hardware reduces CPU consumption per sensor stream to less than 1%. Optical ingestion removes camera data processing entirely from the graphics processing unit (GPU), leaving only minor GPU compute requirements for time-of-flight depth processing.

Preserving primary compute capacity allows Hexagon Robotics to directly allocate central GPU resources to high-level spatial AI, dynamic object classification, and motion planning routines. Because it uses Jetson Thor network interfaces, the engineering team can expand sensor arrays or integrate network switches to support additional modalities without providing secondary central processing units.

The shift from localized MIPI data buses to high-bandwidth network protocols reflects a broader change toward software-defined sensor pipelines in high-performance robotics. Industrial humanoid architectures often contend with trade-offs between distributed microcontrollers and central high-performance compute, especially when routing high-bandwidth MIPI or USB 3.0 lines along articulated joints such as robotic necks and multi-axis gimbals.

Standard MIPI CSI-2 transmission is limited by trace and cable lengths (typically under 30 centimeters without active redrivers) and requires point-to-point connections, making cable harness routing through rotary joints a common point of failure. By encapsulating serialized camera and depth feeds into Ethernet packets carried over fiber optics, the architecture overcomes the electromagnetic interference risks and mechanical bending limits associated with copper cable harnesses.

In competing benchmark architectures, humanoid systems often manage sensing pipelines via discrete edge vision processors or industrial PCs running ROS 2 nodes over standard copper Gigabit Ethernet. Such arrangements often consume significant host CPU overhead to manage serialization, deserialization, and memory copies (IPCs) between discrete processing blocks. Combining hardware-accelerated network interfaces with dedicated FPGA pre-processing enables zero-copy sensor delivery to GPU memory via direct memory access mechanisms, representing a standard design pattern for low-latency autonomous operations requiring continuous environmental mapping.