发表机构
ShanghaiTech University; Digital Trust Centre, Nanyang Technological University（上海科技大学; 南洋理工大学数字信托中心）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
Sen-Cap是一种无需传感器间校准的激光雷达-相机融合人体运动捕捉框架，通过跨传感器运动估计器与抗噪轨迹跟踪器实现灵活部署与强鲁棒性，在多数据集上达最优性能，适用于真实场景。
AI 中文摘要
我们提出Sen-Cap，这是一个基于激光雷达与相机多模态数据融合的传感器灵活且抗噪的三维人体运动捕捉框架。尽管多模态传感器比单模态传感器能提供更丰富的信息，但现有方法仍存在两个核心挑战：其一，任意部署的传感器间的多模态对齐/匹配通常通过显式校准处理，这会在视角变化时传播误差，进而将部署限制在固定、高度重叠的布局中；其二，现有方法在严重噪声或传感器部分失效时性能下降，而这在真实环境中十分常见。为解决这些挑战，Sen-Cap引入了无需传感器间校准即可在以人为中心的空间中重建局部姿态与形状的统一跨传感器运动估计器，支持灵活数量的传感器，以及通过迭代优化在严重点云噪声下保持鲁棒性的抗噪轨迹跟踪器。这些传感器灵活且抗噪的特性使Sen-Cap在真实部署中更具实用性。值得注意的是，Sen-Cap可实时运行，在Human-M3和FreeMotion数据集的主要指标上达到了当前最优性能，在LiDARHuman26M和RELI11D上也展现出较强的跨域性能。这种灵活性与鲁棒性的结合为真实场景中的运动捕捉开辟了新机遇，例如体育分析、野外机器人及大规模沉浸式环境等领域。
英文摘要
We propose Sen-Cap, a Sensor-Flexible and Noise-Resilient 3D human motion Capture framework that integrates multi-modal data from LiDAR and camera. While multi-modal sensors provide richer information than single-modal sensors, existing approaches still suffer from two core challenges. First, multi-modal alignment/matching across arbitrarily deployed sensors is typically handled by explicit calibration, which propagates errors under changing viewpoints and in turn constrains deployment to fixed, highly overlapped layouts. Second, prior methods degrade under severe noise or partial sensor failures, which are common in real-world environments. To address these challenges, Sen-Cap introduces a Unified Across-Sensor Motion Estimator that reconstructs local pose and shape in a human-centric space without calibrations between sensors, supporting a flexible number of sensors, as well as a Noise-Resistant Trajectory Tracker that maintains robustness under severe point cloud noise through iterative refinement. These sensor-flexible and noise-resilient features make Sen-Cap more practical in real-world deployment. Notably, operating in real time, Sen-Cap achieves state-of-the-art performance on major metrics on Human-M3 and FreeMotion, as well as strong cross-domain performance on LiDARHuman26M and RELI11D. This combination of flexibility and robustness opens new opportunities for motion capture in real-world scenarios, e.g. sports analytics, field robotics, and large-scale immersive environments.
Comments16 pages, 8 figures, 4 tables. Accepted at ECCV 2026. Aoru Xue and Yujing Sun contributed equally. Yuexin Ma is the corresponding author
