发表机构
School of Mechatronic Engineering and Automation, Shanghai University; SHU General Intelligent Robotics Research Institute; School of Physics, Changchun University of Science and Technology（上海大学机电工程与自动化学院; 上海大学通用智能机器人研究院; 长春理工大学物理学院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究提出基于Prophesee EVK4事件相机的实时上半身人形机器人遥操作框架，在低光照等场景下性能优于RGB方案，端到端延迟23-34 ms，为相关应用提供实用解决方案。
AI 中文摘要
我们提出了一种由神经形态事件驱动视觉驱动的实时上半身人到人形机器人运动模仿框架。本工作解决了传统基于帧的RGB传感器的实际感知瓶颈，特别是由于固定积分时间导致其在高动态范围（HDR）场景和快速运动中表现不佳。通过利用Prophesee EVK4事件相机，该相机异步运行，具有高时间分辨率和超过120 dB的动态范围，我们的系统在标准视觉管道性能下降的条件下（如严重逆光和低于5 lux的极低光照环境）支持稳定跟踪。该架构集成了低延迟感知模块（利用优化的事件积累和重力对齐的惯性融合），以及执行在线运动重定向的因果运动模块（TWIST）。我们在嵌入式NVIDIA Booster T1平台和18自由度（DoF）人形机器人上半身装置上验证了该系统，展示了23-34 ms的端到端光子到动作延迟，且在我们的实验设置下优于RGB基线。结果表明存在一种实用的权衡：对于快速或光线不佳的上半身遥操作，事件可能更合适，而光线良好的静态场景可能更适合RGB或混合传感。
英文摘要
We present a real-time upper-body human-to-humanoid motion imitation framework driven by neuromorphic event-based vision. This work addresses practical perceptual bottlenecks of conventional frame-based RGB sensors, specifically their difficulty in high dynamic range (HDR) scenes and rapid motions due to fixed integration times. By leveraging the Prophesee EVK4 event camera, which operates asynchronously with high temporal resolution and a dynamic range exceeding 120 dB, our system supports stable tracking in conditions where standard vision pipelines degrade, such as severe backlighting and very low light environments below 5 lux. The architecture integrates a low-latency Perception Module, utilizing optimized event accumulation and gravity-aligned inertial fusion, with a causal Motion Module (TWIST) that performs online kinematic retargeting. We validate the system on an embedded NVIDIA Booster T1 platform and an 18-DoF humanoid upper-body setup, demonstrating an end-to-end photon-to-action latency of 23-34 ms and advantages over RGB baselines under our experimental setup. The results indicate a practical trade-off: events can be preferable for fast or poorly lit upper-body teleoperation, whereas well-lit static scenes may favor RGB or hybrid sensing.
