arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2611.08097cs.RO
收藏

STEP:用于感知型人形机器人运动的显式预测时空训练

STEP: Spatiotemporal Training with Explicit Prediction for Perceptive Humanoid Locomotion

Beijing University of Chemical Technology (BUCT)(北京化工大学);National University of Defense Technology (NUDT)(中国人民解放军国防科技大学);Noetix Robotics(诺泰克斯机器人公司);University of Science and Technology of China (USTC)(中国科学技术大学)

Youhao Lyu, Chenghao Sun, Zhenxuan Ding, Shuting Zhang, Ziyang Dong, Wenhao Cui

AI总结:

提出STEP单阶段框架,通过跨模态关系与动作依赖状态演化的联合监督,提升人形机器人地形遍历性能,在23自由度Noetix E1硬件试验中100次里98次成功完成多场景运动。

摘要:

对于感知型人形机器人运动,地形几何的相关性取决于机器人的身体状态以及两者在其动作下的演化。现有研究已在地形感知的感官融合和跨模态预测方面取得进展,改进了视觉与本体感觉信息的结合及关联方式。然而,仅特征聚合无法明确应学习哪些跨模态关系,而同时观测间的预测也未显式约束其依赖动作的演化。本文提出显式预测时空训练(STEP),这是一个单阶段框架,在共享控制表示内联合监督跨模态关系与依赖动作的状态演化。多个学习到的查询将深度和本体感觉标记聚合为紧凑的策略状态。双向掩码预测利用每种可见模态预测被省略模态的潜在特征,显式监督地形-身体关系。动作条件预测器将当前策略状态与执行的动作映射到下一个融合状态,其指数移动平均目标直接定义在策略表示空间中。仿真对比显示,与评估的融合变体及 ablation 相比,地形遍历性能整体提升;深度 dropout 和延迟实验证明了在退化视觉输入下两个预测目标的益处。STEP 部署在具有机载传感的 23 自由度 Noetix E1 人形机器人上,在 100 次硬件试验中成功完成了楼梯、40 厘米平台、70 厘米间隙、踏脚石及无后方地形可见性的后退楼梯遍历,共成功 98 次。

英文摘要:

For perceptive humanoid locomotion, the relevance of terrain geometry depends on the robot's body state and how both evolve under its actions. Existing work has advanced terrain-aware sensory fusion and cross-modal prediction, improving how visual and proprioceptive information is combined and related. However, feature aggregation alone does not specify which cross-modal relationships should be learned, while prediction between simultaneous observations does not explicitly constrain their action-dependent evolution. We present Spatiotemporal Training with Explicit Prediction (STEP), a one-stage framework that jointly supervises cross-modal relationships and action-dependent state evolution within a shared control representation. Multiple learned queries aggregate depth and proprioceptive tokens into a compact policy state. Bidirectional masked prediction uses each visible modality to predict latent features of the omitted modality, explicitly supervising terrain-body relationships. An action-conditioned predictor maps the current policy state and executed action to the next fused state, with an exponential-moving-average target defined directly in the policy representation space. Simulation comparisons show overall improvements in terrain traversal over evaluated fusion alternatives and ablations, while depth-dropout and delay experiments demonstrate the benefits of both predictive objectives under degraded visual input. Deployed on a 23-DoF Noetix E1 humanoid with onboard sensing, STEP succeeds in 98 of 100 hardware trials across stairs, a 40\,cm platform, a 70\,cm gap, stepping stones, and backward stair traversal without rearward terrain visibility.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑