arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.25829cs.ROcs.AI
收藏

OASIS: 通过SE(3)轨迹预测实现机器人操作中的观测-动作空间对齐

OASIS: Observation-Action Space Alignment via SE(3) Trajectory Prediction for Robotic Manipulation

National Key Laboratory of Human-Machine Hybrid Augmented Intelligence(人机混合增强智能国家重点实验室);Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University(西安交通大学人工智能与机器人研究所)

Xinzhe Chen, Sihua Ren, Liqi Huang, Haowen Sun, Mingyang Li, Zeyang Liu, Xingyu Chen, Xuguang Lan

AI总结:

提出OASIS视觉运动策略,通过SE(3)末端执行器轨迹预测对齐中间表示与动作空间,在仿真和真实实验中优于VLA和WAM基线。

摘要:

最近的视觉-语言-动作(VLA)模型和世界动作模型(WAMs)通过用辅助空间特征或未来视觉状态预测丰富中间表示来推进机器人操作。然而,这些表示在很大程度上仍停留在观测空间内,不共享动作空间的刚体几何,迫使动作解码器隐式恢复该几何。我们提出OASIS,一种通过$SE(3)$末端执行器轨迹预测将中间表示与动作空间对齐的视觉运动策略。OASIS将融合视觉-语言和度量深度特征的3D感知特征编码器与生成相机帧末端执行器轨迹的$SE(3)$轨迹预测器耦合。以预测器的姿态监督隐藏状态为条件,动作解码器生成与刚体运动一致的动作块。在仿真和真实世界实验中,OASIS在成功率和分布外泛化方面优于VLA和WAM基线。我们的项目页面位于https://npuhandsome.github.io/OASIS_web。

英文摘要:

Recent vision-language-action (VLA) models and world action models (WAMs) advance robotic manipulation by enriching intermediate representations with auxiliary spatial features or future visual-state prediction. However, these representations largely remain within the observation space and do not share the rigid-body geometry of the action space, forcing the action decoder to implicitly recover this geometry. We propose OASIS, a visuomotor policy that aligns the intermediate representation with the action space via $SE(3)$ end-effector trajectory prediction. OASIS couples a 3D-aware feature encoder that fuses vision-language and metric-depth features with an $SE(3)$ trajectory predictor that produces a camera-frame end-effector trajectory. Conditioned on the predictor's pose-supervised hidden states, the action decoder generates action chunks consistent with rigid-body motion. Across simulation and real-world experiments, OASIS outperforms VLA and WAM baselines in success rate and out-of-distribution generalization. Our project page is available at https://oasis-neurips.github.io.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑