arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2606.12759cs.RO
收藏

Sparse2Act: 学习跨域机器人操作的动作对齐稀疏3D表示

Sparse2Act: Learning Action-Aligned Sparse 3D Representations for Cross-Domain Robot Manipulation

University of California, Los Angeles(加州大学洛杉矶分校);University of California, San Diego(加州大学圣迭戈分校);University of Utah(犹他大学)

Yu Guo, Chang Yu, Siyu Ma, Yunuo Chen, Yin Yang, Ying Nian Wu, Chenfanfu Jiang

AI总结:

提出Sparse2Act框架,通过动作对齐的掩码稀疏3D编码预训练,实现跨域机器人操作,在LIBERO-10上达86.9%成功率,并支持域迁移和sim-to-real。

摘要:

显式3D表示对于操作任务具有吸引力,因为它们以度量坐标暴露物体形状、工作空间几何以及机器人-物体关系。然而,稀疏3D编码器通常通过下游任务目标学习,将表示与特定数据分布、策略架构和动作参数化绑定。我们引入Sparse2Act,一个用于预训练稀疏点云编码器的观察-动作对齐框架。关键思想是使用任务空间末端执行器动作作为几何监督:训练掩码稀疏3D令牌以组织场景特征,使其围绕与观察配对的工作空间运动。预训练后,仅编码器初始化被下游策略重用,允许它们保留自己的架构和动作空间,包括关节空间命令。在LIBERO-10基准上,我们的方法在500步微调后达到86.9%的平均成功率。相同的预训练编码器支持LIBERO到Meta-World的跨域迁移,在Meta-World-5基准上达到73.4%的平均成功率。关于目标和解码器容量的消融实验表明,增益来自掩码动作对齐信号,并且在下游动作解码器中仍然有用。在真实世界实验中,模拟预训练后跟有限真实数据微调,在四个任务上平均成功率达到72.5%,展示了有效的模拟到真实迁移。这些结果表明,机器人动作可以为可重用的稀疏3D表示提供紧凑的几何监督。

英文摘要:

Explicit 3D representations are attractive for manipulation because they expose object shape, workspace geometry, and robot-object relations in metric coordinates. However, sparse 3D encoders are often learned through downstream task objectives, tying the representation to a particular data distribution, policy architecture, and action parameterization. We introduce Sparse2Act, an observation-action alignment framework for pretraining sparse point-cloud encoders. The key idea is to use task-space end-effector actions as geometric supervision: masked sparse 3D tokens are trained to organize scene features around the workspace motion paired with the observation. After pretraining, only the encoder initialization is reused by downstream policies, allowing them to retain their own architectures and action spaces, including joint-space commands. On the LIBERO-10 benchmark, our method achieves 86.9% average success after 500 fine-tuning steps. The same pretrained encoder supports LIBERO-to-Meta-World cross-domain transfer, achieving 73.4% average success on the Meta-World-5 benchmark. Ablations on the objective and decoder capacity show that the gains come from the masked action-alignment signal and remain useful across downstream action decoders. In real-world experiments, simulation pretraining followed by limited real-data fine-tuning achieves an average success rate of 72.5% across four tasks, demonstrating effective sim-to-real transfer. These results suggest that robot actions can provide compact geometric supervision for reusable sparse 3D representations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑