CLAR: 通过融合掩码重建与多层级对比对齐学习用于机器人操作的3D表示
CLAR: Learning 3D Representations for Robotic Manipulation by Fusing Masked Reconstruction with Multi-Level Contrastive Alignment
AI总结：
提出CLAR框架，融合掩码自编码与全局跨模态对比学习，并引入基于可变形注意力的局部自适应对齐机制，解决3D预训练中空间几何与语义细节的权衡问题，在视觉运动策略学习中达到最优性能。
摘要：
3D点云固有的空间信息对机器人操作至关重要。然而，现有的3D预训练方法面临一个基本权衡：掩码自编码（MAE）擅长捕捉空间几何特征但缺乏语义，而对比学习虽然能从2D基础模型中提炼语义，却不适合操作任务所需的细粒度细节。为解决这些挑战，我们提出CLAR，一种新颖的3D预训练框架，协同全局理解与细粒度局部对齐。我们的框架统一了MAE与全局跨模态对比学习，以整合鲁棒的空间感知与丰富的语义理解。为增强对细粒度细节的关注，在局部层面，我们引入一种自适应对齐机制，利用可变形注意力强制局部3D几何与2D视觉特征之间的精确对应，从而克服传统全局对齐在操作任务中的局限性。在仿真和真实世界中的大量实验表明，CLAR实现了最先进的性能，在视觉运动策略学习中显著优于现有方法。
英文摘要：
The spatial information inherent in 3D point clouds is crucial for robotic manipulation. However, existing 3D pre-training methods face a fundamental trade-off: Masked Autoencoding (MAE) excels at capturing spatial-geometric features but lacks semantics, whereas contrastive learning, while able to distill semantics from 2D foundation models, is ill-suited for the fine-grained details required for manipulation tasks. To address these challenges, we propose CLAR, a novel 3D pre-training framework that synergizes global understanding with fine-grained local alignment. Our framework unifies MAE with global cross-modal contrastive learning to integrate robust spatial awareness with rich semantic understanding. To enhance its focus on fine-grained details, at the local level, we introduce an adaptive alignment mechanism that leverages deformable attention to force precise correspondences between local 3D geometry and 2D visual features, thereby overcoming the limitations of conventional global alignment in manipulation tasks. Extensive experiments in simulation and the real world demonstrate that CLAR achieves state-of-the-art performance, significantly outperforming existing methods in visuomotor policy learning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
