学习具有部分感知的密集3D特征场以实现通用的可变形物体操作
Learning Part-Aware Dense 3D Feature Field for Generalizable Articulated Object Manipulation
AI总结：
本文提出PA3FF，一种具有部分感知的密集3D特征场，用于提升可变形物体操作的泛化能力，通过对比学习训练，实现高效且通用的机器人操作
摘要：
可变形物体操作对于各种现实世界的机器人任务至关重要，但跨多样物体的泛化仍然是主要挑战。泛化的关键在于理解功能部分（例如门把手和旋钮），这表明在不同的物体类别和形状中在哪里以及如何操作。先前的工作试图通过引入基础特征来实现泛化，但这些特征大多是2D的，并不专门考虑功能部分。当将这些2D特征提升到几何深入的3D空间时，会出现挑战，例如运行时间长、多视图不一致和空间分辨率低，几何信息不足。为了解决这些问题，我们提出了Part-Aware 3D Feature Field（PA3FF），一种新的密集3D特征，具有部分感知，用于通用的可变形物体操作。PA3FF通过从大规模标注数据集中获取3D部分提案，通过对比学习公式进行训练。给定点云作为输入，PA3FF以前馈方式预测连续的3D特征场，其中点特征之间的距离反映功能部分的接近程度：特征相似的点更可能属于同一部分。基于此特征，我们引入了Part-Aware Diffusion Policy（PADP），一种旨在提高机器人操作样本效率和泛化能力的模仿学习框架。我们在多个模拟和现实任务上评估了PADP，证明PA3FF在操作场景中始终优于多种2D和3D表示，包括CLIP、DINOv2和Grounded-SAM。除了模仿学习外，PA3FF还使各种下游方法成为可能，包括对应学习和分割任务，使其成为机器人操作的多功能基础。项目页面：https://pa3ff.github.io
英文摘要：
Articulated object manipulation is essential for various real-world robotic tasks, yet generalizing across diverse objects remains a major challenge. A key to generalization lies in understanding functional parts (e.g., door handles and knobs), which indicate where and how to manipulate across diverse object categories and shapes. Previous works attempted to achieve generalization by introducing foundation features, while these features are mostly 2D-based and do not specifically consider functional parts. When lifting these 2D features to geometry-profound 3D space, challenges arise, such as long runtimes, multi-view inconsistencies, and low spatial resolution with insufficient geometric information. To address these issues, we propose Part-Aware 3D Feature Field (PA3FF), a novel dense 3D feature with part awareness for generalizable articulated object manipulation. PA3FF is trained by 3D part proposals from a large-scale labeled dataset, via a contrastive learning formulation. Given point clouds as input, PA3FF predicts a continuous 3D feature field in a feedforward manner, where the distance between point features reflects the proximity of functional parts: points with similar features are more likely to belong to the same part. Building on this feature, we introduce the Part-Aware Diffusion Policy (PADP), an imitation learning framework aimed at enhancing sample efficiency and generalization for robotic manipulation. We evaluate PADP on several simulated and real-world tasks, demonstrating that PA3FF consistently outperforms a range of 2D and 3D representations in manipulation scenarios, including CLIP, DINOv2, and Grounded-SAM. Beyond imitation learning, PA3FF enables diverse downstream methods, including correspondence learning and segmentation tasks, making it a versatile foundation for robotic manipulation. Project page: https://pa3ff.github.io
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
