RoboPCA：基于姿态的从人类示范中学习空间可及性的方法用于机器人操作
RoboPCA: Pose-centered Affordance Learning from Human Demonstrations for Robot Manipulation
AI总结：
RoboPCA通过联合预测接触区域和姿态，提升机器人操作中从人类示范中学习空间可及性的性能。
摘要：
理解空间可及性——包括物体交互的接触区域和相应的接触姿态——对于机器人有效操作物体和完成多样化任务至关重要。然而，现有的空间可及性预测方法主要集中在定位接触区域，而将姿态交给独立的姿态估计方法，这可能导致由于预测的接触区域与候选姿态之间不一致而任务失败。在本工作中，我们提出了RoboPCA，一种以姿态为中心的可及性预测框架，该框架根据指令联合预测任务合适的接触区域和姿态。为了实现以姿态为中心的可及性学习的可扩展数据收集，我们设计了Human2Afford，一个数据整理流程，可以自动恢复场景级3D信息，并从人类示范中推断出以姿态为中心的可及性注释。通过Human2Afford，场景深度和交互物体的掩码被提取，以提供3D上下文和物体定位，同时通过跟踪接触区域内的物体点并分析手-物体交互模式，建立从3D手网格到机器人末端执行器方向的映射。通过将几何-外观线索通过RGB-D编码器整合，并将增强的掩码特征融入扩散框架中，RoboPCA在图像数据集、模拟和真实机器人上均优于基线方法，并在任务和类别之间表现出强大的泛化能力。
英文摘要：
Understanding spatial affordances -- comprising the contact regions of object interaction and the corresponding contact poses -- is essential for robots to effectively manipulate objects and accomplish diverse tasks. However, existing spatial affordance prediction methods mainly focus on locating the contact regions while delegating the pose to independent pose estimation approaches, which can lead to task failures due to inconsistencies between predicted contact regions and candidate poses. In this work, we propose RoboPCA, a pose-centered affordance prediction framework that jointly predicts task-appropriate contact regions and poses conditioned on instructions. To enable scalable data collection for pose-centered affordance learning, we devise Human2Afford, a data curation pipeline that automatically recovers scene-level 3D information and infers pose-centered affordance annotations from human demonstrations. With Human2Afford, scene depth and the interaction object's mask are extracted to provide 3D context and object localization, while pose-centered affordance annotations are obtained by tracking object points within the contact region and analyzing hand-object interaction patterns to establish a mapping from the 3D hand mesh to the robot end-effector orientation. By integrating geometry-appearance cues through an RGB-D encoder and incorporating mask-enhanced features to emphasize task-relevant object regions into the diffusion-based framework, RoboPCA outperforms baseline methods on image datasets, simulation, and real robots, and exhibits strong generalization across tasks and categories.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
