稀疏与稠密结合：基于对应关系的刚-柔交互机器人操纵方法
Sparse Meets Dense: Correspondence Guided Robotic Manipulation with Rigid-Deformable Interactions
浏览论文内容
中文总结 AI 辅助
针对刚-柔交互操纵任务，提出混合对应关系表征方法，结合稀疏关键点与稠密对应关系，实现少量演示下的新任务一次性迁移，实验验证其有效性与适用性。
中文摘要 AI 辅助
涉及刚-柔交互的操纵任务（如挂衣服或为人类穿衣）在日常生活中十分常见，对家用机器人而言至关重要。与单物体操纵或刚体间交互相比，这类任务因存在丰富的多点接触以及交互过程中柔体的复杂动力学特性而极具挑战性。因此，以6D位姿或结构点为代表的、不包含任务特定信息的以物体为中心的表征方式，已不足以应对这类交互任务。本研究提出一种专为刚-柔交互设计的混合对应关系表征方法。首先，为捕捉复杂的交互信息，我们引入了感知结构、任务及交互的稀疏关键点，这些关键点基于刚体与柔体的全局结构生成，并通过其局部交互接触进行筛选。然而，由于柔体的高维动力学特性，在交互过程中跟踪这些稀疏关键点仍存在困难。因此，我们进一步在柔体上构建稠密对应关系，以确保在整个操纵过程中实现精准的关键点跟踪。这种混合设计结合了两种表征的优势：稀疏关键点编码丰富的、任务特定的信息，用于细粒度操纵；而稠密对应关系则保证了高效的跟踪能力，并可泛化到新的变形、形状和场景中。二者结合，使机器人仅需少量演示即可一次性迁移到新任务。大量实验证明了该方法的有效性和广泛适用性。
英文摘要
Manipulation involving rigid-deformable interactions, such as hanging clothes or dressing humans, is common in daily life, making it essential for household robots. Compared to single-object manipulation or interactions between rigid bodies, these tasks are particularly challenging due to the rich multi-point contacts and the complex dynamics of the deformable bodies during interaction. Therefore, object-centric representations such as 6D poses or structural points without task-specific information become insufficient for these interactions. In this work, we propose a hybrid correspondence-based representation tailored for rigid-deformable interactions. First, to capture intricate interaction information, we introduce structure-, task-, and interaction-aware sparse keypoints. The keypoints are generated based on the global structures of both rigid and deformable objects, and filtered by their local interaction contacts. However, tracking these sparse keypoints through the interaction remains difficult due to the high-dimensional dynamics of deformable objects. Therefore, we further construct dense correspondences on the deformable objects for accurate keypoint tracking throughout the manipulation. This hybrid design combines the advantages of both representations: sparse keypoints encode rich, task-specific information for fine-grained manipulation, while dense correspondences ensure efficient tracking and generalization to novel deformations, shapes, and scenarios. Together, they enable one-shot transfer to new tasks with minimal demonstrations. Extensive experiments demonstrate the effectiveness and broad applicability of our method.
发表机构
- School of Computer Science, PKU（北京大学计算机学院）
- School of EECS, PKU（北京大学电子工程与计算机科学学院）
- CFCS（计算与数字科学中心）
机构由 AI 辅助整理，请以论文原文为准。
