RoboHitch: 从无序关键点学习视觉可供性用于系结
RoboHitch: Learning Visual Affordance from Disordered Keypoints for Hitch Knots Tying
AI总结：
提出RoboHitch框架，利用无序3D关键点和RGB图像从人类演示中学习系结，通过动态图自编码器和卷积自编码器融合特征，预测抓取和放置可供性，实现遮挡下的系结。
摘要：
由于复杂的动力学和频繁的自遮挡，可变形线性物体的机器人操作面临重大挑战。现有的机器人打结方法通常依赖于有序关键点和显式边缘连接的精确拓扑状态跟踪。这种依赖使得它们在打结过程中因重复弯曲和交叉导致的跟踪漂移和拓扑不匹配而容易失败。为了解决这些限制，我们引入了RoboHitch，一个新颖的框架，它仅使用无序的3D关键点和RGB图像从人类演示中学习执行系结。这消除了对显式拓扑顺序的需求，允许更灵活的操作。我们的方法采用动态图自编码器从未跟踪的关键点中提取几何特征，并辅以卷积自编码器捕获必要的视觉上下文。然后，双向交叉注意力机制融合这些模态，共同预测抓取和放置可供性，促进对绳子状态的隐式推理，并在遮挡下实现系结。真实世界实验证明了我们方法的有效性和泛化能力，成功完成了自遮挡场景中的系结。
英文摘要：
Robotic manipulation of deformable linear objects (DLOs) presents significant challenges due to complex dynamics and frequent self-occlusions. Existing robotic knot tying methods typically rely on precise topological state tracking with ordered keypoints and explicit edge connectivity. This reliance makes them prone to failures due to tracking drift and topology mismatch caused by repeated bending and crossings during knot formation.To address these limitations, we introduce RoboHitch, a novel framework that learns to perform hitch knot tying from human demonstrations using only disordered 3D keypoints and RGB images. This eliminates the need for explicit topological order, allowing for more flexible manipulation. Our method employs a dynamic Graph Autoencoder to extract geometric features from untracked keypoints, complemented by a Convolutional Autoencoder that captures essential visual context. A bidirectional cross-attention mechanism then fuses these modalities to jointly predict pick and place affordances, facilitating implicit reasoning about the rope's state and enabling knot tying under occlusion.Real-world experiments demonstrate the effectiveness and generalizability of our approach, successfully completing hitch knots in scenarios with self-occlusions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
