DexStitch：通过接触感知机器人图像修复从人类视频中学习与组合灵巧技能
DexStitch: Learning and Composing Dexterous Skills from Human Videos via Contact-Aware Robotic Inpainting
AI总结：
DexStitch 是从人类视频学习并拼接机器人灵巧技能的框架，通过接触感知图像修复实现技能迁移，在家庭任务中取得高单技能与序列成功率。
摘要：
长 horizon 操纵任务由基础技能及连接它们的过渡环节构成。学习可按顺序执行的技能需要对技能和过渡环节两者进行监督，而在机器人上收集这类监督的成本很高。人类视频是一种可扩展的演示来源，但将其迁移到机器人需要提取机器人兼容的观测值、动作以及对应的技能结构。我们提出 DexStitch，这一框架可将长 horizon 人类视频切割为技能和过渡环节，将其转换为对齐的机器人观测值-动作-技能数据，并将学习到的技能拼接为请求的序列。技能分类器首先将视频分割为操纵技能和过渡环节。随后，DexStitch 将人类演示转换为机器人观测值和动作，同时保留对应的技能标签。在转换过程中，逐指的人-物接触引导灵巧手重定向和机器人图像修复，保留接触几何结构和手指-物体遮挡顺序。为将学习到的技能拼接为序列，我们在生成的合成数据上预训练一个共享的技能条件策略，并在合成数据与少量单技能远程操作片段的混合数据上进行微调。以当前和下一个技能为条件引导每个过渡环节朝向后续技能，而预测的技能进展实现闭环切换。在保留的会话中，分类器与手动标注的帧一致性达92.4%；在四项家庭技能上，DexStitch 实现了80.0%的单技能成功率和77.5%的序列成功率。
英文摘要：
A long-horizon manipulation task consists of primitive skills and the transitions connecting them. Learning skills that can be executed in sequence requires supervision for both, which is costly to collect on a robot. Human videos offer a scalable source of demonstrations, but transferring them to robots requires extracting robot-compatible observations and actions together with their skill structure. We present DexStitch, a framework that cuts long-horizon human videos into skills and transitions, converts them into aligned robot observation-action-skill data, and stitches the learned skills into a requested sequence. A skill classifier first segments the videos into manipulation skills and transitions. DexStitch then converts the human demonstrations into robot observations and actions while retaining their corresponding skill labels. During conversion, per-finger human-object contacts guide dexterous hand retargeting and robotic inpainting, preserving contact geometry and finger-object occlusion ordering. To stitch the learned skills into a sequence, a shared skill-conditioned policy is pretrained on the resulting synthetic data and finetuned on a mixture of synthetic data and a small set of single-skill teleoperation episodes. Conditioning on the current and next skills guides each transition toward the following skill, while predicted skill progression enables closed-loop switching. On a held-out session, the classifier agrees with manual annotations on 92.4% of frames. Across four household skills, DexStitch achieves 80.0% single-skill success and 77.5% sequence success.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
