KineDex：通过动觉示教学习触觉信息引导的视觉运动策略以实现灵巧操作
KineDex: Learning Tactile-Informed Visuomotor Policies via Kinesthetic Teaching for Dexterous Manipulation
AI总结：
KineDex通过手把手动觉示教直接传递运动并采集真实触觉反馈，结合图像修复和力控制训练视觉运动策略，在接触密集型灵巧操作任务中显著提升成功率和数据收集效率。
摘要：
收集富含细粒度触觉信息的演示对于灵巧操作至关重要，尤其是在需要精确力控制和物理交互的接触密集型任务中。先前的工作主要聚焦于遥操作或基于视频的重定向，但它们常常受到运动学不匹配和缺乏实时触觉反馈的困扰，阻碍了高保真触觉数据的获取。为缓解这一问题，我们提出了KineDex，一种手把手式的动觉示教范式，其中操作者的运动被直接传递到灵巧手，从而能够收集具有准确触觉反馈的物理真实演示。为解决人手造成的遮挡，我们应用图像修复技术对视觉观测进行预处理。基于这些演示，我们随后使用触觉增强输入训练视觉运动策略，并在部署过程中实施力控制以实现精确的接触密集型操作。我们在一系列具有挑战性的接触密集型操作任务上评估了KineDex，包括特别困难的场景，例如将牙膏挤到牙刷上，这需要精确的多指协调和稳定的力调节。在这些任务中，KineDex实现了74.4%的平均成功率，比无力控制的变体提高了57.7%。与遥操作的对比实验和用户研究进一步验证了KineDex在数据收集效率和可操作性方面的优势。具体而言，在两个不同难度的任务中，KineDex收集数据的速度是遥操作的两倍以上，同时保持接近100%的成功率，而遥操作的成功率低于50%。
英文摘要：
Collecting demonstrations enriched with fine-grained tactile information is critical for dexterous manipulation, particularly in contact-rich tasks that require precise force control and physical interaction. While prior works primarily focus on teleoperation or video-based retargeting, they often suffer from kinematic mismatches and the absence of real-time tactile feedback, hindering the acquisition of high-fidelity tactile data. To mitigate this issue, we propose KineDex, a hand-over-hand kinesthetic teaching paradigm in which the operator's motion is directly transferred to the dexterous hand, enabling the collection of physically grounded demonstrations enriched with accurate tactile feedback. To resolve occlusions from human hand, we apply inpainting technique to preprocess the visual observations. Based on these demonstrations, we then train a visuomotor policy using tactile-augmented inputs and implement force control during deployment for precise contact-rich manipulation. We evaluate KineDex on a suite of challenging contact-rich manipulation tasks, including particularly difficult scenarios such as squeezing toothpaste onto a toothbrush, which require precise multi-finger coordination and stable force regulation. Across these tasks, KineDex achieves an average success rate of 74.4%, representing a 57.7% improvement over the variant without force control. Comparative experiments with teleoperation and user studies further validate the advantages of KineDex in data collection efficiency and operability. Specifically, KineDex collects data over twice as fast as teleoperation across two tasks of varying difficulty, while maintaining a near-100% success rate, compared to under 50% for teleoperation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
