arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2401.01734cs.CV
收藏

利用合成数据学习用于机器人布料操作的关键点

Learning Keypoints for Robotic Cloth Manipulation using Synthetic Data

Ghent University(根特大学);imec(imec(校际微电子中心))

Thomas Lips, Victor-Louis De Gusseme, Francis wyffels

AI总结:

针对布料操作中合成数据存在的sim-to-real gap问题,提出用于训练近乎展平布料关键点检测器的合成数据流水线,经真实数据集验证,微调后mAP达74%,并分析了失效模式与差距优化方向。

摘要:

辅助机器人应当具备清洗、折叠或熨烫衣物的能力。然而,由于衣物的多样性、可变形性以及自遮挡特性,构建用于布料操作的机器人系统极具挑战性。合成数据是提升模型泛化能力的一个极具前景的方向,但仿真到现实的差距(sim-to-real gap)限制了其有效性。为推动合成数据在机器人折叠等布料操作任务中的应用,我们提出了一种合成数据流水线,用于训练针对近乎展平的布料物品的关键点检测器。为评估其性能,我们还收集了一个真实世界数据集。我们针对T恤、毛巾和短裤训练了检测器,获得了64%的平均精度(average precision)以及18像素的平均关键点距离。在真实世界数据上进行微调后,性能提升至74%的平均精度均值(mAP),平均距离仅为9像素。此外,我们描述了关键点检测器的失效模式,并对比了获取布料网格和材质的不同方法。我们还量化了剩余的仿真到现实差距,并指出要进一步缩小这一差距,需要提升布料资产的保真度。代码、数据集和训练好的模型已公开。

英文摘要:

Assistive robots should be able to wash, fold or iron clothes. However, due to the variety, deformability and self-occlusions of clothes, creating robot systems for cloth manipulation is challenging. Synthetic data is a promising direction to improve generalization, but the sim-to-real gap limits its effectiveness. To advance the use of synthetic data for cloth manipulation tasks such as robotic folding, we present a synthetic data pipeline to train keypoint detectors for almost-flattened cloth items. To evaluate its performance, we have also collected a real-world dataset. We train detectors for both T-shirts, towels and shorts and obtain an average precision of 64% and an average keypoint distance of 18 pixels. Fine-tuning on real-world data improves performance to 74% mAP and an average distance of only 9 pixels. Furthermore, we describe failure modes of the keypoint detectors and compare different approaches to obtain cloth meshes and materials. We also quantify the remaining sim-to-real gap and argue that further improvements to the fidelity of cloth assets will be required to further reduce this gap. The code, dataset and trained models are available

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑