Good in Bad (GiB): 从用户演示中筛选以学习更好的策略
Good in Bad (GiB): Sifting Through End-user Demonstrations for Learning a Better Policy
AI总结：
GiB算法通过自动识别并剔除演示中的错误子任务，保留高质量子任务，提升策略鲁棒性。
摘要：
模仿学习为机器人从人类用户获取多样化技能提供了有前景的框架。然而，大多数模仿学习算法假设能获得高质量演示，这在非专家用户收集数据时是不现实的，因为演示常包含无意错误。直接学习此类演示可能导致不安全的策略行为，而因偶尔错误丢弃整个演示会浪费宝贵数据，尤其是在数据稀缺的情况下。本文介绍GiB（Good-in-Bad）算法，该算法可自动识别并剔除演示中的错误子任务，同时保留高质量子任务。过滤后的数据可被任何策略学习算法用于训练更稳健的策略。GiB首先训练一个自监督模型学习潜在特征，并为每个演示分配二元权重以标记为好或坏。然后建模高质量片段的潜在特征分布，并使用马氏距离检测和评估低质量子任务。我们在Franka机器人上验证GiB在模拟和现实多步任务中的性能，证明在学习混合质量的人类演示时，策略性能得到提升。
英文摘要：
Imitation learning offers a promising framework for enabling robots to acquire diverse skills from human users. However, most imitation learning algorithms assume access to high-quality demonstrations an unrealistic expectation when collecting data from non-expert users, whose demonstrations often contain inadvertent errors. Naively learning from such demonstrations can result in unsafe policy behavior, while discarding entire demonstrations due to occasional mistakes wastes valuable data, especially in low-data settings. In this work, we introduce GiB (Good-in-Bad), an algorithm that automatically identifies and discards erroneous subtasks within demonstrations while preserving high-quality subtasks. The filtered data can then be used by any policy learning algorithm to train more robust policies. GiB first trains a self-supervised model to learn latent features and assigns binary weights to label each demonstration as good or bad. It then models the latent feature distribution of high-quality segments and uses the Mahalanobis distance to detect and evaluate poor-quality subtasks. We validate GiB on the Franka robot in both simulated and real-world multi-step tasks, demonstrating improved policy performance when learning from mixed-quality human demonstrations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
