面向少样本模仿学习的带重要性权重的数据检索方法
Data Retrieval with Importance Weights for Few-Shot Imitation Learning
AI总结：
针对少样本模仿学习中现有基于检索的数据选择方法存在高方差、未考虑先验数据分布的问题，提出重要性加权检索方法，通过高斯核密度估计分布比值优化检索，在仿真和真实场景均提升了现有方法性能。
摘要：
尽管大规模机器人数据集推动了模仿学习的近期进展，但在新环境和未见过的任务中部署时，从规模较小的特定任务数据集学习仍然至关重要。少样本模仿学习的一类方法是基于检索的模仿学习，该方法从大规模、广泛可用的先验数据集中提取相关样本，以扩充有限的演示数据集。为了从先验数据集中确定相关数据，基于检索的方法最常计算先验数据点与目标数据集中的点在隐空间中的最小距离。虽然基于检索的方法使用该指标进行数据选择已取得成效，但我们证明了它等价于目标数据分布的高斯核密度估计（KDE）的极限情况。这揭示了先前工作中使用的检索规则的两个缺陷：首先，它依赖高方差的最近邻估计，易受噪声影响；其次，它在检索数据时未考虑先验数据的分布。为解决这些问题，我们提出了重要性加权检索（IWR），该方法使用高斯KDE估计重要性权重，即用于检索的目标数据分布与先验数据分布之间的比值。通过考虑概率比值，IWR旨在缓解先前选择规则的偏差；通过使用合理的建模参数，IWR利用所有数据点有效平滑了估计结果。在仿真环境和基于Bridge数据集的真实世界评估中，我们发现我们的方法IWR仅需少量修改，就能持续提升现有基于检索的方法的性能。
英文摘要：
While large-scale robot datasets have propelled recent progress in imitation learning, learning from smaller task specific datasets remains critical for deployment in new environments and unseen tasks. One such approach to few-shot imitation learning is retrieval-based imitation learning, which extracts relevant samples from large, widely available prior datasets to augment a limited demonstration dataset. To determine the relevant data from prior datasets, retrieval-based approaches most commonly calculate a prior data point's minimum distance to a point in the target dataset in latent space. While retrieval-based methods have shown success using this metric for data selection, we demonstrate its equivalence to the limit of a Gaussian kernel density (KDE) estimate of the target data distribution. This reveals two shortcomings of the retrieval rule used in prior work. First, it relies on high-variance nearest neighbor estimates that are susceptible to noise. Second, it does not account for the distribution of prior data when retrieving data. To address these issues, we introduce Importance Weighted Retrieval (IWR), which estimates importance weights, or the ratio between the target and prior data distributions for retrieval, using Gaussian KDEs. By considering the probability ratio, IWR seeks to mitigate the bias of previous selection rules, and by using reasonable modeling parameters, IWR effectively smooths estimates using all data points. Across both simulation environments and real-world evaluations on the Bridge dataset we find that our method, IWR, consistently improves performance of existing retrieval-based methods, despite only requiring minor modifications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
