差异感知的模仿学习检索策略
Difference-Aware Retrieval Policies for Imitation Learning
AI总结：
提出DARP，一种半参数检索式模仿学习方法，通过基于k近邻的局部邻域结构重参数化，解决行为克隆的分布外泛化问题，在连续控制和机器人操作任务中性能提升15-46%。
摘要：
通过行为克隆的参数化模仿学习可能因部署期间的复合误差而在分布外状态上泛化能力差。我们表明，在推理期间通过半参数检索式模仿学习方法重用训练数据可以缓解这一挑战。我们提出差异感知的模仿学习检索策略（DARP），这是一种半参数检索式模仿学习方法，通过根据局部邻域结构而非直接的状态到动作映射来重新参数化模仿学习问题，从而解决这一局限性。DARP不学习全局策略，而是训练一个模型，基于专家演示中的k近邻、它们对应的动作以及邻居状态与查询状态之间的相对距离向量来预测动作。DARP不需要超出标准行为克隆所做的额外假设——它不需要额外的数据收集、在线专家反馈或任务特定知识。我们在不同领域（包括连续控制和机器人操作）以及不同表示（包括高维视觉特征）上展示了比标准行为克隆持续15-46%的性能提升。代码和演示可在https://weirdlabuw.github.io/darp-site/获取。
英文摘要：
Parametric imitation learning via behavior cloning can suffer from poor generalization to out-of-distribution states due to compounding errors during deployment. We show that reusing the training data during inference via a semi-parametric retrieval-based imitation learning approach can alleviate this challenge. We present Difference-Aware Retrieval Policies for Imitation Learning (DARP), a semi-parametric retrieval-based imitation learning approach that addresses this limitation by reparameterizing the imitation learning problem in terms of local neighborhood structure rather than direct state-to-action mappings. Instead of learning a global policy, DARP trains a model to predict actions based on $k$-nearest neighbors from expert demonstrations, their corresponding actions, and the relative distance vectors between neighbor states and query states. DARP requires no additional assumptions beyond those made for standard behavior cloning -- it does not require additional data collection, online expert feedback, or task-specific knowledge. We demonstrate consistent performance improvements of 15-46% over standard behavior cloning across diverse domains, including continuous control and robotic manipulation, and across different representations, including high-dimensional visual features. Code and demos are available at https://weirdlabuw.github.io/darp-site/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
