离线目标条件强化学习对未见目标泛化的关键要素是什么？
What is Essential for Unseen Goal Generalization of Offline Goal-conditioned RL?
AI总结：
本文研究离线GCRL对未见目标的OOD泛化，结合实证发现与理论分析提出GOAT，并在含9个IID和17个OOD任务的新基准上显著超越现有方法。
摘要：
离线目标条件强化学习（GCRL）提供了一种从完全离线数据集中训练通用智能体的方法。除了在数据集内保持保守性之外，实现未见目标的泛化能力是离线GCRL面临的另一项根本挑战。然而，据我们所知，该问题尚未得到充分研究。本文从理论和实证两方面研究离线GCRL的分布外（OOD）泛化，以识别重要因素。在大量实验中，我们观察到加权模仿学习比基于悲观主义的离线强化学习方法具有更好的泛化性。基于这一洞见，我们推导了一套OOD泛化理论，刻画了若干重要的设计选择。随后，我们结合理论研究与实证研究的发现，提出了一种新的离线GCRL方法，即可泛化离线目标条件强化学习（Generalizable Offline goAl-condiTioned RL，GOAT）。在一个包含9个独立同分布（IID）任务和17个OOD任务的新基准上，GOAT大幅优于当前最先进的方法。
英文摘要：
Offline goal-conditioned RL (GCRL) offers a way to train general-purpose agents from fully offline datasets. In addition to being conservative within the dataset, the generalization ability to achieve unseen goals is another fundamental challenge for offline GCRL. However, to the best of our knowledge, this problem has not been well studied yet. In this paper, we study out-of-distribution (OOD) generalization of offline GCRL both theoretically and empirically to identify factors that are important. In a number of experiments, we observe that weighted imitation learning enjoys better generalization than pessimism-based offline RL method. Based on this insight, we derive a theory for OOD generalization, which characterizes several important design choices. We then propose a new offline GCRL method, Generalizable Offline goAl-condiTioned RL (GOAT), by combining the findings from our theoretical and empirical studies. On a new benchmark containing 9 independent identically distributed (IID) tasks and 17 OOD tasks, GOAT outperforms current state-of-the-art methods by a large margin.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
