MGDA：用于离线目标条件加权监督学习的基于模型的目标数据增强
MGDA: Model-based Goal Data Augmentation for Offline Goal-conditioned Weighted Supervised Learning
AI总结：
针对GCWSL缺乏轨迹拼接能力的问题，提出基于模型的目标数据增强方法MGDA，利用动力学模型并引入局部Lipschitz连续性假设采样合适目标，有效减轻复合误差并显著提升了拼接能力与性能。
摘要：
近期，一类被称为 Goal-Conditioned Weighted Supervised Learning (GCWSL) 的最先进算法族被引入，以应对离线目标条件强化学习 (RL) 中的挑战。GCWSL 优化了目标条件 RL 目标的下界，并在各种目标达成任务中展现出卓越的性能，提供了一种简单、有效且稳定的解决方案。然而，先前的研究指出了 GCWSL 的一个关键局限性：缺乏轨迹拼接能力。为了解决这一问题，研究人员提出了目标数据增强策略来增强这些方法。尽管如此，现有技术往往难以有效地为 GCWSL 采样到合适的增强目标。在本文中，我们为目标数据增强建立了统一的原则，重点关注目标多样性、动作最优性和目标可达性。基于这些原则，我们提出了一种基于模型的目标数据增强 (MGDA) 方法，该方法利用学习到的动力学模型来采样更合适的增强目标。MGDA 独特地在学习模型中引入了局部 Lipschitz 连续性假设，以减轻复合误差的影响。实证结果表明，MGDA 显著提升了 GCWSL 方法在基于状态和基于视觉的迷宫数据集上的性能，在提升拼接能力方面超越了以往的目标数据增强技术。
英文摘要：
Recently, a state-of-the-art family of algorithms, known as Goal-Conditioned Weighted Supervised Learning (GCWSL) methods, has been introduced to tackle challenges in offline goal-conditioned reinforcement learning (RL). GCWSL optimizes a lower bound of the goal-conditioned RL objective and has demonstrated outstanding performance across diverse goal-reaching tasks, providing a simple, effective, and stable solution. However, prior research has identified a critical limitation of GCWSL: the lack of trajectory stitching capabilities. To address this, goal data augmentation strategies have been proposed to enhance these methods. Nevertheless, existing techniques often struggle to sample suitable augmented goals for GCWSL effectively. In this paper, we establish unified principles for goal data augmentation, focusing on goal diversity, action optimality, and goal reachability. Based on these principles, we propose a Model-based Goal Data Augmentation (MGDA) approach, which leverages a learned dynamics model to sample more suitable augmented goals. MGDA uniquely incorporates the local Lipschitz continuity assumption within the learned model to mitigate the impact of compounding errors. Empirical results show that MGDA significantly enhances the performance of GCWSL methods on both state-based and vision-based maze datasets, surpassing previous goal data augmentation techniques in improving stitching capabilities.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
