GTA：用于离线强化学习的带引导的生成式轨迹增强
GTA: Generative Trajectory Augmentation with Guidance for Offline Reinforcement Learning
AI总结：
GTA是一种基于扩散模型的生成式轨迹增强方法，通过部分加噪和回报引导去噪生成高奖励且动态合理的轨迹，提升离线强化学习算法的性能和数据质量。
摘要：
离线强化学习（Offline RL）面临着从静态数据集学习有效决策策略而不进行任何在线交互的挑战。数据增强技术，如噪声注入和数据合成，旨在通过平滑学习到的状态-动作区域来改进Q函数逼近。然而，这些方法往往在直接提升离线数据集质量方面效果不佳，导致结果次优。为此，我们提出了GTA（生成式轨迹增强），一种新颖的生成式数据增强方法，旨在通过增强轨迹使其既具有高奖励又具有动态合理性来丰富离线数据。GTA在数据增强框架中应用扩散模型。GTA对原始轨迹进行部分加噪，然后通过基于放大回报值的分类器自由引导进行去噪。我们的结果表明，GTA作为一种通用数据增强策略，在具有独特挑战的各种任务上提升了广泛使用的离线RL算法的性能。此外，我们对GTA增强的数据进行了质量分析，并证明GTA提高了数据质量。我们的代码可在https://github.com/Jaewoopudding/GTA获取。
英文摘要：
Offline Reinforcement Learning (Offline RL) presents challenges of learning effective decision-making policies from static datasets without any online interactions. Data augmentation techniques, such as noise injection and data synthesizing, aim to improve Q-function approximation by smoothing the learned state-action region. However, these methods often fall short of directly improving the quality of offline datasets, leading to suboptimal results. In response, we introduce GTA, Generative Trajectory Augmentation, a novel generative data augmentation approach designed to enrich offline data by augmenting trajectories to be both high-rewarding and dynamically plausible. GTA applies a diffusion model within the data augmentation framework. GTA partially noises original trajectories and then denoises them with classifier-free guidance via conditioning on amplified return value. Our results show that GTA, as a general data augmentation strategy, enhances the performance of widely used offline RL algorithms across various tasks with unique challenges. Furthermore, we conduct a quality analysis of data augmented by GTA and demonstrate that GTA improves the quality of the data. Our code is available at https://github.com/Jaewoopudding/GTA
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
