DiffStitch：利用基于扩散的轨迹拼接提升离线强化学习
DiffStitch: Boosting Offline Reinforcement Learning with Diffusion-based Trajectory Stitching
AI总结：
本文提出DiffStitch，一种基于扩散的轨迹拼接数据增强方法，通过连接低奖励与高奖励轨迹生成全局最优轨迹，在D4RL上显著提升多种离线强化学习算法的性能。
摘要：
在离线强化学习（RL）中，所学策略的性能高度依赖于离线数据集的质量。然而，在许多情况下，离线数据集包含的最优轨迹非常有限，这给离线RL算法带来了挑战，因为智能体必须获得向高奖励区域转移的能力。为了解决这一问题，我们提出了基于扩散的轨迹拼接（DiffStitch），这是一种新颖的基于扩散的数据增强流程，能够系统地在轨迹之间生成拼接转移。DiffStitch有效地将低奖励轨迹与高奖励轨迹连接起来，形成全局最优轨迹，以应对离线RL算法所面临的挑战。在D4RL数据集上进行的实证实验证明了DiffStitch在多种RL方法上的有效性。值得注意的是，DiffStitch在一阶段方法（IQL）、模仿学习方法（TD3+BC）和轨迹优化方法（DT）的性能上均表现出显著提升。
英文摘要：
In offline reinforcement learning (RL), the performance of the learned policy highly depends on the quality of offline datasets. However, in many cases, the offline dataset contains very limited optimal trajectories, which poses a challenge for offline RL algorithms as agents must acquire the ability to transit to high-reward regions. To address this issue, we introduce Diffusion-based Trajectory Stitching (DiffStitch), a novel diffusion-based data augmentation pipeline that systematically generates stitching transitions between trajectories. DiffStitch effectively connects low-reward trajectories with high-reward trajectories, forming globally optimal trajectories to address the challenges faced by offline RL algorithms. Empirical experiments conducted on D4RL datasets demonstrate the effectiveness of DiffStitch across RL methodologies. Notably, DiffStitch demonstrates substantial enhancements in the performance of one-step methods (IQL), imitation learning methods (TD3+BC), and trajectory optimization methods (DT).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
