面向扩散规划器的状态覆盖轨迹拼接
State-Covering Trajectory Stitching for Diffusion Planners
AI总结：
针对扩散规划器受限于离线数据质量与多样性而难以泛化的问题，提出无奖励轨迹增强方法 SCoTS，通过学习时间距离保持的潜在表示并迭代拼接轨迹片段来覆盖和扩展潜在空间，显著提升了扩散规划器及离线强化学习算法在长程推理任务上的性能与泛化能力。
摘要：
基于扩散的生成模型正成为强化学习（RL）中长程规划的有力工具，特别是在离线数据集场景下。然而，其性能从根本上受限于训练数据的质量和多样性。这通常限制了它们对训练分布之外的任务或更长规划时程的泛化能力。为克服这一挑战，我们提出了 State-Covering Trajectory Stitching (SCoTS)，一种新颖的无奖励轨迹增强方法，该方法增量地拼接短轨迹片段，系统地生成多样且扩展的轨迹。SCoTS 首先学习一种保持时间距离的潜在表示，以捕捉环境的底层时间结构，然后在方向探索和新颖性的引导下迭代拼接轨迹片段，从而有效地覆盖和扩展该潜在空间。我们证明，在需要拼接和长程推理的离线目标条件基准上，SCoTS 显著提升了扩散规划器的性能和泛化能力。此外，由 SCoTS 生成的增强轨迹显著提升了广泛使用的离线目标条件 RL 算法在多种环境中的性能。
英文摘要：
Diffusion-based generative models are emerging as powerful tools for long-horizon planning in reinforcement learning (RL), particularly with offline datasets. However, their performance is fundamentally limited by the quality and diversity of training data. This often restricts their generalization to tasks outside their training distribution or longer planning horizons. To overcome this challenge, we propose State-Covering Trajectory Stitching (SCoTS), a novel reward-free trajectory augmentation method that incrementally stitches together short trajectory segments, systematically generating diverse and extended trajectories. SCoTS first learns a temporal distance-preserving latent representation that captures the underlying temporal structure of the environment, then iteratively stitches trajectory segments guided by directional exploration and novelty to effectively cover and expand this latent space. We demonstrate that SCoTS significantly improves the performance and generalization capabilities of diffusion planners on offline goal-conditioned benchmarks requiring stitching and long-horizon reasoning. Furthermore, augmented trajectories generated by SCoTS significantly improve the performance of widely used offline goal-conditioned RL algorithms across diverse environments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
