如何可证明地改进回报条件监督学习？
How to Provably Improve Return Conditioned Supervised Learning?
AI总结：
针对RCSL缺乏轨迹拼接能力、受限于离线数据策略质量的问题，本文提出Reinforced RCSL框架，通过分布内最优return-to-go机制实现可证明的性能改进，并在多个基准上验证显著提升。
摘要：
在序贯决策问题中，Return-Conditioned Supervised Learning（RCSL，回报条件监督学习）因其在现代决策任务中的简洁性与稳定性而获得越来越多认可。与传统离线强化学习（RL）算法不同，RCSL通过同时将状态和回报作为输入，把策略学习构建为一个监督学习问题。该方法消除了离线RL中常与时序差分（TD）学习相关的不稳定性。然而，RCSL一直因缺乏拼接特性而受到批评，这意味着其性能本质上受限于用于生成离线数据集的策略质量。为解决这一局限，我们提出了一个有原则且简单的框架，名为Reinforced RCSL。该框架的关键创新是引入我们所称的分布内最优回报待付（in-distribution optimal return-to-go）概念。该机制利用我们的策略，根据当前状态识别数据集中可实现的最佳未来回报，从而无需复杂的回报增强技术。我们的理论分析表明，Reinforced RCSL能够持续优于标准RCSL方法。实证结果进一步验证了我们的主张，显示其在一系列基准测试上均取得显著性能提升。
英文摘要：
In sequential decision-making problems, Return-Conditioned Supervised Learning (RCSL) has gained increasing recognition for its simplicity and stability in modern decision-making tasks. Unlike traditional offline reinforcement learning (RL) algorithms, RCSL frames policy learning as a supervised learning problem by taking both the state and return as input. This approach eliminates the instability often associated with temporal difference (TD) learning in offline RL. However, RCSL has been criticized for lacking the stitching property, meaning its performance is inherently limited by the quality of the policy used to generate the offline dataset. To address this limitation, we propose a principled and simple framework called Reinforced RCSL. The key innovation of our framework is the introduction of a concept we call the in-distribution optimal return-to-go. This mechanism leverages our policy to identify the best achievable in-dataset future return based on the current state, avoiding the need for complex return augmentation techniques. Our theoretical analysis demonstrates that Reinforced RCSL can consistently outperform the standard RCSL approach. Empirical results further validate our claims, showing significant performance improvements across a range of benchmarks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
