从无奖励离线数据中学习：基于潜在动力学模型规划的案例研究
Learning from Reward-Free Offline Data: A Case for Planning with Latent Dynamics Models
AI总结：
该研究针对无奖励离线数据场景，系统对比了强化学习与基于潜在动力学模型的规划方法在导航任务中的表现，发现基于模型的规划泛化性和数据效率更优，是处理次优离线数据的有效途径。
摘要：
人工智能领域的一个长期目标是开发能够在多种环境中解决各类任务的智能体，包括训练期间从未见过的环境。应对这一挑战的两种主流范式是：（i）强化学习（RL），通过试错学习策略；（ii）最优控制，使用已知或习得的动力学模型规划动作。然而，在离线场景下——即智能体必须从无奖励轨迹中学习的场景——它们的相对优势仍未得到充分探究。在本研究中，我们使用不同质量的离线数据集，在一系列导航任务上系统评估了基于RL和基于控制的方法。在RL方面，我们考虑了目标条件化方法和零样本方法。在控制方面，我们使用联合嵌入预测架构（JEPA）训练了一个潜在动力学模型，并将其用于规划。我们研究了数据多样性、轨迹质量和环境可变性等因素如何影响这些方法的性能。我们的结果表明，无模型RL从大量高质量数据中获益最多，而基于模型的规划对未见过的布局泛化能力更强，数据效率更高，同时实现了与领先的无模型方法相当的轨迹拼接性能。值得注意的是，使用潜在动力学模型进行规划被证明是处理次优离线数据和适应多样化环境的有效方法。
英文摘要：
A long-standing goal in AI is to develop agents capable of solving diverse tasks across a range of environments, including those never seen during training. Two dominant paradigms address this challenge: (i) reinforcement learning (RL), which learns policies via trial and error, and (ii) optimal control, which plans actions using a known or learned dynamics model. However, their comparative strengths in the offline setting - where agents must learn from reward-free trajectories - remain underexplored. In this work, we systematically evaluate RL and control-based methods on a suite of navigation tasks, using offline datasets of varying quality. On the RL side, we consider goal-conditioned and zero-shot methods. On the control side, we train a latent dynamics model using the Joint Embedding Predictive Architecture (JEPA) and employ it for planning. We investigate how factors such as data diversity, trajectory quality, and environment variability influence the performance of these approaches. Our results show that model-free RL benefits most from large amounts of high-quality data, whereas model-based planning generalizes better to unseen layouts and is more data-efficient, while achieving trajectory stitching performance comparable to leading model-free methods. Notably, planning with a latent dynamics model proves to be a strong approach for handling suboptimal offline data and adapting to diverse environments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
