迈向解析与提升世界模型中的泛化能力
Towards Unraveling and Improving Generalization in World Models
AI总结：
本文通过随机微分方程框架分析世界模型中潜在表示误差对泛化的影响，发现零漂移误差可作隐式正则化，并提出Jacobian正则化以缓解非零漂移误差，从而提升训练稳定性、收敛速度和长时程预测精度。
摘要：
世界模型近年来已成为强化学习（RL）中一种颇具前景的方法，在广泛的视觉控制任务上取得了最先进的性能。本工作旨在深入理解世界模型的鲁棒性与泛化能力。基于这一动机，我们将世界模型学习视为一个随机动力系统，建立了一个随机微分方程形式化框架，并针对零漂移表示误差和非零漂移表示误差两种情况，刻画了潜在表示误差对鲁棒性和泛化能力的影响。我们基于理论和实验研究得出的发现有些出人意料：在零漂移情况下，适度的潜在表示误差实际上可以起到隐式正则化的作用，从而提升鲁棒性。我们进一步提出了一种Jacobian正则化方案，以缓解非零漂移带来的复合误差传播效应，从而增强训练稳定性和鲁棒性。我们的实验研究证实，该正则化方法不仅能稳定训练，还能加速收敛并提高长时程预测的准确性。
英文摘要：
World models have recently emerged as a promising approach to reinforcement learning (RL), achieving state-of-the-art performance across a wide range of visual control tasks. This work aims to obtain a deep understanding of the robustness and generalization capabilities of world models. Thus motivated, we develop a stochastic differential equation formulation by treating the world model learning as a stochastic dynamical system, and characterize the impact of latent representation errors on robustness and generalization, for both cases with zero-drift representation errors and with non-zero-drift representation errors. Our somewhat surprising findings, based on both theoretic and experimental studies, reveal that for the case with zero drift, modest latent representation errors can in fact function as implicit regularization and hence result in improved robustness. We further propose a Jacobian regularization scheme to mitigate the compounding error propagation effects of non-zero drift, thereby enhancing training stability and robustness. Our experimental studies corroborate that this regularization approach not only stabilizes training but also accelerates convergence and improves accuracy of long-horizon prediction.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
