Sandwich-Residuals：世界模型的参数高效测试时自适应

中文摘要 AI 辅助

潜在世界模型通过在学习的表示空间中预测动作的效果来实现规划，但当测试时条件与训练不同时，其预测可能变得不可靠。现有的测试时自适应方法通过更新预训练模型的部分参数来解决这一问题，但通常需要修改数百万个参数，并需要选择要适应的内部组件。我们提出了Sandwich-Residuals，一种轻量级替代方案，它保持预训练世界模型冻结，仅学习预测器周围的小型残差修正。这些残差使用模型的自监督预测误差进行在线优化，不需要奖励、标签或源域数据。在AdaJEPA基准的21种条件下，我们的方法达到了冻结模型成功率的1.3倍，同时保留了最强AdaJEPA变体性能的95%，并减少了97-99%的自适应参数。在复合偏移下，这一优势增加到冻结模型成功率的1.9倍，同时与内部块自适应方法相当。我们还在用于3D操作的DINO-WM模型上展示了相同的自适应原理。这些结果表明，世界模型的有效测试时自适应不一定需要修改其预训练的内部权重。

英文摘要

Latent world models enable planning by predicting the effects of actions in a learned representation space, but their predictions can become unreliable when test-time conditions differ from training. Existing test-time adaptation methods address this by updating parts of the pretrained model, often modifying millions of parameters and requiring a choice of which internal components to adapt. We introduce Sandwich-Residuals, a lightweight alternative that keeps the pretrained world model frozen and learns only small residual corrections around the predictor. The residuals are optimized online using the model's self-supervised prediction error and require no rewards, labels, or source-domain data. Across 21 conditions on the AdaJEPA benchmark, our method achieves $1.3\times$ the success rate of the frozen model while retaining 95% of the performance of the strongest AdaJEPA variant and adapting 97-99% fewer parameters. Under compound shifts, this advantage increases to $1.9\times$ the success rate of the frozen model, while remaining comparable to internal block adaptation. We further demonstrate the same adaptation principle on a DINO-WM model for 3-D manipulation. These results suggest that effective test-time adaptation of world models does not necessarily require modifying their pretrained internal weights.

发表机构

• Montanuniversität Leoben（莱奥本矿业大学）

机构由 AI 辅助整理，请以论文原文为准。