发表机构
Kaliber Labs（卡利伯实验室）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对JEPA式世界模型的物理表示懒惰问题，提出训练时添加轻量傅里叶辅助头的方法，可提升动态环境等的规划成功率与数据效率，避免表示懒惰。
AI 中文摘要
潜在世界模型作为在潜在空间而非像素空间中进行预测与规划的方法，正变得越来越流行。近期的架构如LeWorldModel（LeWM），会使用SIGReg等正则化技术联合训练编码器与预测器，以防止表示崩溃。即便有这类正则化防止表示崩溃，我们仍发现了一种新的世界模型失效模式——“物理表示懒惰”，在高度动态环境中尤为明显。在这些懒惰情形下，学习到的潜在状态并未崩溃，但仍无法表示关键物理属性，导致普遍存在的下游规划失败。为解决该问题，我们提出在训练时使用轻量的“傅里叶辅助头”进行辅助监督，该方法可强制潜在空间的物理感知结构，且无额外推理时间成本，能推广到任何环境。实验表明，在基线LeWM表现出物理表示懒惰的动态环境中，该辅助头大幅提升了规划成功率；在其他环境中，即便基线未表现出物理表示懒惰，也能带来适度提升。我们还观察到，更优的规划性能与潜在空间和关键物理属性的更高相关性相伴，这既表明我们的方法能对潜在状态进行物理结构化，也说明学习到的表示被物理结构化后对规划有潜在益处。在低数据 regime 下，辅助监督在提升成功率方面影响尤为显著。这些发现支持使用我们的傅里叶辅助头方法，以提高整体成功率和数据效率，同时避免潜在世界模型中的表示懒惰。
英文摘要
Latent world models have become increasingly popular as a method to predict and plan in latent space rather than pixel space. Recent architectures, such as LeWorldModel (LeWM), jointly train the encoder and predictor using regularization techniques like SIGReg to prevent representation collapse. Even with such regularization preventing representation collapse, we identify a new world model failure mode of physical representation laziness, particularly noted in highly dynamic environments. For these lazy cases, the learned latent states do not collapse but nonetheless fail to represent key physical properties, causing ubiquitous downstream planning failure. To resolve this issue, we propose training-time auxiliary supervision with a lightweight "Fourier auxiliary head", which enforces physically-informed structuring of the latent space with no additional inference-time cost and can be generalized to any environment. Experimentally, we show that the auxiliary head substantially improves planning success rates in dynamic environments where the baseline LeWM exhibits physical representation laziness. It also leads to modest improvements in other environments, even when the baseline does not exhibit physical representation laziness. We further observe superior planning performance being accompanied by higher latent space correlations with key physical properties, indicating both the ability of our method to physically structure latent states and the potential planning-side benefit to the learned representation being physically structured. We also see in low-data regimes, auxiliary supervision is particularly impactful in increasing success rate. These findings support the use of our Fourier auxiliary head method to improve both overall success rate and data efficiency, while avoiding representation laziness in latent world models.
Comments9 pages, 4 figures; updated method based on new results
