发表机构
Rice University; Brown University（莱斯大学; 布朗大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出用雅可比质心监测JEPA世界模型的行为对齐，发现编码器与预测器的结构分离可预测规划失败，并优于基线作为分布偏移检测器。
AI 中文摘要
检测基于世界模型（WM）的规划中的失败，需要监测模型是否与当前任务行为对齐，这进而需要研究其内部表示。在此，我们表明质心——子组件雅可比行和——能有效识别WM的行为属性，补充了传统的基于激活的知识信号。模型的质心可通过雅可比向量积轻松计算，并刻画模型如何组织其输入空间的几何结构，从而提供对内部表示的高效视角，包括生成任务相关的显著性图。在连续控制任务上使用JEPA WM进行评估时，这种行为视角揭示了一种结构性分离，其中编码器正确表示目标，而预测器保持行为无响应。这种失败模式在任何动作执行之前直接预测规划失败，允许目标重采样以重新捕获分布外成功。此外，基于质心的方法作为分布偏移检测器优于基线方法。总之，这些工具构成一个在分布偏移下可操作且具有影响力的行为监测栈。
英文摘要
Detecting failures in World Model (WM)-based planning requires monitoring whether the model is behaviorally aligned with the current task, which in turn requires studying its internal representations. Here, we show that centroids---sub-component Jacobian row-sums---effectively identify the behavioral properties of WMs, complementing traditional activation-based knowledge signals. The centroids of a model are easily computed through Jacobian vector products and characterize how the model organizes the geometry of its input space, yielding an efficient perspective on internal representations, including the generation of task-relevant saliency maps. Evaluated on continuous control tasks using JEPA WMs, this behavioral view reveals a structural dissociation, where the encoder correctly represents the goal while the predictor remains behaviorally unresponsive. This failure mode directly predicts planning failure before any action is taken, allowing for goal resampling to recapture out-of-distribution success. Moreover, centroid-based methods outperform baseline methods as distribution-shift detectors. Together, these tools yield a behavioral monitoring stack that is operational and consequential under distribution shifts.
CommentsPresented at Workshop on World Models - Hosted by Chicago Booth
