反事实动作评估、观测瓶颈与联合嵌入预测世界模型中的表示几何
Counterfactual Action Evaluation, Observation Bottlenecks, and Representation Geometry in Joint-Embedding Predictive World Models
- Computer Science and Artificial Intelligence Laboratory (CSAIL)（计算机科学与人工智能实验室（CSAIL））
- Massachusetts Institute of Technology（麻省理工学院）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本文提出一种贯穿模拟器状态、观测、嵌入和预测器的反事实评估协议，发现预测误差低不代表动作区分充分，并揭示观测瓶颈、动作路径利用不足及表示几何问题，强调需分别审计物理效应、可见性、几何和动作依赖性。
AI中文摘要：
低潜在预测误差并不能证明世界模型能够区分其动作的后果。我们引入了一种评估协议，该协议将相同的干预措施贯穿模拟器状态、栅格观测、目标嵌入和预测器输出。在受控的可变形物理测试平台上，精确的模拟器状态分叉揭示了不同的瓶颈。改变命令会改变粒子运动，然而41.5%的单步栅格对是相同的。观测损失并非全部原因：在579个高可见性反事实中，两个种子下的预测器对目标响应的中位数分别为0.0051和0.0217，经方差归一化后降至0.0027和0.0116。与目标反事实嵌入位移匹配的各向同性状态扰动，在相同的可见对上产生190倍和53倍更大的预测器变化，从而将动作路径利用不足与预测器死亡或全局收缩区分开来。仅使用MSE训练在匹配种子下使10步潜在误差降低8.36倍，但这是在谱集中空间中；一个VICReg目标编码器也高度集中，因此误差或秩单独都不能证明物理状态。最后，即使从全分辨率栅格和机械状态出发，刚度仍接近随机水平，而特权材料参数可完美解码，这表明在该激励下可辨识性较弱，而非编码器丢弃。这些结果促使分别审计物理效应、观测可见性、表示几何和动作依赖性。
英文摘要：
Low latent prediction error does not establish that a world model distinguishes the consequences of its actions. We introduce an evaluation protocol that traces the same intervention through simulator state, raster observations, target embeddings, and predictor outputs. Exact simulator-state forks in a controlled deformable-physics testbed reveal distinct bottlenecks. Changed commands alter particle motion, yet 41.5% of one-step raster pairs are identical. Observation loss is not the whole explanation: among 579 high-visibility counterfactuals, median predictor-to-target response is 0.0051 and 0.0217 across two seeds, falling to 0.0027 and 0.0116 after variance normalization. An isotropic state perturbation matched to the target counterfactual embedding shift produces 190x and 53x larger predictor changes on the same visible pairs, isolating action-path under-use rather than a dead or globally shrunk predictor. MSE-only training gives 8.36x lower 10-step latent error in matched seeds, but in spectrally concentrated spaces; one VICReg target encoder is also strongly concentrated, so neither error nor rank alone certifies physical state. Finally, stiffness remains near chance even from full-resolution rasters and mechanical state while privileged material parameters decode perfectly, indicating weak identifiability under this excitation rather than encoder discard. These results motivate auditing physical effect, observation visibility, representation geometry, and action dependence separately.
