arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2606.12979cs.LG
收藏

EPM-JEPA:JEPA系列世界模型中的算子侧经验调制

EPM-JEPA: Operator-Side Experience Modulation in JEPA-Family World Models

School of Artificial Intelligence and Data Engineering (SAIDE), Indian Institute of Technology Jodhpur(印度理工学院焦特布尔分校人工智能与数据工程学院)

Vedant Pandya

AI总结:

提出EPM-JEPA,通过LoRA在权重层面调制预测器,以应对测试时动态偏移;实验表明其优于无记忆基线,但效果弱于预期,并揭示了三种独立动力学过程。

摘要:

JEPA系列世界模型使用静态预测器,其权重在测试时动态偏离训练时不会自适应。我们比较了在分布偏移下将累积经验融入JEPA预测器的两种机制:操作数侧注入(EI-JEPA),将压缩的经验表示作为残差添加到预测器的隐藏状态;以及算子侧调制(EPM-JEPA),通过应用于预测器权重的LoRA生成低秩权重增量。在预注册的比较(Moving MNIST,重力偏移)中,EPM-JEPA(D_shift^{n=50} = 0.7848 +/- 0.0078,三个种子)与EI-JEPA(0.8238)相差delta = 4.74% - 根据我们声明的标准,结果C:零结果 - 是一个有效结果。作为次要的、非预注册的观察,EPM-JEPA在无记忆基线(0.8000)上提高了1.90%,且在所有种子上一致,而EI-JEPA低于基线,表明收益特定于权重级调制。我们的主要贡献是机制分析:D_shift^{n=50}轨迹反映了三个独立的动力学过程——缓冲区循环、EMA目标漂移和内在的LoRA稳定瞬态(+0.021)——而非收敛到平衡。这些发现推动了PEM-JEPA,一个基于物理的后续模型,以解决这一动力学峰值限制。

英文摘要:

JEPA-family world models use a static predictor whose weights do not adapt when test-time dynamics diverge from training. We compare two mechanisms for incorporating accumulated experience into a JEPA predictor under distribution shift: operand-side injection, where a compressed experience representation is added as a residual to the predictor's hidden state (EI-JEPA), and operator-side modulation, where the same representation generates low-rank weight deltas via LoRA applied to the predictor's weights (EPM-JEPA). On a pre-registered comparison (Moving MNIST, gravity shift), EPM-JEPA (D_shift^{n=50} = 0.7848 +/- 0.0078, three seeds) differs from EI-JEPA (0.8238) by delta = 4.74% - Outcome C: a null result - by our stated criterion, a valid outcome. As a secondary, non-pre-registered observation, EPM-JEPA improves 1.90% over a no-memory baseline (0.8000), consistently across seeds, while EI-JEPA underperforms the baseline, indicating the benefit is specific to weight-level modulation. Our primary contribution is a mechanism analysis: the D_shift^{n=50} trajectory reflects three independent dynamical processes - buffer cycling, EMA target drift, and an intrinsic LoRA settling transient of +0.021 - rather than convergence to equilibrium. These findings motivate PEM-JEPA, a physics-grounded successor addressing this dynamical-peak limitation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑