发表机构
University of Aberdeen（阿伯丁大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究揭示概率联合嵌入预测架构（JEPA）等价于隐马尔可夫模型（HMM），提出马尔可夫链 JEPA（MCJEPA）并通过实验验证其对应关系，为时序 JEPA 提供了状态空间解释。
AI 中文摘要
隐马尔可夫模型（HMM）包含三个核心角色：从观测中推断隐状态置信度、通过马尔可夫转移进行传播、再发射回观测空间。本文表明，带时间索引的完整预测信息瓶颈变分联合嵌入预测架构（PIB-VJEPA）展现出相同的计算结构：随机上下文编码器充当 amortized 滤波分布，概率预测器定义隐状态动力学，解码器、逆目标编码器或诱导隐式条件提供发射方向。我们区分了4种逐步增强的对应级别，并给出了序列级 HMM 完全等价的充分条件。为使该关联更具体，我们引入马尔可夫链联合嵌入预测架构（MCJEPA），其用学习到的转移矩阵替代隐预测器；在有限时间齐次情况下，矩阵幂保证精确的多步 Chapman-Kolmogorov 一致性。离散状态条件转移、连续状态马尔可夫核及连续时间动力学可扩展该构造，而确定性时序 JEPA 则作为退化的 Dirac 核特例存在。我们进一步将预测信息瓶颈学习解释为寻求紧凑预测状态：压缩促进极小性，残余可预测性测试充分性。受控实验支持转移合成、滤波解释、已知合成过程中的预测马尔可夫化，以及 JEPA 隐预测与 HMM 式序列学习的区别。这些结果为时序 JEPA 提供了有原则的状态空间解释。
英文摘要
A hidden Markov model (HMM) combines three roles: inference of a hidden-state belief from observations, propagation through a Markov transition, and emission back to observation space. We show that full, time-indexed Predictive Information Bottleneck VJEPA (PIB-VJEPA) exposes the same computational structure: a stochastic context encoder plays the role of an amortized filtering distribution, a probabilistic predictor defines latent-state dynamics, and a decoder, inverse target encoder, or induced implicit conditional supplies the emission direction. We distinguish 4 progressively stronger levels of correspondence and give sufficient conditions for exact sequence-level HMM equivalence. To make the connection concrete, we introduce Markov-Chain JEPA (MCJEPA), which replaces the latent predictor by a learned transition matrix; in the finite time-homogeneous case, matrix powers guarantee exact multi-horizon Chapman--Kolmogorov consistency. Conditioned discrete-state transitions, continuous-state Markov kernels, and continuous-time dynamics extend this construction, while deterministic temporal JEPA appears as a degenerate Dirac-kernel special case. We further interpret predictive information-bottleneck learning as seeking a compact predictive state: compression promotes minimality, while residual predictability tests sufficiency. Controlled experiments support transition composition, the filtering interpretation, predictive Markovization in a known synthetic process, and the distinction between JEPA latent prediction and HMM-style sequence learning. Together, these results give temporal JEPA a principled state-space interpretation.
Comments69 pages
