子空间分解的JEPA:解耦潜在世界模型中的进展与内容
Subspace-Decomposed JEPAs: Disentangling Progression and Content in Latent World Models
AI总结:
提出SD-JEPA方法,通过将JEPA潜在空间分解为正交的进展子空间和内容子空间,利用余弦边际三元组损失和SIGReg正则化分别约束,在控制基准上优于LeWM基线,并证明进展坐标可作为场景感知的指南针。
摘要:
联合嵌入预测架构(JEPA)通过预测未来嵌入来学习紧凑的潜在世界模型,但潜在空间的任何单一坐标都未被指定用于编码任务进展。我们将JEPA潜在空间分解为两个具有不相交角色的正交子空间:一个由余弦边际三元组损失塑造的低维进展子空间,以及一个由LeWM现有SIGReg目标正则化的高维内容子空间。我们证明两个抗坍塌力作用于不相交的坐标,因此它们加性组合而非在同一维度上竞争。我们的方法SD-JEPA在大多数控制基准上以匹配的计算量优于LeWM基线,并在Push-T上优于最强的非LeWM JEPA基线;子空间消融验证了分解是关键因素。除了规划之外,得到的一维角进展坐标在潜在空间中充当场景感知的指南针。它随任务进展而前进,当智能体回溯时后退,在受控扰动下既会尖峰也会重新定位到语义上合适的新任务阶段区域,以预测误差标量无法做到的方式将惊讶时刻与其意义分离。三个定量测试支持这一点:在40个保留的立方体情节中,|Δθ_t|在定位语义事件方面优于标准潜在预测误差惊讶度,最高可达+0.18的合并AUROC(在±1步容差下每情节胜率97.5%);在所有四个环境(每个环境40个情节)的情节内线性探针显示,8维进展子空间(潜在空间的4.2%)解释了72-95%的任务进展方差。
英文摘要:
World models used for planning learn to predict how a scene evolves, yet they rarely hold a global view of how far a task has progressed. We argue that such a view should be part of the representation, and that it can be compact: describing a scene takes many dimensions, but where the scene stands on its way to the goal can be summarised by only a few. We propose SD-JEPA, a joint-embedding predictive world model whose latent is split into a small progression subspace and a large content subspace. A temporal triplet loss keeps consecutive frames of a trajectory close in the progression coordinates and separates different trajectories, while a distributional regulariser keeps the content coordinates from collapsing. On four control benchmarks, the model plans as well as the same architecture without the split. In return, its latent becomes easier to read. A few percent of its dimensions are enough to predict progress within an episode, and the angle they form follows the task, moves backward when the agent backtracks and changes fastest around events such as grasps and releases. Surprise also becomes anchored in the trajectory: when the scene changes mid-episode, the angle spikes and then relocalizes on a new part of the progression, anchoring surprise in the agent's own map of the task.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。