发表机构
L3S Research Centre; Leibniz University Hannover; TIB Leibniz Information Centre for Science and Technology（L3S研究中心; 汉诺威莱布尼茨大学; TIB莱布尼茨科技信息中心）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文针对Graph-JEPA出现的类别条件崩溃问题，通过实验诊断出退化解的问题并进行修复，发布了带可约性审计和目标门的工具包，揭示了联合嵌入预测架构的潜在缺陷。
AI 中文摘要
联合嵌入预测架构几乎被线性探测和有效秩普遍选用。我们报告一个案例：这两个指标都表现健康，但表示却不携带任何可用的实例信息。我们修复了该问题，却出现了第二个失败：修复后的指标在不携带结构信息的目标上达到饱和。我们的语料库是一个包含57903篇文章的科学推理图，每篇文章都是一个子图。Graph-JEPA从子图的其余部分预测一个被掩码的方面，线性探测准确率达到0.871，有效秩为18-47，但检索仅恢复了14.4比特中的0.00比特（MRR为1.9e-4，而随机水平为1.99e-4，p=0.98）。同一池中的三个上界和代码几乎恢复了所有内容（+14.28、+14.34、+14.22比特），排除了语料库、掩码、池和指标作为原因。我们将其追溯到方差分配：冻结输入将86.05%的方差放在子图身份上，0.40%放在方面身份上，而训练后的潜变量则为0.39%和99.61%。这是目标最优解的属性：退化解是耦合预测器/EMA目标目标的全局最小值，初始化时已存在。修复后的配置达到14.379比特中的14.377比特，高于13.865比特的 oracle；将损失还原为回归则降至0.307比特，证实了这一点。然而，修复并未带来任何推理能力：目标是可约的，因为子图内边是节点普查的确定性函数。oracle达到上限的96.4%，我们最大的影响是学习率调度，而非架构。在十个单元中，比特数和推理探测器之间无关联。数据衍生的目标未通过质量门：25.96%的节点是重复占位符，其余部分比支持证据更通用。秩、探测器和指标都可能在无支持的评估中达到饱和。我们发布了一个带有可约性审计和目标门的工具包。
英文摘要
Joint-embedding predictive architectures are selected almost universally by linear probing and effective rank. We report a case where both read healthily while the representation carries zero usable instance information. We repair it, and a second failure appears: the repaired metric saturates on a target carrying no structural information. Our corpus is a scientific-reasoning graph over 57,903 articles, each a subgraph. A Graph-JEPA predicts one masked aspect from a subgraph's remaining aspects, attaining linear-probe accuracy 0.871 and effective rank 18-47, yet retrieval recovers 0.00 of 14.4 bits (MRR 1.9e-4 vs chance 1.99e-4, p=0.98). Three upper bounds on the same pool and code recover nearly everything (+14.28, +14.34, +14.22 bits), ruling out corpus, masking, pool, and metric as causes. We trace this to variance allocation - frozen inputs place 86.05% of variance on subgraph identity and 0.40% on aspect identity, while trained latents place 0.39% and 99.61%. This is a property of the objective's optimum: the degenerate solution is a global minimum of the coupled predictor/EMA-target objective, present already at init. A repaired configuration reaches 14.377 of 14.379 bits, above the 13.865-bit oracle; reverting the loss to regression drops it to 0.307 bits, confirming it. Yet the repair licenses nothing about reasoning: the target is reducible, since intra-subgraph edges are a deterministic function of node census. The oracle reaches 96.4% of the ceiling, and our largest effect is the learning-rate schedule, not architecture. Bits and a reasoning probe show no relation across ten cells. A data-derived target fails a quality gate - 25.96% of nodes are duplicate placeholders, and the rest is more generic than supporting evidence. Rank, probes, and metrics can all saturate on an unsupportive evaluation. We release a harness with a reducibility audit and target gate.
