arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2607.14144cs.AIcs.ITmath.IT

能力源于访问结构,而非规模:混合序列模型的下界与预注册测试

The Capability Convergence Hypothesis: Capability from Access Structure, Not Scale

  • University of Macau(澳门大学)
  • South China University of Technology(华南理工大学)

机构由 AI 辅助整理,请以论文原文为准。

Wenhui Chen, Jianlin Chen, Ziyao Lin, Chi Man Vong

AI总结:

研究提出能力收敛假设,指出能力收敛取决于访问结构而非规模。基于牛顿苹果问题等,明确三个资源壁垒,通过信息论下界和预注册实验区分证明与推测,报告小规模测试结果,验证了相关假设及结论。

AI中文摘要:

柏拉图式表征假设(PRH)认为随着模型规模扩大,异构网络的表征会趋向于一个共享的现实模型。我们提出了它的后续及边界,即能力收敛假设(CCH):在固定的每令牌推理预算下,表征收敛并不意味着能力收敛。能力反而趋向于一类,即访问完备混合:任何同时拥有压缩的 O(1) 状态通道和可扩展逐字索引通道的架构。我们将其锚定在一个见证任务上,即无限流中的牛顿苹果问题,并指出三个资源壁垒:香农壁垒禁止任何 o(Nb) 状态架构,视界壁垒禁止任何固定窗口,以及电路壁垒禁止固定深度仅注意力组合(条件是 TC0 != NC1)。在明确的可分性假设下,混合通过支付每个壁垒的代价跨越所有三个壁垒,因此能力在组合下严格超加性。我们区分了已证明的和推测的内容:访问完备性原则基于信息论下界和预注册实验,而领域级收敛趋势是一个受经济学驱动的推测。我们报告了在数据之前冻结标准下的首次预注册小规模测试:测量了预测的剪刀差(一旦 64 标量状态获得一个全局注意力层,精确检索误差为 0.994 对 0.000),状态跟踪分岔落在注册边界,并且一个联合见证显示了一个不可约的双通道解决方案;一个预测方向相反失败并如实报告。表征收敛由规模免费提供;能力收敛必须通过访问结构来购买。

英文摘要:

The Platonic Representation Hypothesis (PRH) holds that as models scale, representations of heterogeneous networks converge toward a shared model of reality. We propose its sequel and boundary, the Capability Convergence Hypothesis (CCH): under a fixed per-token inference budget, representational convergence does not entail capability convergence. Capability instead converges toward a class, the access-complete hybrid: any architecture holding both a compressive O(1)-state channel and a scalable verbatim-index channel. We anchor it on a witness task, the Newton's-apple problem in an infinite stream, and name three resource walls: a Shannon wall barring any o(Nb)-state architecture, a horizon wall barring any fixed window, and a circuit wall barring fixed-depth attention-only composition (conditional on TC0 != NC1). Under an explicit separability assumption a hybrid crosses all three by paying each wall's price, so capability is strictly super-additive under composition. We separate what we prove from what we conjecture: the access-completeness principle rests on information-theoretic lower bounds and pre-registered experiments, while the field-level convergence trend is an economics-motivated conjecture. We report the first pre-registered small-scale tests under criteria frozen before the data: the predicted scissors gap is measured (exact-retrieval error 0.994 vs. 0.000 once a 64-scalar state gains one global-attention layer), the state-tracking bifurcation lands at the registered boundary, and a conjunction witness shows an irreducibly two-channel solution; one prediction failed with its direction reversed and is reported as such. Representational convergence is given freely by scale; capability convergence must be purchased by access structure.

补充信息

↑