arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.09127cs.LGcs.ITmath.IT
收藏

认知吞吐量:受注意力限制的推断基本限制

Epistemic Throughput: Fundamental Limits of Attention-Constrained Inference

Technical University of Denmark(丹麦技术大学)

Lei You

AI总结:

本文提出认知吞吐量概念,研究受注意力限制的推断基本限制,揭示了信息杠杆与筛选质量的关系。

摘要:

最近的生成和工具使用的人工智能系统能够以较低的边际成本产生大量候选者,但只有少量可以被仔细检查。这导致了解码器端的瓶颈:下游决策者必须在众多公共记录中形成可靠的后验。我们通过注意力受限推断(ACI)正式化这一情形,其中一个便宜的筛选阶段处理K条记录,一个昂贵的验证阶段只能跟进其中至多B条。在贝叶斯对数损失下,我们研究了每窗口可实现的最大后验不确定性减少量,我们称之为认知吞吐量。我们的主要结果是一个“JaKoB”缩放定律,表明认知吞吐量有一个随验证和流行度线性增长的基线项,还有一个额外的信息杠杆项,其规模为√(JKB),其中J总结筛选质量。因此,扩展便宜筛选可以非线性放大稀缺验证,即使信息性记录稀少。我们进一步表明,这种缩放在弱筛选极限下是紧致的,并且在稀疏验证情形(B<<K)下,显著的杠杆作用要求重尾分数分布;对于轻尾分数分布,放大作用仅是对数级别的。

英文摘要:

Recent generative and tool-using AI systems can surface a large volume of candidates at low marginal cost, yet only a small fraction can be checked carefully. This creates a decoder-side bottleneck: downstream decision-makers must form reliable posteriors from many public records under scarce attention. We formalize this regime via Attention-Constrained Inference (ACI), in which a cheap screening stage processes $K$ records and an expensive verification stage can follow up on at most $B$ of them. Under Bayes log-loss, we study the maximum achievable reduction in posterior uncertainty per window, which we call \emph{epistemic throughput}. Our main result is a ``JaKoB'' scaling law showing that epistemic throughput has a baseline term that grows linearly with verification and prevalence, and an additional \emph{information-leverage} term that scales as $\sqrt{JKB}$, where $J$ summarizes screening quality. Thus, expanding cheap screening can nonlinearly amplify scarce verification, even when informative records are rare. We further show that this scaling is tight in a weak-screening limit, and that in the sparse-verification regime ($B \ll K$), substantial leverage requires heavy-tailed score distributions; for light-tailed scores the amplification is only logarithmic.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑