上下文确定性有限证书与语言模型中涌现的阈值理论
Finite Certificates for In-Context Determinacy and a Threshold Theory of Emergence in Language Models
AI总结:
提出用有限语义证书验证上下文条件语言模型行为,证明有限域线性任务族中确定性准则,并证明阈值涌现的反幻象定理,将阈值度量与语义置信度分离。
摘要:
本文开发了一个模型论框架,通过用有限语义证书替代基准标签来验证上下文条件语言模型行为。第一个问题是有限确定性:上下文中的示例何时在不改变模型参数的情况下强制查询答案?在有限域线性任务族中,我们证明了精确的行空间准则,计算了残差假设数量,推导了完整和查询局部识别曲线,并表明即使对于二元输出,提取最小强制子上下文也是NP完全的。第二个问题是阈值涌现:何时明显的基准跳跃反映语义转换而非评分映射的不连续性?我们证明了一个反幻象定理,将阈值度量与语义置信度分离,并给出了潜在承诺在阈值以上变得可见的速率敏感交叉界。共同的语义对象是可定义事件上的置信度泛函。我们证明它是一个布尔概率测度,等价于相关类型空间上的Keisler测度,其测度一公式构成一个真滤子,且其Stone空间表示在定义扩展下不变。由此产生的演算提供了有限上下文证书、对分隔符击中集、查询教学维度、提示保留准则和尺度极限见证。精确算术辅助脚本重现了有限域和阈值计算,并生成了图表使用的数据。
英文摘要:
This paper develops a model-theoretic framework for verifying context-conditioned language-model behavior by replacing benchmark labels with finite semantic certificates. The first problem is finite determinacy: when do examples in a context force the answer to a query without changing model parameters? In finite-field linear task families, we prove an exact row-space criterion, compute the residual hypothesis count, derive full and query-local identification curves, and show that extracting a smallest forcing subcontext is NP-complete even for binary outputs. The second problem is threshold emergence: when does an apparent benchmark jump reflect a semantic transition rather than a discontinuity of the scoring map? We prove an anti-mirage theorem separating thresholded metrics from semantic confidence and give a rate-sensitive crossing bound for latent commitments becoming visible above threshold. The common semantic object is a confidence functional on definable events. We show that it is a Boolean probability measure, equivalently a Keisler measure on the relevant type space, whose measure-one formulas form a proper filter and whose Stone-space representation is invariant under definitional expansion. The resulting calculus provides finite context certificates, pair-separator hitting sets, query teaching dimension, prompt-preservation criteria, and scale-limit witnesses. Exact-arithmetic ancillary scripts reproduce the finite-field and threshold calculations and generate the data used by the figures.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。