知识过滤与集体幻觉:一个适用于置信度校准代理的陪审团定理
Epistemic Filtering and Collective Hallucination: A Jury Theorem for Confidence-Calibrated Agents
AI总结:
研究异质代理在时间中学习自我可靠性估计并选择性 abstain 的集体准确性,提出基于置信度校准的框架,推导非渐近下界并证明其在置信度门控设置下的泛化能力,通过蒙特卡洛模拟验证,探讨其在AI安全中的应用。
摘要:
我们研究了异质代理在时间中学习自我可靠性估计并选择性 abstain 的集体准确性。经典知识投票结果,如陪审团定理(CJT),假设固定参与,而现实聚合往往受益于允许代理说『我不知道』。我们提出一个概率框架,其中代理经历一个校准阶段,更新对自身固定能力的信念,然后面对一个最终置信度门,决定投票或 abstain。我们推导了群体成功概率的非渐近下界,并证明这种选择性参与将CJT的渐近保证推广到顺序、置信度门控设置。通过蒙特卡洛模拟验证这些界限。尽管结果一般,我们讨论其在AI安全中的潜在应用,概述了该框架如何缓解集体LLM决策中的幻觉。
英文摘要:
We investigate the collective accuracy of heterogeneous agents who learn to estimate their own reliability over time and selectively abstain from voting. While classical epistemic voting results, such as the \textit{Condorcet Jury Theorem} (CJT), assume fixed participation, real-world aggregation often benefits from allowing agents to say ``I don't know.'' We propose a probabilistic framework where agents engage in a \textit{calibration} phase, updating beliefs about their own fixed competence, before facing a final confidence gate that determines whether to vote or abstain. We derive a non-asymptotic lower bound on the group's success probability and prove that this \textit{selective participation} generalizes the asymptotic guarantees of the CJT to a sequential, confidence-gated setting. Empirically, we validate these bounds via Monte Carlo simulations. While our results are general, we discuss their potential application to AI safety, outlining how this framework can mitigate \textit{hallucinations} in collective LLM decision-making.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。