并行多智能体推理系统的序贯概率不确定性估计

中文摘要 AI 辅助

基于大语言模型的多智能体系统（MAS）因通过多智能体间的交互提升推理能力而受到越来越多的关注。在本工作中，我们聚焦于并行多智能体推理系统，其中多个智能体在多个轮次中解决同一问题，并将其输出聚合为最终答案。尽管这类系统具有强大的推理性能，但其不确定性估计仍未被充分探索：MAS的可靠性不仅取决于单个智能体的生成结果，还取决于智能体在各轮次间如何交互与演化。我们提出SAUCE（通过共识演化的序贯智能体不确定性），一种轻量级、无需训练的不确定性估计器，将MAS的不确定性建模为对潜在系统级信念的序贯推断。SAUCE通过滤波式更新聚合轮次级一致性与生成不确定性信号。在五个主干模型、五个基准数据集和两种MAS协议上，SAUCE在错误检测、选择性预测和校准方面优于广泛的不确定性估计基线，包括基于标准对数似然的方法和MAS专用估计器。

英文摘要

LLM-based multi-agent systems (MAS) have attracted growing attention for improving reasoning through interaction among multiple agents. In this work, we focus on parallel multi-agent reasoning systems, where several agents solve the same problem over multiple rounds and aggregate their outputs into a final answer. Despite their strong reasoning performance, uncertainty estimation for such systems remains underexplored: the reliability of a MAS depends not only on individual generations, but also on how agents interact and evolve across rounds. We propose SAUCE (Sequential Agent Uncertainty through Consensus Evolution), a lightweight, training-free uncertainty estimator that formulates MAS uncertainty as sequential inference over a latent system-level belief. SAUCE aggregates round-level agreement and generation-uncertainty signals through a filtering-style update. Across five backbones, five benchmarks, and two MAS protocols, SAUCE improves misclassification detection, selective prediction, and calibration over a broad set of uncertainty estimation baselines, including standard log-likelihood-based methods and MAS-specific estimators.

发表机构

• Rutgers University（罗格斯大学）
• University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
• Vanderbilt University（范德堡大学）
• Columbia University（哥伦比亚大学）

机构由 AI 辅助整理，请以论文原文为准。