arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.28360cs.AI
收藏

CoE:协作熵用于代理多语言模型系统中的不确定性量化

CoE: Collaborative Entropy for Uncertainty Quantification in Agentic Multi-LLM Systems

Kangkang Sun, Jun Wu, Jianhua Li, Minyi Guo, Xiuzhen Che, Jianwei Huang

AI总结:

本文提出CoE,一种用于多语言模型协作中语义不确定性的信息理论度量,通过结合模型内语义熵和模型间与集均值的差异,提供更准确的不确定性估计,实验表明其在TriviaQA和SQuAD上优于传统基线。

摘要:

多语言模型系统中的不确定性估计仍主要局限于单模型:现有方法在每个模型内量化不确定性,但未能充分捕捉模型间的语义分歧。为解决这一差距,我们提出协作熵(CoE),一种统一的信息理论度量,用于多语言模型协作中的语义不确定性。CoE定义在共享的语义聚类空间上,并结合两个组成部分:模型内的语义熵和模型间与集均值的差异。CoE不是加权集预测器,而是一种系统级不确定性度量,用于表征协作的自信和分歧。我们分析了CoE的几个核心属性,包括非负性、在完美语义共识下的零值确定性,以及当个体模型坍缩为delta分布时CoE的行为。这些结果澄清了何时减少单个模型不确定性就足够,以及何时残留的模型间分歧仍存在。我们还提出了一种简单的CoE引导、无训练的后处理协调启发式方法作为该度量的实际应用。在TriviaQA和SQuAD上使用LLaMA-3.1-8B-Instruct、Qwen-2.5-7B-Instruct和Mistral-7B-Instruct的实验表明,CoE比标准熵和差异基线提供了更强的不确定性估计,随着引入更多异质模型,增益变得更大。总体而言,CoE为多语言模型协作提供了有用的不确定性意识视角。

英文摘要:

Uncertainty estimation in multi-LLM systems remains largely single-model-centric: existing methods quantify uncertainty within each model but do not adequately capture semantic disagreement across models. To address this gap, we propose Collaborative Entropy (CoE), a unified information-theoretic metric for semantic uncertainty in multi-LLM collaboration. CoE is defined on a shared semantic cluster space and combines two components: intra-model semantic entropy and inter-model divergence to the ensemble mean. CoE is not a weighted ensemble predictor; it is a system-level uncertainty measure that characterizes collaborative confidence and disagreement. We analyze several core properties of CoE, including non-negativity, zero-value certainty under perfect semantic consensus, and the behavior of CoE when individual models collapse to delta distributions. These results clarify when reducing per-model uncertainty is sufficient and when residual inter-model disagreement remains. We also present a simple CoE-guided, training-free post-hoc coordination heuristic as a practical application of the metric. Experiments on \textit{TriviaQA} and \textit{SQuAD} with LLaMA-3.1-8B-Instruct, Qwen-2.5-7B-Instruct, and Mistral-7B-Instruct show that CoE provides stronger uncertainty estimation than standard entropy- and divergence-based baselines, with gains becoming larger as additional heterogeneous models are introduced. Overall, CoE offers a useful uncertainty-aware perspective on multi-LLM collaboration.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑