可靠融合相互冲突的专家意见
Reliable Fusion of Conflicting Experts
- The University of Texas at Dallas（德克萨斯大学达拉斯分校）
- TU Darmstadt（达姆施塔特工业大学）
- The University of Texas at Tyler（德克萨斯大学泰勒分校）
- Air Force Research Lab（空军研究实验室）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
针对专家可靠性随输入变化且易冲突的场景，提出基于概率电路的动态融合框架，利用上下文可信度聚合黑盒专家意见，无需重训练，在多项选择任务中优于静态基线。
AI中文摘要：
我们研究了在噪声大、易发生冲突的环境中聚合多个黑盒专家意见的问题，在该环境中专家可靠性随输入而变化。静态聚合方法（如多数投票）无法捕捉这种变异性，并且在意见分歧时往往产生不可靠的结果。我们提出了一种基于概率电路的可处理融合框架，该框架利用上下文特定的可信度估计动态组合专家响应，从而实现有原则且可靠的推理。该框架与底层专家无关，不需要访问其内部表示或进行任何重新训练。我们在多项选择题回答任务上使用多个大语言模型作为专家进行了实证验证，并与单个模型和静态集成基线进行了比较。我们的方法在冲突情况下持续提高了预测性能并产生了更可靠的决策，凸显了上下文感知可信度建模在鲁棒多专家融合中的有效性。
英文摘要：
We study the problem of aggregating opinions from multiple black-box experts in noisy, conflict-prone settings where expert reliability varies across inputs. Static aggregation methods, such as majority voting, fail to capture this variability and often yield unreliable outcomes under disagreement. We propose a tractable, probabilistic-circuit-based fusion framework that dynamically combines expert responses using context-specific credibility estimates, enabling principled and reliable reasoning. The framework is agnostic to the underlying experts and does not require access to their internal representations or any retraining. We empirically validate our approach on multiple-choice question answering tasks using multiple LLMs as experts, comparing against individual models and static ensemble baselines. Our method consistently improves predictive performance and produces more reliable decisions under conflict, highlighting the effectiveness of context-aware credibility modeling for robust multi-expert fusion.
