DICE: 用于稳定多智能体LLM协调的熵正则化均衡选择
DICE: Entropy-Regularized Equilibrium Selection for Stable Multi-Agent LLM Coordination
AI总结：
提出DICE框架，通过熵正则化均衡选择（HQRE）解决多智能体LLM协调中的不稳定性，实现线性收敛和有限贝叶斯遗憾，在11个基准上平均提升4.3-8.5个百分点。
摘要：
多智能体大语言模型（LLM）系统通常无法可靠地超越配备最佳N采样的单个强模型。我们认为这种不稳定性的一个核心来源是病态的均衡选择：当前系统指定了智能体共享哪些信息，但没有指定应选择哪种协调约定。我们将此类系统的一类广泛形式化为折扣不完全信息马尔可夫博弈，并表明两种常见病理——竞争约定之间的振荡和跨约定漂移——均可导致不稳定的学习和线性贝叶斯遗憾。为了获得一个良定义的目标，我们引入了异质量化响应均衡（HQRE），这是一种具有智能体和状态依赖温度的熵正则化均衡概念。在单调性条件下，HQRE是唯一的，允许线性收敛的镜像更新，并产生有界的贝叶斯遗憾；相同的条件产生可 rollout 测量的稳定性诊断。我们在两种算法中实例化这一目标：DICE-PC，通过提示控制动作协调冻结模型，以及DICE-FT，执行参数高效的镜像微调。在四个领域的十一个基准测试中，DICE在准确性-成本权衡上优于强类内基线；在推理和规划任务上，DICE-PC平均提高4.3个百分点，DICE-FT提高8.5个百分点。
英文摘要：
Multi-agent large language model (LLM) systems often fail to reliably outperform a single strong model equipped with best-of-N sampling. We argue that a core source of this instability is ill-posed equilibrium selection: current systems specify what information agents share, but not which coordination convention should be selected. We formalize a broad class of such systems as discounted incomplete-information Markov games and show that two common pathologies, oscillation between competing conventions and drift across them, can both induce unstable learning and linear Bayesian regret. To obtain a well-posed target, we introduce the Heterogeneous Quantal Response Equilibrium (HQRE), an entropy-regularized equilibrium concept with agent- and state-dependent temperatures. Under a monotonicity condition, HQRE is unique, admits linearly convergent mirror updates, and yields bounded Bayesian regret; the same condition yields rollout-measurable stability diagnostics. We instantiate this objective in two algorithms: DICE-PC, which coordinates frozen models through prompt-control actions, and DICE-FT, which performs parameter-efficient mirror fine-tuning. Across eleven benchmarks in four domains, DICE improves accuracy-cost trade-offs over strong within-class baselines; on reasoning and planning tasks, DICE-PC improves by 4.3 percentage points on average and DICE-FT by 8.5 points.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
