arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2511.07784cs.MA
收藏

LLM智能体真的能辩论吗?逻辑推理中多智能体辩论的受控研究

Can LLM Agents Really Debate? A Controlled Study of Multi-Agent Debate in Logical Reasoning

McGill University(麦吉尔大学);Mila - Quebec AI Institute(米拉-魁北克人工智能研究所);University of South Florida(南佛罗里达大学)

Haolun Wu, Zhenkun Li, Lingyao Li

AI总结:

本研究以Knight--Knave--Spy逻辑谜题为场景,通过控制团队规模、构成等六类因素开展受控实验,揭示内在推理能力和群体多样性是多智能体辩论成功的主导因素,为设计可解释的多智能体推理系统提供指导。

摘要:

多智能体辩论(MAD)是近期提升大型语言模型(LLMs)推理性能的潜在框架,但LLM智能体能否真正参与审慎推理而非简单集成或多数投票仍不明确。本研究通过Knight--Knave--Spy逻辑谜题开展受控研究,该谜题可在可验证的真实场景下对辩论结果和过程进行精准、分步评估。我们系统设置了六个结构与认知因素,包括智能体团队规模、构成、置信度可见性、辩论顺序、辩论深度和任务难度,以区分它们对集体推理的各自影响。结果显示,内在推理能力和群体多样性是辩论成功的主导因素,而顺序或置信度可见性等结构参数增益有限。除结果外,过程层面分析识别出关键行为模式:多数压力抑制独立修正,有效团队推翻错误共识,理性、与有效性一致的推理最能预测改进。这些发现为LLM辩论成败的原因和方式提供了宝贵见解,为设计可解释、求真的多智能体推理系统提供指导。

英文摘要:

Multi-agent debate (MAD) has recently emerged as a promising framework for improving the reasoning performance of large language models (LLMs). Yet, whether LLM agents can genuinely engage in deliberative reasoning, beyond simple ensembling or majority voting, remains unclear. We address this question through a controlled study using the Knight--Knave--Spy logic puzzle, which enables precise, step-wise evaluation of debate outcomes and processes under verifiable ground truth. We systematically set up six structural and cognitive factors, including agent team size, composition, confidence visibility, debate order, debate depth, and task difficulty, to disentangle their respective effects on collective reasoning. Our results show that intrinsic reasoning strength and group diversity are the dominant drivers of debate success, while structural parameters such as order or confidence visibility offer limited gains. Beyond outcomes, process-level analyses identify key behavioral patterns: majority pressure suppresses independent correction, effective teams overturn incorrect consensus, and rational, validity-aligned reasoning most strongly predicts improvement. These findings provide valuable insights into how and why LLM debates succeed or fail, offering guidance for designing interpretable and truth-seeking multi-agent reasoning systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑