当多智能体LLM辩论中的说服力凌驾于真相之上：引入置信度加权说服覆盖率（CW-POR）
When Persuasion Overrides Truth in Multi-Agent LLM Debates: Introducing a Confidence-Weighted Persuasion Override Rate (CW-POR)
AI总结：
本文研究多智能体LLM辩论中说服力凌驾真相的风险，提出置信度加权说服覆盖率（CW-POR）指标，通过TruthfulQA实验发现较小模型也能以高置信度说服评判者接受虚假信息，凸显了稳健校准与对抗性测试的必要性。
摘要：
在许多现实场景中，单个大型语言模型（LLM）可能会遇到相互矛盾的主张——一些准确，另一些则强硬地不正确——并且必须判断哪个是真实的。我们在单轮多智能体辩论框架中研究了这一风险：一个基于LLM的智能体提供来自TruthfulQA的事实性答案，另一个智能体极力为虚假内容辩护，而相同的LLM架构充当评判者。我们引入了置信度加权说服覆盖率（CW-POR），它不仅捕捉评判者被欺骗的频率，还捕捉其对错误选择相信的强烈程度。我们在五个开源LLM（3B-14B参数）上进行的实验中，系统地改变智能体冗长度（30-300词），结果表明即使是较小的模型也能构建有说服力的论据来覆盖真实答案——且往往具有高置信度。这些发现强调了稳健校准和对抗性测试的重要性，以防止LLM自信地支持错误信息。
英文摘要：
In many real-world scenarios, a single Large Language Model (LLM) may encounter contradictory claims-some accurate, others forcefully incorrect-and must judge which is true. We investigate this risk in a single-turn, multi-agent debate framework: one LLM-based agent provides a factual answer from TruthfulQA, another vigorously defends a falsehood, and the same LLM architecture serves as judge. We introduce the Confidence-Weighted Persuasion Override Rate (CW-POR), which captures not only how often the judge is deceived but also how strongly it believes the incorrect choice. Our experiments on five open-source LLMs (3B-14B parameters), where we systematically vary agent verbosity (30-300 words), reveal that even smaller models can craft persuasive arguments that override truthful answers-often with high confidence. These findings underscore the importance of robust calibration and adversarial testing to prevent LLMs from confidently endorsing misinformation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
