事后辩论评判理论
A Theory of Post-hoc Debate Judgement
浏览论文内容
中文总结 AI 辅助
本文提出适用于智能体辩论场景的事后辩论评判理论，探究两种评判方法的属性满足情况，发现辩论语义是辩论驱动型AI原则性评判者的理想候选。
中文摘要 AI 辅助
辩论近年来已成为智能体AI提升性能、增强可解释性和用户参与度的有效方法，例如由大语言模型（LLM）驱动的智能体可进行内部（与自身）或外部（与其他智能体）辩论。在许多使用辩论的场景中，辩论结果及最终输出由事后的外部评判者（通常是LLM）决定。本文开发并测试了一种适用于所有智能体通过为自身观点提供正反方论据参与辩论场景的新型辩论评判理论。具体而言，我们确定了辩论评判通常需满足的多项形式属性，涉及可复现性、鲁棒性、基于事实性和可解释性。随后，我们针对主张验证场景，从形式上和/或实验上探究了两种特定替代辩论评判方法对这些属性的满足情况：一是作为评判者的LLM的变体，二是源自计算辩论的形式语义。研究表明，这两种方法的准确率表现相似，但前者可能不具备后者所能提供的形式保证。总体而言，本研究指出辩论语义是辩论驱动型AI中原则性评判者的理想候选。
英文摘要
Debates have recently emerged as a useful methodology for agentic AI to improve performance as well as to aid explainability and user engagement. For example, LLM-empowered agents may debate internally (with themselves) and/or externally (with other agents). In many settings where debates are used, debates' outcomes and resulting outputs are determined post-hoc by external judges, often LLMs. In this paper we develop and test a novel theory of debate judgement applicable to all settings where agents engage in debates by providing pros and cons for their opinions therein. Specifically, we identify a number of formal properties that debate judgement may be required to satisfy in general, as concerns reproducibility, robustness, groundedness and explainability. Then, we explore their satisfaction formally and/or experimentally, for claim verification settings, for two specific alternative debate judgement methods: variants of the LLMs as a judge idea and formal semantics drawn from computational argumentation. We show that the two methods give similar accuracy performances but the former may lack formal guarantees that the latter brings. Overall, our study indicates argumentation semantics as an ideal candidate for principled judges in debate-driven AI.
