arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2311.08702cs.AIcs.CL
收藏

辩论有助于监督不可靠的专家

Debate Helps Supervise Unreliable Experts

Cohere;New York University(纽约大学);Anthropic, PBC

Julian Michael, Salsabila Mahdi, David Rein, Jackson Petty, Julien Dirani, Vishakh Padmakumar, Samuel R. Bowman

AI总结:

本研究通过辩论机制监督不可靠专家,实验表明辩论比咨询基线显著提升评判准确率(84% vs 74%),且更高效,为监督强大AI系统提供新途径。

摘要:

随着人工智能系统被用于回答更困难的问题,并可能帮助创造新知识,判断其输出的真实性变得更加困难且更加重要。我们如何监督那些能够接触真相但可能无法准确报告真相的不可靠专家,使他们在监督者无法自行区分真伪的情况下,给出系统性地真实而非表面看似真实的答案?在这项工作中,我们表明,两个不可靠专家之间的辩论可以帮助非专家评判者更可靠地识别真相。我们收集了一个关于困难阅读理解问题的人类撰写辩论数据集,其中评判者未阅读源段落,只能看到专家论点和由接触段落的“专家”辩手选择性揭示的简短引文。在我们的辩论中,一位专家为正确答案辩护,另一位为错误答案辩护。将辩论与基线方法(我们称之为咨询)进行比较,在咨询中,单一专家仅为正确答案(正确概率为一半)辩护,我们发现辩论表现显著更好,评判者准确率达到84%,而咨询为74%。辩论也更高效,其长度仅为咨询的68%。通过比较人类与AI辩手,我们发现证据表明,随着辩手技能的提高(在此案例中为人类),辩论的表现上升,而咨询的表现下降。我们的错误分析也支持这一趋势,人类辩论中46%的错误归因于诚实辩手的失误(随着技能提高应减少);而人类咨询中52%的错误归因于辩手混淆评判者的相关证据(随着技能提高应恶化)。总体而言,这些结果表明,辩论是监督日益强大但可能不可靠的AI系统的一种有前景的方法。

英文摘要:

As AI systems are used to answer more difficult questions and potentially help create new knowledge, judging the truthfulness of their outputs becomes more difficult and more important. How can we supervise unreliable experts, which have access to the truth but may not accurately report it, to give answers that are systematically true and don't just superficially seem true, when the supervisor can't tell the difference between the two on their own? In this work, we show that debate between two unreliable experts can help a non-expert judge more reliably identify the truth. We collect a dataset of human-written debates on hard reading comprehension questions where the judge has not read the source passage, only ever seeing expert arguments and short quotes selectively revealed by 'expert' debaters who have access to the passage. In our debates, one expert argues for the correct answer, and the other for an incorrect answer. Comparing debate to a baseline we call consultancy, where a single expert argues for only one answer which is correct half of the time, we find that debate performs significantly better, with 84% judge accuracy compared to consultancy's 74%. Debates are also more efficient, being 68% of the length of consultancies. By comparing human to AI debaters, we find evidence that with more skilled (in this case, human) debaters, the performance of debate goes up but the performance of consultancy goes down. Our error analysis also supports this trend, with 46% of errors in human debate attributable to mistakes by the honest debater (which should go away with increased skill); whereas 52% of errors in human consultancy are due to debaters obfuscating the relevant evidence from the judge (which should become worse with increased skill). Overall, these results show that debate is a promising approach for supervising increasingly capable but potentially unreliable AI systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑