面向大语言模型幻觉消除的带预设立场反事实辩论方法
Counterfactual Debating with Preset Stances for Hallucination Elimination of LLMs
AI总结：
针对大语言模型因固有偏差易产生幻觉的问题，提出预设立场的反事实多智能体辩论框架CFMAD，通过多立场辩论与第三方裁判判定缓解幻觉，在多数据集上表现优于现有方法。
摘要：
大语言模型（LLMs）在各类自然语言处理任务中表现出色，但受幻觉问题困扰。现有方案已尝试利用LLMs的内在推理能力缓解幻觉，例如自校正和多样采样方法，但这类方法因固有偏差往往过度信任LLMs的初始答案。缓解该问题的关键在于打破LLMs的固有偏差以开展答案检验。为此，我们提出反事实多智能体辩论（CounterFactual Multi-Agent Debate, CFMAD）框架。CFMAD预设LLMs的立场，通过迫使LLMs为预设答案的正确性生成理由来打破其固有偏差；持有不同预设立场的LLMs与持怀疑态度的评论员围绕生成理由的合理性展开反事实辩论；最终由第三方裁判评估辩论过程以确定最终答案。在三类任务的四个数据集上开展的大量实验表明，CFMAD的性能优于现有方法。
英文摘要：
Large Language Models (LLMs) excel in various natural language processing tasks but struggle with hallucination issues. Existing solutions have considered utilizing LLMs' inherent reasoning abilities to alleviate hallucination, such as self-correction and diverse sampling methods. However, these methods often overtrust LLMs' initial answers due to inherent biases. The key to alleviating this issue lies in overriding LLMs' inherent biases for answer inspection. To this end, we propose a CounterFactual Multi-Agent Debate (CFMAD) framework. CFMAD presets the stances of LLMs to override their inherent biases by compelling LLMs to generate justifications for a predetermined answer's correctness. The LLMs with different predetermined stances are engaged with a skeptical critic for counterfactual debate on the rationality of generated justifications. Finally, the debate process is evaluated by a third-party judge to determine the final answer. Extensive experiments on four datasets of three tasks demonstrate the superiority of CFMAD over existing methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
