LLMs能为事实核查生成忠实的解释吗?基于多智能体辩论的忠实可解释事实核查
Can LLMs Produce Faithful Explanations For Fact-checking? Towards Faithful Explainable Fact-Checking via Multi-Agent Debate
AI总结:
研究LLMs在事实核查中生成忠实解释的能力,提出多智能体辩论精炼(MADR)框架,通过多智能体迭代精炼显著提升解释的忠实度和可信度。
摘要:
事实核查研究已广泛探索了验证,但较少涉及自然语言解释的生成,而这对于用户信任至关重要。尽管大型语言模型(LLMs)在文本生成方面表现出色,但它们在事实核查中生成忠实解释的能力仍有待深入研究。我们的研究调查了LLMs生成此类解释的能力,发现零样本提示通常会导致不忠实。为了应对这些挑战,我们提出了多智能体辩论精炼(MADR)框架,利用多个LLMs作为具有不同角色的智能体,在迭代精炼过程中增强生成解释的忠实度。MADR确保最终解释经过严格验证,显著降低不忠实元素的可能性,并与提供的证据紧密对齐。实验结果表明,MADR显著提高了LLM生成解释对证据的忠实度,提升了这些解释的可信度和可靠性。
英文摘要:
Fact-checking research has extensively explored verification but less so the generation of natural-language explanations, crucial for user trust. While Large Language Models (LLMs) excel in text generation, their capability for producing faithful explanations in fact-checking remains underexamined. Our study investigates LLMs' ability to generate such explanations, finding that zero-shot prompts often result in unfaithfulness. To address these challenges, we propose the Multi-Agent Debate Refinement (MADR) framework, leveraging multiple LLMs as agents with diverse roles in an iterative refining process aimed at enhancing faithfulness in generated explanations. MADR ensures that the final explanation undergoes rigorous validation, significantly reducing the likelihood of unfaithful elements and aligning closely with the provided evidence. Experimental results demonstrate that MADR significantly improves the faithfulness of LLM-generated explanations to the evidence, advancing the credibility and trustworthiness of these explanations.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。