FaithEval：你的语言模型能否对上下文保持忠实，即使“月亮是棉花糖做的”
FaithEval: Can Your Language Model Stay Faithful to Context, Even If "The Moon is Made of Marshmallows"
AI总结：
FaithEval是一个新基准，通过不可回答、不一致和反事实三类上下文任务，评估LLM对上下文的忠实性，发现即使最先进模型也常失守，且模型增大未必提升忠实性。
摘要：
在大型语言模型（LLMs）和检索增强生成（RAG）系统中，确保对上下文的忠实性对于在现实世界应用中的可靠部署至关重要，因为不正确或不支持的信息可能会削弱用户的信任。尽管在标准基准测试上取得了进展，但忠实性幻觉——即模型生成与所提供上下文不一致的响应——仍然是一个重大挑战。在这项工作中，我们引入了FaithEval，一个新颖且全面的基准测试，专门用于评估LLMs在上下文场景中的忠实性，涵盖三个不同的任务：不可回答、不一致和反事实上下文。这些任务模拟了现实世界中的挑战，其中检索机制可能浮现不完整、矛盾或捏造的信息。FaithEval总共包含4.9K个高质量问题，通过一个严格的四阶段上下文构建和验证框架进行验证，该框架采用基于LLM的自动评估和人工验证。我们对各种开源和专有模型的广泛研究表明，即使是最先进的模型也常常难以保持对给定上下文的忠实，并且更大的模型不一定表现出更好的忠实性。项目可在以下网址获取：https://github.com/SalesforceAIResearch/FaithEval。
英文摘要：
Ensuring faithfulness to context in large language models (LLMs) and retrieval-augmented generation (RAG) systems is crucial for reliable deployment in real-world applications, as incorrect or unsupported information can erode user trust. Despite advancements on standard benchmarks, faithfulness hallucination-where models generate responses misaligned with the provided context-remains a significant challenge. In this work, we introduce FaithEval, a novel and comprehensive benchmark tailored to evaluate the faithfulness of LLMs in contextual scenarios across three diverse tasks: unanswerable, inconsistent, and counterfactual contexts. These tasks simulate real-world challenges where retrieval mechanisms may surface incomplete, contradictory, or fabricated information. FaithEval comprises 4.9K high-quality problems in total, validated through a rigorous four-stage context construction and validation framework, employing both LLM-based auto-evaluation and human validation. Our extensive study across a wide range of open-source and proprietary models reveals that even state-of-the-art models often struggle to remain faithful to the given context, and that larger models do not necessarily exhibit improved faithfulness.Project is available at: https://github.com/SalesforceAIResearch/FaithEval.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
