大型语言模型中用于幻觉评估的忠实度指标综述
A review of faithfulness metrics for hallucination assessment in Large Language Models
AI总结：
本文综述了开放式任务中的忠实度评估方法，发现使用LLM作为评估器与人类判断最相关，并指出RAG和提示框架能有效缓解幻觉，强调开放式生成评估比多选题基准更能全面衡量LLM性能并提升信任。
摘要：
本综述考察了在开放式摘要、问答和机器翻译任务中评估忠实度的方法。我们发现，使用LLM作为忠实度评估器通常是与人类判断最高度相关的指标。文章讨论了其他研究缓解幻觉的方法，其中检索增强生成（RAG）和提示框架方法都与更高的忠实度相关，同时还提供了其他缓解建议。对忠实度的研究对于LLM的持续广泛使用至关重要，因为不忠实的回答可能对许多原本适合使用LLM的领域构成重大风险。此外，评估开放式生成比常用的多选题基准测试提供了更全面的LLM性能衡量标准，这有助于提升对LLM的信任。
英文摘要：
This review examines the means with which faithfulness has been evaluated across open-ended summarization, question-answering and machine translation tasks. We find that the use of LLMs as a faithfulness evaluator is commonly the metric that is most highly correlated with human judgement. The means with which other studies have mitigated hallucinations is discussed, with both retrieval augmented generation (RAG) and prompting framework approaches having been linked with superior faithfulness, whilst other recommendations for mitigation are provided. Research into faithfulness is integral to the continued widespread use of LLMs, as unfaithful responses can pose major risks to many areas whereby LLMs would otherwise be suitable. Furthermore, evaluating open-ended generation provides a more comprehensive measure of LLM performance than commonly used multiple-choice benchmarking, which can help in advancing the trust that can be placed within LLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
