arXiv 2407.046932024-12-19cs.CLcs.AI

ANAH-v2：大语言模型分析性幻觉标注的扩展

ANAH-v2: Scaling Analytical Hallucination Annotation of Large Language Models

Shanghai Jiao Tong University（上海交通大学）；Shanghai AI Laboratory（上海人工智能实验室）；The Chinese University of Hong Kong（香港中文大学）；HKGAI under InnoHK（香港人工智能研究院（InnoHK框架下））；Hong Kong University of Science and Technology（香港科技大学）

Yuzhe Gu, Ziwei Ji, Wenwei Zhang, Chengqi Lyu, Dahua Lin, Kai Chen

备注：: Accepted by NeurIPS 2024. Dataset, code, and model are released at https://github.com/open-compass/ANAH

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出ANAH-v2迭代自训练框架，基于EM算法同时扩展幻觉标注数据集并提升标注器准确性，最终7B参数标注器超越GPT-4，在HaluEval和HalluQA上取得SOTA，并有效缓解LLM幻觉。

摘要：

大语言模型（LLMs）在跨领域和广泛应用的的长篇问答任务中表现出幻觉问题。当前的幻觉检测和缓解数据集在领域和规模上存在局限，由于高昂的人工成本和现有幻觉标注器可靠性不足，难以进行扩展。为促进对LLM幻觉的可扩展监督，本文提出了一种迭代自训练框架，能够同时、渐进地扩展幻觉标注数据集并提升幻觉标注器的准确性。基于期望最大化（EM）算法，在每次迭代中，该框架首先应用幻觉标注流程对扩展后的数据集进行标注，然后在该数据集上训练更准确的幻觉标注器。这个新的幻觉标注器将被用于下一次迭代的幻觉标注流程中。大量实验结果表明，最终获得的仅有7B参数的幻觉标注器超越了GPT-4的性能，并在HaluEval和HalluQA上通过零样本推理获得了新的最先进幻觉检测结果。该标注器不仅能评估各种LLM在大规模数据集上的幻觉水平，还能帮助缓解LLM生成中的幻觉，在HaluEval上的自然语言推理（NLI）指标从25%提升至37%。

英文摘要：

Large language models (LLMs) exhibit hallucinations in long-form question-answering tasks across various domains and wide applications. Current hallucination detection and mitigation datasets are limited in domains and sizes, which struggle to scale due to prohibitive labor costs and insufficient reliability of existing hallucination annotators. To facilitate the scalable oversight of LLM hallucinations, this paper introduces an iterative self-training framework that simultaneously and progressively scales up the hallucination annotation dataset and improves the accuracy of the hallucination annotator. Based on the Expectation Maximization (EM) algorithm, in each iteration, the framework first applies a hallucination annotation pipeline to annotate a scaled dataset and then trains a more accurate hallucination annotator on the dataset. This new hallucination annotator is adopted in the hallucination annotation pipeline used for the next iteration. Extensive experimental results demonstrate that the finally obtained hallucination annotator with only 7B parameters surpasses the performance of GPT-4 and obtains new state-of-the-art hallucination detection results on HaluEval and HalluQA by zero-shot inference. Such an annotator can not only evaluate the hallucination levels of various LLMs on the large-scale dataset but also help to mitigate the hallucination of LLMs generations, with the Natural Language Inference (NLI) metric increasing from 25% to 37% on HaluEval.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。