FACTOID: FACtual enTailment fOr hallucInation Detection
AI Institute, University of South Carolina(南卡罗来纳大学AI学院);Islamic University of Technology(伊斯兰技术大学);National Institute of Technology, Silchar(锡尔卡尔国家理工学院);Indira Gandhi Delhi Technical University for Women(英迪拉·甘地德里女子技术大学);Stanford University(斯坦福大学);Amazon AI(亚马逊人工智能)
Vipula Rawte, S. M Towhidul Islam Tonmoy, Krishnav Rajbangshi, Shravani Nag, Aman Chadha, Amit P. Sheth, Amitava Das
大语言模型(LLMs)的广泛采用带来了诸多益处。然而,幻觉是一个重大关切。为此,检索增强生成(RAG)已成为一种极具前景的范式,通过将LLM输出建立在事实信息之上来改进其输出。RAG依赖文本蕴含(TE)或类似方法,来检查LLM生成的文本相对于检索到的文档是得到支持还是相互矛盾。本文认为,传统的TE方法不足以发现LLM生成内容中的幻觉。例如,考虑一个关于“美国在乌克兰战争上的立场”的提示。AI生成的文本写道:“……美国总统巴拉克·奥巴马表示美国不会向乌克兰派兵……”然而,在战争期间,美国总统是乔·拜登,这与事实现实相矛盾。此外,当前的TE系统无法准确标注给定文本并识别出被矛盾的确切部分。为解决这一问题,我们引入了一种新型TE,称为“事实蕴含(FE)”,旨在检测LLM生成内容中的事实不准确之处,同时突出与事实现实相矛盾的具体文本片段。我们提出了FACTOID(FACTual enTAILment for hallucInation Detection),一个用于FE的基准数据集。我们提出了一个用于FE的多任务学习(MTL)框架,融合了最先进的(SoTA)长文本嵌入,如e5-mistral-7b-instruct,以及GPT-3、SpanBERT和RoFormer。与SoTA TE方法相比,所提出的用于FE的MTL架构在FACTOID基准上平均实现了40%的准确率提升。由于FE能够自动检测幻觉,我们评估了15个现代LLM,并使用我们提出的自动幻觉脆弱性指数(HVI_auto)对它们进行排名。该指数量化并提供了一种比较尺度,以根据LLM的幻觉情况对其进行评估和排名。
英文摘要:
The widespread adoption of Large Language Models (LLMs) has facilitated numerous benefits. However, hallucination is a significant concern. In response, Retrieval Augmented Generation (RAG) has emerged as a highly promising paradigm to improve LLM outputs by grounding them in factual information. RAG relies on textual entailment (TE) or similar methods to check if the text produced by LLMs is supported or contradicted, compared to retrieved documents. This paper argues that conventional TE methods are inadequate for spotting hallucinations in content generated by LLMs. For instance, consider a prompt about the 'USA's stance on the Ukraine war''. The AI-generated text states, ...U.S. President Barack Obama says the U.S. will not put troops in Ukraine...'' However, during the war the U.S. president is Joe Biden which contradicts factual reality. Moreover, current TE systems are unable to accurately annotate the given text and identify the exact portion that is contradicted. To address this, we introduces a new type of TE called ``Factual Entailment (FE).'', aims to detect factual inaccuracies in content generated by LLMs while also highlighting the specific text segment that contradicts reality. We present FACTOID (FACTual enTAILment for hallucInation Detection), a benchmark dataset for FE. We propose a multi-task learning (MTL) framework for FE, incorporating state-of-the-art (SoTA) long text embeddings such as e5-mistral-7b-instruct, along with GPT-3, SpanBERT, and RoFormer. The proposed MTL architecture for FE achieves an avg. 40\% improvement in accuracy on the FACTOID benchmark compared to SoTA TE methods. As FE automatically detects hallucinations, we assessed 15 modern LLMs and ranked them using our proposed Auto Hallucination Vulnerability Index (HVI_auto). This index quantifies and offers a comparative scale to evaluate and rank LLMs according to their hallucinations.
Gold Points Sniper: Self-guided Visual Reasoning in VLM for Fine-grained Action Understanding
Beijing National Research Center for Information Science and Technology (BNRist), Department of Automation, Tsinghua University(清华大学自动化系北京信息科学与技术国家研究中心);State Key Laboratory of General Artificial Intelligence, Beijing Institute for General Artificial Intelligence (BIGAI)(北京通用人工智能研究院通用人工智能国家重点实验室)
Logical Structure as Knowledge: Enhancing LLM Reasoning via Structured Logical Knowledge Density Estimation
Huzhou University(湖州大学);Zhejiang Key Laboratory of Intelligent Education Technology and Application(浙江智能教育技术与应用重点实验室);Banbu AI Foundation(Banbu AI基金会);Baichuan Inc(白川公司);University of Edinburgh(爱丁堡大学);Zhejiang University(浙江大学)
FactEHR: A Dataset for Evaluating Factuality in Clinical Notes Using LLMs
Khoury College of Computer Sciences, Northeastern University(东北大学克劳尔计算机科学学院);Center for Biomedical Informatics Research, Stanford University(斯坦福大学生物医学信息学研究中心);Department of Biomedical Data Science, Stanford School of Medicine(斯坦福医学院生物医学数据科学系);Stanford Health Care(斯坦福健康系统);Department of Medicine, Stanford School of Medicine(斯坦福医学院医学系);Clinical Excellence Research Center, Stanford School of Medicine(斯坦福医学院临床卓越研究中心);Department of Anesthesiology, Perioperative & Pain Medicine, Stanford School of Medicine(斯坦福医学院麻醉学、手术及疼痛医学系);Department of Dermatology, Stanford School of Medicine(斯坦福医学院皮肤病学系);Technology and Digital Solutions, Stanford Health Care(斯坦福健康系统技术与数字解决方案)
Beyond Factual Accuracy: Evaluating Global Reasoning Integrity in RAG Systems with LogicScore
Shanxi University(山西大学);Nanjing University of Science and Technology(南京理工大学);University of Manchester(曼彻斯特大学);Singapore University of Technology and Design(新加坡科技设计大学);University of Edinburgh(爱丁堡大学)
Do LVLMs Understand Charts? Analyzing and Correcting Factual Errors in Chart Captioning
University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Columbia University(哥伦比亚大学);Salesforce AI Research(Salesforce AI研究院);DAMO Academy, Alibaba Group(阿里巴巴达摩院)