Large language models (LLMs) often fail to synthesize information from their context to generate an accurate response. This renders them unreliable in knowledge intensive settings where reliability of the output is key. A critical component for reliable LLMs is the integration of a robust fact-checking system that can detect hallucinations across various formats. While several open-access fact-checking models are available, their functionality is often limited to specific tasks, such as grounded question-answering or entailment verification, and they perform less effectively in conversational settings. On the other hand, closed-access models like GPT-4 and Claude offer greater flexibility across different contexts, including grounded dialogue verification, but are hindered by high costs and latency. In this work, we introduce VERITAS, a family of hallucination detection models designed to operate flexibly across diverse contexts while minimizing latency and costs. VERITAS achieves state-of-the-art results considering average performance on all major hallucination detection benchmarks, with $10\%$ increase in average performance when compared to similar-sized models and get close to the performance of GPT4 turbo with LLM-as-a-judge setting.
Hallucination to Truth: A Review of Fact-Checking and Factuality Evaluation in Large Language Models
Department of Computer Science and Engineering, United International University, Dhaka 1212, Bangladesh(乌姆特国际大学计算机科学与工程系);Department of Computer Science and Engineering, Daffodil International University, Dhaka-1341, Bangladesh(达芙尼国际大学计算机科学与工程系);Faculty of Science and Technology, Charles Darwin University, Casuarina, NT 0909, Australia(查尔斯·达尔文大学科学与技术学院)
Mitigating hallucinations in healthcare LLMs with granular fact-checking and domain-specific adaptation
Applied Artificial Intelligence and Intelligent Systems (AAIINS) Laboratory(应用人工智能与智能系统实验室);Department of Computer Science and Engineering(计算机科学与工程系);Department of Data Science and Artificial Intelligence(数据科学与人工智能系);Department of Software Engineering(软件工程系);Faculty of Science and Information Technology(科学与信息技术学院);Faculty of Science and Technology(科学与技术学院)
Improving Reliability and Explainability of Medical Question Answering through Atomic Fact Checking in Retrieval-Augmented LLMs
Technical University of Munich(慕尼黑工业大学);TUM University Hospital Rechts der Isar(慕尼黑工业大学 Rechts der Isar 大学医院);German Heart Center Munich(慕尼黑德国心脏中心);Helmholtz Zentrum München(慕尼黑亥姆霍兹中心);German Consortium for Translational Cancer Research(德国转化癌症研究联盟)
Fact-Checking the Output of Large Language Models via Token-Level Uncertainty Quantification
Center for Artificial Intelligence Technology(人工智能技术中心);MBZUAI(穆罕默德·本·扎耶德人工智能大学);AIRI(人工智能研究院);FRC CSC RAS(俄罗斯科学院计算机科学与控制研究中心联邦研究中心);The University of Melbourne(墨尔本大学);QCRI(卡塔尔计算研究所);HSE University(高等经济大学)
TrumorGPT: Graph-Based Retrieval-Augmented Large Language Model for Fact-Checking
Yam Pak Charitable Foundation School of Computing and Information Sciences, Saint Francis University, Hong Kong(Yam Pak Charitable Foundation 计算与信息科学学院,圣弗朗西斯大学,香港);Department of Applied Mathematics, Chung Yuan Christian University(应用数学系, Chung Yuan Christian University);College of Computing and Data Science, Nanyang Technological University(计算与数据科学学院,南洋理工大学)
AutoHall: Automated Factuality Hallucination Dataset Generation for Large Language Models
AGI Institute, School of Computer Science, Shanghai Jiao Tong University(AGI研究院,计算机科学学院,上海交通大学);Shanghai Key Laboratory of Trusted Data Circulation and Governance in Web3(上海可信数据流通与治理Web3重点实验室);School of Media & Communication, Shanghai Jiao Tong University(媒体与传播学院,上海交通大学)
AI Hallucination from Students' Perspective: A Thematic Analysis
Department of Computer and Information Engineering, Center for Cyber Physical Systems, Khalifa University(计算机与信息工程系,跨物理系统中心,哈利法大学);Department of Psychology, University Malaysia Sabah(心理学系,马来西亚Sabah大学)
Towards Detecting LLMs Hallucination via Markov Chain-based Multi-agent Debate Framework
Wangxuan Institute of Computer Technology, Peking University(北京大学王选计算机研究所);Gaoling School of Artifical Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院)
RSHallu: Dual-Mode Hallucination Evaluation for Remote-Sensing Multimodal Large Language Models with Domain-Tailored Mitigation
College of Computer Science, Chongqing University(重庆大学计算机科学学院);Heavy Rainfall Research Center of China(中国强降水研究中心);CMA Key Open Laboratory of Transforming Climate Resources to Economy, Chongqing Institute of Meteorological Sciences(中国气象局气候资源转化经济重点开放实验室、重庆气象科学研究院);Chongqing Metropolitan College of Science(重庆科学理工学院);School of Intelligence Science(智能科学学院);College of Artificial Intelligence, Harbin Institute of Technology(人工智能学院、哈尔滨工业大学);BNU-UIC Institute of Artificial Intelligence(北京师范大学-国际学院人工智能与未来网络研究院);Future Networks, Beijing Normal University at Zhuhai(未来网络、北京师范大学珠海分校)
Intelligent Identification and Repair of Design Defects in BIM via Domain-Specific Large Language Models
Tsinghua University(清华大学);Key Laboratory of Digital Construction and Digital Twin, Ministry of Housing and Urban-Rural Development(住房和城乡建设部数字建造与数字孪生重点实验室);PetroChina Planning & Engineering Institute(中国石油规划总院)
Are Large Language Models Good Fact Checkers: A Preliminary Study
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院)
Phrase-grounded Fact-checking for Automatically Generated Chest X-ray Reports
Rensselaer Polytechnic Institute, NY, USA(罗文学院);IBM Research, Almaden, CA, USA(IBM研究院);Stanford University, CA, USA(斯坦福大学);Massachusetts General Hospital (MGH), Boston, USA(麻省总医院)
Specula: Scaling formal specifications for autonomous model checking of system code
Nanjing University(南京大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Microsoft Research Asia(微软亚洲研究院);University of British Columbia(不列颠哥伦比亚大学)
SERC: LDPC-Inspired Semantic Error Correction for Retrieval-Augmented Generation
Department of Information Communications Engineering, Hankuk University of Foreign Studies, Republic of Korea(韩国外国语大学信息通信工程系);Division of Computer Engineering, Hankuk University of Foreign Studies, Republic of Korea(韩国外国语大学计算机工程系);Department of Statistics, Hankuk University of Foreign Studies, Republic of Korea(韩国外国语大学统计学系)