本研究探讨了大型语言模型(LLM)智能体检测并纠正 AI 生成内容中幻觉的能力。一个主智能体被要求撰写一篇关于名为 Flipfloppidy 的虚构丹麦艺术家的博客,随后由另一个智能体审查其中的事实不准确之处。大多数 LLM 都幻觉该艺术家存在。在涉及主智能体与审查智能体不同组合的 4,900 次测试运行中,Llama3-70b 和 GPT-4 变体等先进 AI 模型在识别幻觉方面表现出近乎完美的准确率,并在接收反馈后于 85% 至 100% 的情况下成功修订输出。这些发现强调了先进 AI 模型显著提升生成内容准确性和可靠性的潜力,为改进 AI 工作流编排提供了一种有前景的方法。
英文摘要:
This study explores the ability of Large Language Model (LLM) agents to detect and correct hallucinations in AI-generated content. A primary agent was tasked with creating a blog about a fictional Danish artist named Flipfloppidy, which was then reviewed by another agent for factual inaccuracies. Most LLMs hallucinated the existence of this artist. Across 4,900 test runs involving various combinations of primary and reviewing agents, advanced AI models such as Llama3-70b and GPT-4 variants demonstrated near-perfect accuracy in identifying hallucinations and successfully revised outputs in 85% to 100% of cases following feedback. These findings underscore the potential of advanced AI models to significantly enhance the accuracy and reliability of generated content, providing a promising approach to improving AI workflow orchestration.
Mitigating Hallucination in Large Language Models: A Capability-Oriented Survey on RAG, Reasoning, and Agentic Systems
Electronic Information School, Wuhan University(武汉大学电子信息学院);School of Computing, Dublin City University(都柏林城市大学计算机学院);School of Public Health, Wuhan University(武汉大学公共卫生学院);Insight Centre for Data Analytics, University of Galway(Galway大学数据分析研究中心);Insight Centre for Data Analytics, Dublin City University(都柏林城市大学数据分析研究中心)
MMD-Flagger: Leveraging Maximum Mean Discrepancy to Detect Hallucinations
Université Côte d’Azur, CNRS, LJAD, France(法国埃克塞特大学、法国国家科学研究中心、LJAD研究所);Center for Artificial Intelligence and Data Science (CAIDAS)(人工智能与数据科学中心);Julius-Maximilans-Universität Würzburg, Germany(德国维尔茨堡约利乌斯-马克斯米利安大学)
AgentHallu: Benchmarking Automated Hallucination Attribution of LLM-based Agents
Beijing University of Posts and Telecommunications(北京邮电大学);Department of Computer Science & Technology, Tsinghua University(清华大学计算机科学与技术系);University of California, Santa Barbara(加州大学圣芭芭拉分校);Center for Research on Intelligent Perception and Computing, NLPR, CASIA(智能感知与计算研究中心,国家工程实验室)
A Comprehensive Survey on Reinforcement Learning-based Agentic Search: Foundations, Roles, Optimizations, Evaluations, and Applications
The Pennsylvania State University(宾夕法尼亚州立大学);The University of Utah(犹他大学);Amazon(亚马逊);Microsoft(微软);IBM T.J. Watson Research Center(IBM T.J.沃森研究中心);Michigan State University(密歇根州立大学)
CiteLLM: An Agentic Platform for Trustworthy Scientific Reference Discovery
Hong Kong Polytechnic University(香港理工大学);Beijing University of Posts and Telecommunications(北京邮电大学);Swiss Federal Technology Institute of Lausanne (EPFL)(洛桑联邦理工学院);Hong Kong University of Science and Technology (HKUST)(香港科学大学)
A Geometric Analysis of Small-sized Language Model Hallucinations
Engineering (CEMSE) division, King Abdullah University of Science and Technology (KAUST)(卡塔尔科技大学工程学院(CEMSE));Istituto di Informatica e Telematica (IIT), National Research Council of Italy (CNR)(意大利国家研究理事会信息与电信研究所(IIT));Department of Information Engineering, University of Pisa(比萨大学信息工程系)
Knowledge-Grounded Agentic Large Language Models for Multi-Hazard Understanding from Reconnaissance Reports
organization= Department One , addressline= Address One , city= City One , postcode= 00000 , state= State One , country= Country One;organization= Department Two , addressline= Address Two , city= City Two , postcode= 22222 , state= State Two , country= Country Two;organization= Zachry Department of Civil \& Environmental Engineering, Texas A\&M University , addressline= 3136 TAMU , city= College Station , postcode= 77843 , state= TX , country= USA;organization= Department of Engineering Technology;Industrial Distribution, Texas A\&M University , city= College Station , postcode= 77843 , state= TX , country= USA;organization= Department of Civil;Environmental Engineering, University of Illinois Urbana-Champaign , city= Urbana , postcode= 61801 , state= IL , country= USA
End-to-End Agentic RAG System Training for Traceable Diagnostic Reasoning
Shanghai Jiao Tong University(上海交通大学);Shanghai AI Laboratory(上海人工智能实验室);Xinhua Hospital affiliated to Shanghai Jiao Tong University School of Medicine(上海交通大学医学院附属新华医院);Shanghai Ninth People’s Hospital, Shanghai Jiao Tong University School of Medicine(上海交通大学医学院附属第九人民医院);Shanghai Jiao Tong University School of Medicine(上海交通大学医学院)
Agentic and Generative AI for Open-Source Intelligence and Cyber Investigations: Taxonomy, Evaluation, Challenges, and Future Directions
School of Computer Science and Mathematics, Keele University(基尔大学计算机科学与数学学院);Cybersecurity Institute, University of Liverpool(利物浦大学网络安全研究所);Department of Applied Computing IICL, University of Wales Trinity Saint David(威尔士特里尼达大学应用计算系);Deakin Cyber Research and Innovation Hub, Deakin University(德金大学网络安全研究与创新中心);Department of Information Systems and Cyber Security, The University of Texas at San Antonio(德克萨斯大学圣安东尼奥分校信息系与网络安全系)