Most hallucination evaluations focus on English, leaving it unclear whether findings transfer to lower-resource languages. We investigate faithfulness hallucinations, defined as model-generated content that is fluent and plausible but diverges from the provided input or is internally inconsistent. Leveraging the multilingual MultiWikiQA dataset, we utilize the LettuceDetect framework to create synthetic hallucination datasets for 306 languages, from which we train token-level hallucination classifiers for 30 European languages. In this work, we present evaluations of model hallucinations on a selection of languages: English, Danish, German, and Icelandic. Using these classifiers, we evaluate the hallucination rates for Qwen3-0.6B, Qwen3-14B, Gemma-3-12B-IT, cogito-v1-preview-qwen-32B, and cogito-v1-preview-llama-70B. Our classifiers reveal notably higher hallucination rates for Qwen3-0.6B (up to 60\% of answers containing at least one hallucination, peaking in Icelandic) and generally lower rates for larger models, with cogito-v1-preview-qwen-32B and cogito-v1-preview-llama-70B performing best on most languages. Hallucination rates are consistently higher for lower-resource languages, particularly Icelandic.
Hallucinations in Large Multilingual Translation Models
Instituto de Telecomunicações(电信研究所);Instituto Superior Técnico, University of Lisbon(里斯本大学高等技术学院);University of Edinburgh(爱丁堡大学);CentraleSupélec, Université Paris-Saclay(巴黎萨克雷大学中央理工-高等电力学院);Unbabel(Unbabel公司)
ChartInsighter: An Approach for Mitigating Hallucination in Time-series Chart Summary Generation with A Benchmark Dataset
Henan Institute of Advanced Technology, Zhengzhou University(郑州大学河南先进技术研究院);Zhengzhou Zhongke Institute of Integrated Circuit and System Application(郑州中科集成电路与系统应用研究院);Fudan University(复旦大学);School of Computer Science, Fudan University(复旦大学计算机科学技术学院)
Can Humans Dream of Electric Sheep? Human-Written Samples for Fine-Grained Vision-and-Language Hallucination Benchmarking
University of Helsinki(赫尔辛基大学);Politecnico di Torino(都灵理工大学);Universiteit Utrecht(乌得勒支大学);Université Bretagne Sud(南布列塔尼大学);University of Copenhagen(哥本哈根大学);University Grenoble Alpes(格勒诺布尔阿尔卑斯大学);University of Lorraine(洛林大学)
MULTITuDE: Large-Scale Multilingual Machine-Generated Text Detection Benchmark
Kempelen Institute of Intelligent Technologies(肯佩伦智能技术研究院);The Pennsylvania State University(宾夕法尼亚州立大学);University of Mississippi(密西西比大学);MIT Lincoln Laboratory(麻省理工学院林肯实验室)
AraHalluEval: A Fine-grained Hallucination Evaluation Framework for Arabic LLMs
Information and Computer Science Department, King Fahd University of Petroleum and Minerals(信息与计算机科学系,国王法赫德石油和矿物大学);SDAIA-KFUPM Joint Research Center for Artificial Intelligence(SDAIA-KFUPM人工智能联合研究中心)
SemEval-2025 Task 3: Mu-SHROOM, the Multilingual Shared Task on Hallucinations and Related Observable Overgeneration Mistakes
University of Helsinki(赫尔辛基大学);SiLO;Université de Lorraine & ICANS Strasbourg(洛林大学及斯特拉斯堡ICANS);Université Bretagne Sud(布列塔尼南方大学);CIMAT A. C.;University of Milano-Bicocca(米兰-布雷拉大学);TU Darmstadt(德累斯顿技术大学);Aveni;Charles University(查尔斯大学);HiTZ Basque Center for Language Technology - Ixa(巴斯克语言技术中心-ixa)
CCNU at SemEval-2025 Task 3: Leveraging Internal and External Knowledge of Large Language Models for Multilingual Hallucination Annotation
Hubei Provincial Key Laboratory of Artificial Intelligence and Smart Learning(湖北省人工智能与智能学习重点实验室);National Language Resources Monitoring and Research Center for Network Media(网络媒体语言资源监测与研究中心);School of Computer Science, Central China Normal University(中央财经大学计算机学院)
MASSIVE Multilingual Abstract Meaning Representation: A Dataset and Baselines for Hallucination Detection
Paul G. Allen School of Computer Science & Engineering(保罗·G·艾伦计算机科学与工程学院);Georgetown University(乔治城大学);University of Colorado Boulder(科罗拉多大学博尔德分校);Amazon(亚马逊)
Litmus (Re)Agent: A Benchmark and Agentic System for Predictive Evaluation of Multilingual Models
Microsoft Corporation, India(微软公司(印度));Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)(穆罕默德·本·扎耶德人工智能大学);Indian Institute of Technology Hyderabad(印度理工学院海得拉巴分校)
Mitigating Multilingual Hallucination in Large Vision-Language Models
School of Computer Science & Technology, Huazhong University of Science and Technology(华中科技大学计算机科学与技术学院);School of Computing Science and Technology, Fudan University(复旦大学计算机科学技术学院);Wangxuan Institute of Computer Technology, Peking University(北京大学王选计算机研究所);School of Computing Science and Technology, Zhejiang Gongshang University(浙江工商大学计算机科学与技术学院);Department of Computer Science and Engineering, The Chinese University of Hong Kong(香港中文大学计算机科学与工程学系)
Output Language Confusion under Multilingual Prompt Contamination
University of Tübingen(蒂宾根大学);Artificial Intelligence Institute, University of South Carolina(南卡罗来纳大学人工智能研究所);Thapar Institute of Engineering & Technology(塔帕尔工程技术学院);Indian Institute of Technology Kanpur(印度理工学院坎普尔分校);Indian AI Research Organization(印度人工智能研究组织)
NCL-UoR at SemEval-2025 Task 3: Detecting Multilingual Hallucination and Related Observable Overgeneration Text Spans with Modified RefChecker and Modified SeflCheckGPT
School of Computing, Newcastle University(新萨勒姆大学计算机学院);Department of Computer Science, University of Reading(阅读大学计算机科学系);Previously at School of Computing, Newcastle University(新萨勒姆大学计算机学院)
Confabulations from ACL Publications (CAP): A Dataset for Scientific Hallucination Detection
Charles University, Prague(查尔斯大学,布拉格);Independent Researcher(独立研究者);University of Helsinki, Finland(赫尔辛基大学);University of British Columbia, Vancouver, Canada(不列颠哥伦比亚大学,温哥华,加拿大);Chosun University, South Korea(成均馆大学,韩国)