Unsupervised hallucination detection aims to identify hallucinated content generated by large language models (LLMs) without relying on labeled data. While unsupervised methods have gained popularity by eliminating labor-intensive human annotations, they frequently rely on proxy signals unrelated to factual correctness. This misalignment biases detection probes toward superficial or non-truth-related aspects, limiting generalizability across datasets and scenarios. To overcome these limitations, we propose IRIS, an unsupervised hallucination detection framework, leveraging internal representations intrinsic to factual correctness. IRIS prompts the LLM to carefully verify the truthfulness of a given statement, and obtain its contextualized embedding as informative features for training. Meanwhile, the uncertainty of each response is considered a soft pseudolabel for truthfulness. Experimental results demonstrate that IRIS consistently outperforms existing unsupervised methods. Our approach is fully unsupervised, computationally low cost, and works well even with few training data, making it suitable for real-time detection.
School of Computer Science, McGill University（麦吉尔大学计算机科学学院）；MILA（蒙特利尔学习算法研究所）
State-of-the-art abstractive summarization systems often generate \emph{hallucinations}; i.e., content that is not directly inferable from the source text. Despite being assumed incorrect, we find that much hallucinated content is factual, namely consistent with world knowledge. These factual hallucinations can be beneficial in a summary by providing useful background information. In this work, we propose a novel detection approach that separates factual from non-factual hallucinations of entities. Our method utilizes an entity's prior and posterior probabilities according to pre-trained and f…
Probabilistic distances-based hallucination detection in LLMs with RAG
Markov Lab, Department of Mathematics（马尔可夫实验室，数学系）；Computer Science, Saint-Petersburg University（计算机科学，圣彼得堡大学）；AI Center, Skoltech（人工智能中心，斯克里普丘克技术学院）；SB AI Lab（SB人工智能实验室）；AI Center, Skoltech, Risk department, Sber（人工智能中心，斯克里普丘克技术学院，风险部门）
Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models
Yale University（耶鲁大学）；Yale Law School（耶鲁法学院）；Stanford University（斯坦福大学）；Stanford Law School（斯坦福法学院）；Stanford Institute for Economic Policy Research（斯坦福经济政策研究所）；Regulation, Evaluation, and Governance Lab（规制、评估与治理实验室）
Efficient Hallucination Detection for LLMs Using Uncertainty-Aware Attention Heads
Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)（莫扎德人工智能大学）；ETH Zurich（苏黎世联邦理工学院）；Independent Researcher（独立研究者）；Applied AI Institute（应用人工智能研究所）
Optimal Transport for Unsupervised Hallucination Detection in Neural Machine Translation
Instituto de Telecomunicações（电信研究所）；Instituto Superior Técnico（高等技术学院）；University of Lisbon（里斯本大学）；CentraleSupélec（中央理工-高等电力学院）；Université Paris-Saclay（巴黎萨克雷大学）；CNRS（法国国家科学研究中心）；Unbabel（Unbabel公司）
Data mining allows the exploration of sequences of phenomena, whereas one usually tends to focus on isolated phenomena or on the relation between two phenomena. It offers invaluable tools for theoretical analyses and exploration of the structure of sentences, texts, dialogues, and speech. We report here the results of an attempt at using it for inspecting sequences of verbs from French accounts of road accidents. This analysis comes from an original approach of unsupervised training allowing the discovery of the structure of sequential data. The entries of the analyzer were only made of the ve…
Shanghai Jiao Tong University（上海交通大学）；Tencent AI Lab（腾讯人工智能实验室）
Back-translation is a critical component of Unsupervised Neural Machine Translation (UNMT), which generates pseudo parallel data from target monolingual data. A UNMT model is trained on the pseudo parallel data with translated source, and translates natural source sentences in inference. The source discrepancy between training and inference hinders the translation performance of UNMT models. By carefully designing experiments, we identify two representative characteristics of the data gap in source: (1) style gap (i.e., translated vs. natural text style) that leads to poor generalization capab…
