Detecting hallucinations in Large Language Model-generated text is crucial for their safe deployment. While probing classifiers show promise, they operate on isolated layer-token pairs and are LLM-specific, limiting their effectiveness and hindering cross-LLM applications. In this paper, we introduce a novel approach to address these shortcomings. We build on the natural sequential structure of activation data in both axes (layers $\times$ tokens) and advocate treating full activation tensors akin to images. We design ACT-ViT, a Vision Transformer-inspired model that can be effectively and efficiently applied to activation tensors and supports training on data from multiple LLMs simultaneously. Through comprehensive experiments encompassing diverse LLMs and datasets, we demonstrate that ACT-ViT consistently outperforms traditional probing techniques while remaining extremely efficient for deployment. In particular, we show that our architecture benefits substantially from multi-LLM training, achieves strong zero-shot performance on unseen datasets, and can be transferred effectively to new LLMs through fine-tuning. Full code is available at https://github.com/BarSGuy/ACT-ViT.
Detection Without Correction: A Robust Asymmetry in Activation-Based Hallucination Probing
Department of Computer Science and Engineering, Indian Institute of Technology Patna, India(印度理工学院帕纳分校计算机科学与工程系);Department of Computer Science, Rajarshi School of Management and Technology, Varanasi, India(印度瓦拉纳西 Rajarshi 管理与技术学校计算机科学系);Department of Electronics and Communication Engineering, Jaypee Institute of Information Technology, Noida, India(印度诺伊达 Jaypee 信息科技学院电子与通信工程系)
Unified Hallucination Detection for Multimodal Large Language Models
Ant Group(蚂蚁集团);College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);School of Software Technology, Zhejiang University(浙江大学软件学院)
In-Context Sharpness as Alerts: An Inner Representation Perspective for Hallucination Mitigation
City University of Hong Kong(香港城市大学);National University of Singapore(新加坡国立大学);Shanghai Jiao Tong University(上海交通大学);Stanford University(斯坦福大学);Penn State University(宾夕法尼亚州立大学);Hong Kong University of Science and Technology(香港科技大学)