Recently, multimodal large language models (MLLMs) have demonstrated remarkable performance in visual-language tasks. However, the authenticity of the responses generated by MLLMs is often compromised by object hallucinations. We identify that a key cause of these hallucinations is the model's over-susceptibility to specific image frequency features in detecting objects. In this paper, we introduce Multi-Frequency Perturbations (MFP), a simple, cost-effective, and pluggable method that leverages both low-frequency and high-frequency features of images to perturb visual feature representations and explicitly suppress redundant frequency-domain features during inference, thereby mitigating hallucinations. Experimental results demonstrate that our method significantly mitigates object hallucinations across various model architectures. Furthermore, as a training-time method, MFP can be combined with inference-time methods to achieve state-of-the-art performance on the CHAIR benchmark.
ClinHallu: A Benchmark for Diagnosing Stage-Wise Hallucinations in Medical MLLM Reasoning
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));DAMO Academy, Alibaba Group(阿里巴巴达摩院);Hupan Lab(湖畔实验室);Zhejiang University(浙江大学)
HalluShift++: Bridging Language and Vision through Internal Representation Shifts for Hierarchical Hallucinations in MLLMs
Netaji Subhash Engineering College (NSEC)(奈尔贾伊·萨布哈工程学院);TCG Crest;Electronics and Communication Sciences Unit (ECSU)(电子与通信科学单位);Indian Statistical Institute(印度统计研究所)
EmotionHallucer: Evaluating Emotion Hallucinations in Multimodal Large Language Models
Lappeenranta-Lahti University of Technology LUT(拉佩兰塔-拉赫蒂技术大学);University of Oulu(奥卢大学);Southeast University(东南大学);Shanghai Jiao Tong University(上海交通大学);Brno University of Technology(布拉格技术大学)
Mitigating Visual Hallucinations via Semantic Curriculum Preference Optimization in MLLMs
School of Engineering, Westlake University, Hangzhou, China(西湖大学工程学院);School of Cyberspace Security, Nanjing University of Science and Technology, Nanjing, China(南京理工大学网络安全学院)
COPO: Causal-Oriented Policy Optimization for Hallucinations of MLLMs
University of Chinese Academy of Sciences(中国科学院大学);Institute of Software Chinese Academy of Sciences(中国科学院软件研究所);Wangxuan Institute of Computer Technology, Peking University(北京大学王轩计算机技术研究所);Amazon.com, Inc.(亚马逊公司)
Mitigating Behavioral Hallucination in Multimodal Large Language Models for Sequential Images
Provable Responsible AI and Data Analytics (PRADA) Lab(可证明负责任的人工智能与数据分析实验室);King Abdullah University of Science and Technology(国王阿卜杜勒阿齐兹大学);University of Electronic Science and Technology of China(中国电子科学技术大学);University of Copenhagen(哥本哈根大学)
Robust Multimodal Large Language Models Against Modality Conflict
School of Artificial Intelligence(人工智能学院);Data Science, University of Science(数据科学,科学大学);Department of Electronic Engineering(电子工程系);Information Science, University of Science(信息科学,科学大学);Huawei Technologies Co., Ltd.(华为技术有限公司)
FINER: MLLMs Hallucinate under Fine-grained Negative Queries
Technical University of Munich(慕尼黑技术大学);Munich Center for Machine Learning(慕尼黑机器学习中心);Helmholtz Munich(海德堡-慕尼黑亥姆霍尔茨中心);Google(谷歌);LTCI, Télécom Paris, Institut Polytechnique de Paris(LTCI,巴黎电信学院,巴黎理工学院)
MHSA: A Lightweight Framework for Mitigating Hallucinations via Steered Attention in LVLMs
Tsinghua University(清华大学);Tsinghua University, Tencent(清华大学腾讯);Tencent(腾讯);University of Science and Technology Beijing(北京科技大学);University of Macau(澳门大学)
Mitigating Multilingual Hallucination in Large Vision-Language Models
School of Computer Science & Technology, Huazhong University of Science and Technology(华中科技大学计算机科学与技术学院);School of Computing Science and Technology, Fudan University(复旦大学计算机科学技术学院);Wangxuan Institute of Computer Technology, Peking University(北京大学王选计算机研究所);School of Computing Science and Technology, Zhejiang Gongshang University(浙江工商大学计算机科学与技术学院);Department of Computer Science and Engineering, The Chinese University of Hong Kong(香港中文大学计算机科学与工程学系)
Prescribing the Right Remedy: Mitigating Hallucinations in Large Vision-Language Models via Targeted Instruction Tuning
Beijing Key Lab of Traffic Data Analysis and Mining(北京交通大数据分析与挖掘重点实验室);Beijing Jiaotong University(北京交通大学);School of Information Technology and Management(信息科学技术学院)
ADAPT: Attention Dynamics Alignment with Preference Tuning for Faithful MLLMs
University of Science and Technology of China(中国科学技术大学);Huawei Technologies Ltd.(华为技术有限公司);State Key Laboratory of Communication Content Cognition, People’s Daily Online(人民网传播内容认知国家重点实验室)
Tell Model Where to Look: Mitigating Hallucinations in MLLMs by Vision-Guided Attention
School of Computer Science and Technology, Beijing Institute of Technology(北京理工大学计算机科学与技术学院);Zhongguancun Academy(中关村学院);Southeast Academy of Information Technology, Beijing Institute of Technology(北京理工大学信息科技东南学院);Zhongguancun Laboratory(中关村实验室)
Mitigating Hallucination in Large Language Models: A Capability-Oriented Survey on RAG, Reasoning, and Agentic Systems
Electronic Information School, Wuhan University(武汉大学电子信息学院);School of Computing, Dublin City University(都柏林城市大学计算机学院);School of Public Health, Wuhan University(武汉大学公共卫生学院);Insight Centre for Data Analytics, University of Galway(Galway大学数据分析研究中心);Insight Centre for Data Analytics, Dublin City University(都柏林城市大学数据分析研究中心)