本文提出Decoding by Perturbation方法，通过动态文本扰动缓解多模态大语言模型的幻觉问题，通过控制文本干预减少语言先验的影响，提升解码稳定性。
摘要：
多模态大语言模型经常出现推理幻觉，部分原因在于语言先验主导视觉证据。现有免训练缓解方法要么扰动视觉表示偏离自然图像分布，要么强制侵入性操作损害模型生成流畅性。本文引入新的视角，认为多模态幻觉表现为解码阶段视觉 grounding 对文本表述的超敏感性。基于此，提出Decoding by Perturbation（DeP）框架，通过受控文本干预缓解先验诱导的幻觉。DeP采用动态探针应用多级文本扰动以激发潜在语言先验。利用注意力方差，增强稳定证据区域并抑制特征空间中的可疑噪声。进一步，通过logits统计构建可解释的先验漂移方向以对抗文本共现的概率偏见。大量实验验证DeP有效减少幻觉并实现多个基准测试中的优越性能。
英文摘要：
Multimodal Large Language Models frequently suffer from inference hallucinations, partially stemming from language priors dominating visual evidence. Existing training-free mitigation methods either perturb the visual representation and deviate from the natural image distribution, or enforce intrusive manipulations that compromise the model's inherent generative fluency. We introduce a novel perspective that multimodal hallucination manifests as the hypersensitivity of visual grounding to textual phrasing during the decoding phase. Building on this insight, we propose Decoding by Perturbation (DeP), a training-free framework mitigating prior-induced hallucinations via controlled textual interventions. DeP employs a dynamic probe applying multi-level textual perturbations to elicit latent language priors. Leveraging attention variance, it enhances stable evidence regions while suppressing suspicious noise in the feature space. Furthermore, it constructs an interpretable prior drift direction using logits statistics to counteract probability biases from textual co-occurrences. Extensive experiments confirm DeP effectively reduces hallucinations and achieves superior performance across multiple benchmarks.
ClinHallu: A Benchmark for Diagnosing Stage-Wise Hallucinations in Medical MLLM Reasoning
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；DAMO Academy, Alibaba Group（阿里巴巴达摩院）；Hupan Lab（湖畔实验室）；Zhejiang University（浙江大学）
HalluShift++: Bridging Language and Vision through Internal Representation Shifts for Hierarchical Hallucinations in MLLMs
Netaji Subhash Engineering College (NSEC)（奈尔贾伊·萨布哈工程学院）；TCG Crest；Electronics and Communication Sciences Unit (ECSU)（电子与通信科学单位）；Indian Statistical Institute（印度统计研究所）
EmotionHallucer: Evaluating Emotion Hallucinations in Multimodal Large Language Models
Lappeenranta-Lahti University of Technology LUT（拉佩兰塔-拉赫蒂技术大学）；University of Oulu（奥卢大学）；Southeast University（东南大学）；Shanghai Jiao Tong University（上海交通大学）；Brno University of Technology（布拉格技术大学）
Mitigating Visual Hallucinations via Semantic Curriculum Preference Optimization in MLLMs
School of Engineering, Westlake University, Hangzhou, China（西湖大学工程学院）；School of Cyberspace Security, Nanjing University of Science and Technology, Nanjing, China（南京理工大学网络安全学院）
COPO: Causal-Oriented Policy Optimization for Hallucinations of MLLMs
University of Chinese Academy of Sciences（中国科学院大学）；Institute of Software Chinese Academy of Sciences（中国科学院软件研究所）；Wangxuan Institute of Computer Technology, Peking University（北京大学王轩计算机技术研究所）；Amazon.com, Inc.（亚马逊公司）
Mitigating Behavioral Hallucination in Multimodal Large Language Models for Sequential Images
Provable Responsible AI and Data Analytics (PRADA) Lab（可证明负责任的人工智能与数据分析实验室）；King Abdullah University of Science and Technology（国王阿卜杜勒阿齐兹大学）；University of Electronic Science and Technology of China（中国电子科学技术大学）；University of Copenhagen（哥本哈根大学）
Robust Multimodal Large Language Models Against Modality Conflict
School of Artificial Intelligence（人工智能学院）；Data Science, University of Science（数据科学，科学大学）；Department of Electronic Engineering（电子工程系）；Information Science, University of Science（信息科学，科学大学）；Huawei Technologies Co., Ltd.（华为技术有限公司）
