P$^2$-DPO: Grounding Hallucination in Perceptual Processing via Calibration Direct Preference Optimization
Guangdong Provincial Key Laboratory of Computational AI Models and Cognitive Intelligence, School of Computer Science & Engineering, South China University of Technology（广东省计算人工智能模型与认知智能重点实验室，计算机科学与工程学院，华南理工大学）；Pazhou Lab, Guangzhou, China（琶洲实验室，广州，中国）；Engineering Research Center of the Ministry of Education on Health Intelligent Perception and Paralleled Digital-Human, Guangzhou, China（教育部健康智能感知与并行数字人工程研究中心，广州，中国）
Ruipeng Zhang, Zhihao Li, Haozhang Yuan, C. L. Philip Chen, Tong Zhang
Hallucination has recently garnered significant research attention in Large Vision-Language Models (LVLMs). Direct Preference Optimization (DPO) aims to learn directly from the corrected preferences provided by humans, thereby addressing the hallucination issue. Despite its success, this paradigm has yet to specifically target the perceptual bottleneck in attended regions or address insufficient Visual Robustness against image degradation. Furthermore, existing preference pairs are often vision-agnostic and their inherently off-policy nature limits their effectiveness in guiding model learning. To address these challenges, we propose Perceptual Processing Direct Preference Optimization (P$^2$-DPO), a novel training paradigm in which the model generates and learns from its own preference pairs, thereby directly addressing the identified visual bottlenecks while inherently avoiding the issues of vision-agnostic and off-policy data. It introduces: (1) an on-policy preference pairs construction method targeting Focus-and-Enhance perception and Visual Robustness, and (2) a well-designed Calibration Loss to precisely align visual signals with the causal generation of text. Experimental results demonstrate that with a comparable amount of training data and cost, P$^2$-DPO outperforms strong baselines that rely on costly human feedback on benchmarks. Furthermore, evaluations on Attention Region Fidelity (ARF) and image degradation scenarios validate the effectiveness of P$^2$-DPO in addressing perceptual bottleneck in attended regions and improving Visual Robustness against degraded inputs.
Mitigating Hallucinations in Large Vision-Language Models via DPO: On-Policy Data Hold the Key
The Chinese University of Hong Kong（香港中文大学）；Microsoft Research Asia（微软亚洲研究院）；The Chinese University of Hong Kong, Shenzhen Research Institute (SZRI)（香港中文大学深圳研究院）
SECOND: Mitigating Perceptual Hallucination in Vision-Language Models via Selective and Contrastive Decoding
Department of Electrical and Computer Engineering, Seoul National University（首尔大学电气与计算机工程系）；Interdisciplinary Program in Artificial Intelligence, Seoul National University（首尔大学人工智能跨学科项目）
Mitigating Hallucination Through Theory-Consistent Symmetric Multimodal Preference Optimization
Shandong University（山东大学）；Southern University of Science and Technology（南方科技大学）；University of Georgia（佐治亚大学）；National University of Singapore（新加坡国立大学）；Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））
Direct Preference Optimization for Suppressing Hallucinated Prior Exams in Radiology Report Generation
Harvard University（哈佛大学）；Jawaharlal Institute of Postgraduate Medical Education & Research（贾瓦哈拉尔 postgraduate 医学教育与研究学院）；Johns Hopkins University（约翰斯·霍普金斯大学）
Systematic Reward Gap Optimization for Mitigating VLM Hallucinations
School of Software, Beihang University（北京航空航天大学软件学院）；Shanghai Innovation Institute（上海创新研究院）；Shanghai AI Laboratory（上海人工智能实验室）；Tsinghua University（清华大学）
Look Clearly Before Answering: Mitigating Hallucinations in LVLMs via Saliency-Driven Perceptual Realignment
Xidian University（西安电子科技大学）；Tsinghua University（清华大学）；Shanghai Road Transport Development Center（上海市道路运输发展中心）；Hunan Institute of Advanced Technology（湖南先进技术研究院）
Groc-PO: Grounded Context Preference Optimization for Truthful Multimodal LLMs
University of Science and Technology of China（中国科学技术大学）；Xiaomi Corporation（小米公司）；State Key Laboratory of Communication Content Cognition, People’s Daily Online（人民日报社传播内容认知国家重点实验室）
ADAPT: Attention Dynamics Alignment with Preference Tuning for Faithful MLLMs
University of Science and Technology of China（中国科学技术大学）；Huawei Technologies Ltd.（华为技术有限公司）；State Key Laboratory of Communication Content Cognition, People’s Daily Online（人民网传播内容认知国家重点实验室）
FINER: MLLMs Hallucinate under Fine-grained Negative Queries
Technical University of Munich（慕尼黑技术大学）；Munich Center for Machine Learning（慕尼黑机器学习中心）；Helmholtz Munich（海德堡-慕尼黑亥姆霍尔茨中心）；Google（谷歌）；LTCI, Télécom Paris, Institut Polytechnique de Paris（LTCI，巴黎电信学院，巴黎理工学院）
Mitigating Visual Hallucinations via Semantic Curriculum Preference Optimization in MLLMs
School of Engineering, Westlake University, Hangzhou, China（西湖大学工程学院）；School of Cyberspace Security, Nanjing University of Science and Technology, Nanjing, China（南京理工大学网络安全学院）
Mitigating Object Hallucinations via Sentence-Level Early Intervention
Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；The Chinese University of Hong Kong（香港中文大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））
Online Self-Calibration Against Hallucination in Vision-Language Models
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络安全学院）
