通过理论一致的对称多模态偏好优化缓解幻觉
Mitigating Hallucination Through Theory-Consistent Symmetric Multimodal Preference Optimization
AI总结：
SymMPO通过理论一致的对称多模态偏好优化方法，有效缓解多模态大语言模型中的幻觉问题。
摘要：
直接偏好优化（DPO）已经 emerged 作为一种有效的途径，用于缓解多模态大语言模型（MLLMs）中的幻觉。尽管现有方法通过利用视觉导向的对比目标来增强MLLMs对视觉输入的关注，从而减少幻觉，但它们存在非严谨的优化目标函数和间接偏好监督的问题。为了解决这些限制，我们提出了一种对称多模态偏好优化（SymMPO），通过直接偏好监督（即响应对）进行对称偏好学习，以增强视觉理解，同时保持与标准DPO的理论一致性。除了传统的顺序偏好学习外，SymMPO引入了偏好边缘一致性损失，以定量调节对称偏好对之间的偏好差距。在五个基准上的综合评估显示，SymMPO的优越性能验证了其在缓解MLLMs幻觉方面的有效性。
英文摘要：
Direct Preference Optimization (DPO) has emerged as an effective approach for mitigating hallucination in Multimodal Large Language Models (MLLMs). Although existing methods have achieved significant progress by utilizing vision-oriented contrastive objectives for enhancing MLLMs' attention to visual inputs and hence reducing hallucination, they suffer from non-rigorous optimization objective function and indirect preference supervision. To address these limitations, we propose a Symmetric Multimodal Preference Optimization (SymMPO), which conducts symmetric preference learning with direct preference supervision (i.e., response pairs) for visual understanding enhancement, while maintaining rigorous theoretical alignment with standard DPO. In addition to conventional ordinal preference learning, SymMPO introduces a preference margin consistency loss to quantitatively regulate the preference gap between symmetric preference pairs. Comprehensive evaluation across five benchmarks demonstrate SymMPO's superior performance, validating its effectiveness in hallucination mitigation of MLLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
