面向指令的偏好对齐以增强多模态大语言模型（MLLMs）的多模态理解能力
Instruction-Oriented Preference Alignment for Enhancing Multi-Modal Comprehension Capability of MLLMs
AI总结：
针对现有偏好对齐方法忽略多模态理解关键因素、改进局限于幻觉缓解的问题，提出面向指令的偏好对齐框架IPA，通过自动化偏好构建、验证及渐进式收集流水线提升MLLMs通用理解能力，在多基准测试中验证有效。
摘要：
偏好对齐已成为在监督微调后提升多模态大语言模型（Multimodal Large Language Models, MLLMs）性能的有效策略。现有偏好对齐方法主要针对幻觉因素，却忽略了对多模态理解能力至关重要的因素，其改进往往局限于幻觉缓解层面。为弥补这一差距，我们提出面向指令的偏好对齐（Instruction-oriented Preference Alignment, IPA）——这是一个可扩展的框架，旨在基于指令执行效果自动构建对齐偏好。我们的方法包含自动化偏好构建与专门的验证流程，可识别面向指令的因素，避免响应表征出现显著差异。此外，IPA 融合了渐进式偏好收集流水线，通过模型自进化和参考引导的细化进一步召回难例。在 Qwen2VL-7B 上开展的实验表明，IPA 在多个基准测试（包括幻觉评估、视觉问答和文本理解任务）中均表现出有效性，凸显了其提升通用理解能力的潜力。
英文摘要：
Preference alignment has emerged as an effective strategy to enhance the performance of Multimodal Large Language Models (MLLMs) following supervised fine-tuning. While existing preference alignment methods predominantly target hallucination factors, they overlook the factors essential for multi-modal comprehension capabilities, often narrowing their improvements on hallucination mitigation. To bridge this gap, we propose Instruction-oriented Preference Alignment (IPA), a scalable framework designed to automatically construct alignment preferences grounded in instruction fulfillment efficacy. Our method involves an automated preference construction coupled with a dedicated verification process that identifies instruction-oriented factors, avoiding significant variability in response representations. Additionally, IPA incorporates a progressive preference collection pipeline, further recalling challenging samples through model self-evolution and reference-guided refinement. Experiments conducted on Qwen2VL-7B demonstrate IPA's effectiveness across multiple benchmarks, including hallucination evaluation, visual question answering, and text understanding tasks, highlighting its capability to enhance general comprehension.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
