LPOI：面向视觉语言模型的列表式偏好优化
LPOI: Listwise Preference Optimization for Vision Language Models
AI总结：
本文提出LPOI，首个面向VLM的物体感知列表式偏好优化方法，通过掩码关键物体并在正负图像间插值生成图像序列训练模型排序，有效减少幻觉并提升性能。
摘要：
将大型视觉语言模型（VLM）与人类偏好对齐是一项具有挑战性的任务，因为诸如RLHF和DPO等方法常常会过拟合于文本信息或加剧幻觉现象。尽管增加负样本图像在一定程度上缓解了这些缺陷，但由于构建列表式图像样本的复杂性和成本，此前尚无研究在VLM中采用列表式偏好优化。在本研究中，我们提出了LPOI，这是首个旨在减少VLM中幻觉的物体感知列表式偏好优化方法。LPOI识别并掩码图像中的关键物体，随后在正负图像之间对掩码区域进行插值，从而形成一系列逐渐趋于完整的图像序列。模型被训练按照物体可见度的升序对这些图像进行排序，在有效减少幻觉的同时保持视觉保真度。LPOI无需标准成对偏好数据之外的额外标注，因为它通过物体掩码和插值自动构建排序列表。在MMHalBench、AMBER和Object HalBench上的综合实验证实，LPOI在减少幻觉和增强VLM性能方面优于现有的偏好优化方法。我们的代码已在 https://github.com/fatemehpesaran310/lpoi 公开。
英文摘要：
Aligning large VLMs with human preferences is a challenging task, as methods like RLHF and DPO often overfit to textual information or exacerbate hallucinations. Although augmenting negative image samples partially addresses these pitfalls, no prior work has employed listwise preference optimization for VLMs, due to the complexity and cost of constructing listwise image samples. In this work, we propose LPOI, the first object-aware listwise preference optimization developed for reducing hallucinations in VLMs. LPOI identifies and masks a critical object in the image, and then interpolates the masked region between the positive and negative images to form a sequence of incrementally more complete images. The model is trained to rank these images in ascending order of object visibility, effectively reducing hallucinations while retaining visual fidelity. LPOI requires no extra annotations beyond standard pairwise preference data, as it automatically constructs the ranked lists through object masking and interpolation. Comprehensive experiments on MMHalBench, AMBER, and Object HalBench confirm that LPOI outperforms existing preference optimization methods in reducing hallucinations and enhancing VLM performance. We make the code available at https://github.com/fatemehpesaran310/lpoi.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
