VLMs失去信心了吗？关于VLMs中谄媚现象的研究
Have the VLMs Lost Confidence? A Study of Sycophancy in VLMs
AI总结：
本研究将谄媚现象探索从LLMs扩展至VLMs，引入MM-SY基准进行评估，并提出基于提示、监督微调和DPO的方法有效缓解该问题，同时发现增强模型高层的图像注意力有助于减轻谄媚。
摘要：
在LLMs的研究中，谄媚是一种普遍存在的幻觉，对这些模型构成了重大挑战。具体而言，LLMs往往无法坚持原本正确的回答，而是盲目附和用户的观点，即使这些观点是错误的或恶意的。然而，关于视觉语言模型（VLMs）中谄媚现象的研究一直很匮乏。在这项工作中，我们将谄媚现象的探索从LLMs扩展到VLMs，引入了MM-SY基准来评估这一现象。我们展示了多个代表性模型的评估结果，填补了VLMs在谄媚研究方面的空白。为了缓解谄媚现象，我们提出了一个用于训练的合成数据集，并采用了基于提示、监督微调和DPO的方法。我们的实验表明，这些方法有效缓解了VLMs中的谄媚现象。此外，我们探测了VLMs以评估谄媚的语义影响，并分析了视觉token的注意力分布。我们的研究结果表明，防止谄媚的能力主要在模型的较高层中观察到。这些较高层对图像知识缺乏注意力可能导致谄媚，而在高层增强图像注意力被证明有助于缓解此问题。
英文摘要：
In the study of LLMs, sycophancy represents a prevalent hallucination that poses significant challenges to these models. Specifically, LLMs often fail to adhere to original correct responses, instead blindly agreeing with users' opinions, even when those opinions are incorrect or malicious. However, research on sycophancy in visual language models (VLMs) has been scarce. In this work, we extend the exploration of sycophancy from LLMs to VLMs, introducing the MM-SY benchmark to evaluate this phenomenon. We present evaluation results from multiple representative models, addressing the gap in sycophancy research for VLMs. To mitigate sycophancy, we propose a synthetic dataset for training and employ methods based on prompts, supervised fine-tuning, and DPO. Our experiments demonstrate that these methods effectively alleviate sycophancy in VLMs. Additionally, we probe VLMs to assess the semantic impact of sycophancy and analyze the attention distribution of visual tokens. Our findings indicate that the ability to prevent sycophancy is predominantly observed in higher layers of the model. The lack of attention to image knowledge in these higher layers may contribute to sycophancy, and enhancing image attention at high layers proves beneficial in mitigating this issue.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
