arXiv 2506.071802026-04-30cs.CLcs.AIcs.CV

视频中的奉承：视频大语言模型中奉承行为的基准测试与分析

Flattery in Motion: Benchmarking and Analyzing Sycophancy in Video-LLMs

Provable Responsible AI and Data Analytics (PRADA) Lab（可证负责任人工智能与数据 analytics 实验室）；King Abdullah University of Science and Technology（国王阿卜杜勒阿齐兹科学与技术大学）；HKUST（香港科技大学）；MBZUAI（穆罕默德·本·拉希德人工智能研究所）；University of Science and Technology of China（中国科学技术大学）；Kyungpook National University（庆尚国立大学）

Wenrui Zhou, Mohamed Hendy, Shu Yang, Qingsong Yang, Zikun Guo, Yuyu Luo, Lijie Hu, Di Wang

备注：: 27 Pages, Accepted by ACL 2026 Main Conference

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出VISE基准，用于评估视频大语言模型在面对误导性输入时的奉承行为，通过多类型分析和两种无训练缓解策略提升模型可靠性。

摘要：

随着视频大语言模型（Video-LLMs）在需要基于多模态推理的现实应用中日益集成，确保其事实一致性和可靠性至关重要。然而，这些模型倾向于与用户输入一致，即使这与视觉证据相矛盾，这在这些情境中损害了其可信度。当前关于奉承的研究大多忽略了视频语言领域中的具体表现，导致缺乏系统性的基准和有针对性的评估来理解Video-LLMs在误导性用户输入下的响应。为此，我们提出了VISE（Video-LLM Sycophancy Benchmarking and Evaluation），这是首个评估最先进Video-LLMs在多种问题格式、提示偏见和视觉推理任务中奉承行为的基准。具体而言，VISE首次将语言学视角引入视频领域，使能够在多种奉承类型和交互模式上进行细粒度分析。此外，我们提出了两种潜在的无训练缓解策略，揭示了减少奉承偏见的可能路径：（i）通过可解释的关键帧选择增强视觉 grounding，（ii）通过在推理时间对内部神经表示进行针对性干预来引导模型行为远离奉承。我们的代码可在https://anonymous.4open.science/r/VideoSycophancy-567F获取。

英文摘要：

As video large language models (Video-LLMs) become increasingly integrated into real-world applications that demand grounded multimodal reasoning, ensuring their factual consistency and reliability is of critical importance. However, sycophancy, the tendency of these models to align with user input even when it contradicts the visual evidence, undermines their trustworthiness in such contexts. Current sycophancy research has largely overlooked its specific manifestations in the videolanguage domain, resulting in a notable absence of systematic benchmarks and targeted evaluations to understand how Video-LLMs respond under misleading user input. To fill this gap, we propose VISE(Video-LLM Sycophancy Benchmarking and Evaluation), the first benchmark designed to evaluate sycophantic behavior in state-of-the-art Video-LLMs across diverse question formats, prompt biases, and visual reasoning tasks. Specifically, VISEpioneeringly brings linguistic perspectives on sycophancy into the video domain, enabling fine-grained analysis across multiple sycophancy types and interaction patterns. Furthermore, we propose two potential training-free mitigation strategies revealing potential paths for reducing sycophantic bias: (i) enhancing visual grounding through interpretable key-frame selection and (ii) steering model behavior away from sycophancy via targeted, inference-time intervention on its internal neural representations. Our code is available at https://anonymous.4open.science/r/VideoSycophancy-567F.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。