arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2606.04596cs.CL
收藏

多视频摘要中位置偏差的系统评估:基于多模态大语言模型

A Systematic Evaluation of Positional Bias in Multi-Video Summarization with MLLMs

School of Artificial Intelligence, Jilin University(吉林大学人工智能学院);Engineering Research Center of Knowledge-Driven Human-Machine Intelligence, Jilin University(知识驱动人机智能工程研究中心);International Center of Future Science, Jilin University(未来科学国际中心)

Huangchen Xu, Yuan Wu, Yi Chang

AI总结:

本研究系统评估了多模态大语言模型在多视频摘要任务中的位置偏差,通过构建基准和三种互补指标揭示了领域与模型依赖的偏差特性,并分析了提示级缓解方法。

摘要:

多模态大语言模型(MLLMs)越来越多地用于视频理解,但它们在多视频输入下的可靠性仍知之甚少。我们研究了多视频摘要中的位置偏差,即每个视频摘要的质量可能随视频输入槽位的变化而变化,即使底层内容不变。我们从ActivityNet和新闻视频构建了一个基准,涵盖烹饪、家庭、休闲和新闻场景,包含两个和四个视频输入。我们评估了九个开源和专有MLLMs,并使用三种互补指标测量位置效应:覆盖率、方向性位置偏差(DPB)和中间边缘差距(MEG)。我们的结果表明,位置效应是领域和模型依赖的:即使中间位置表现不佳,有符号的方向性偏差也可能很小;增加视觉或生成预算并不能均匀地消除不平衡。我们进一步分析了提示级缓解方法。总之,结果表明多视频摘要仍然对输入协议和位置敏感,这促使开发更鲁棒的、顺序不变的多模态系统。

英文摘要:

Multimodal Large Language Models (MLLMs) are increasingly used for video understanding, yet their reliability under multi-video inputs remains poorly understood. We study positional bias in multi-video summarization, where the quality of a per-video summary can change with the video's input slot even when the underlying content is unchanged. We construct a benchmark from ActivityNet and News videos, covering Cooking, Domestic, Leisure, and News settings with two- and four-video inputs. We evaluate nine open-source and proprietary MLLMs and measure position effects with three complementary metrics: Coverage, Directional Positional Bias (DPB), and Middle-Edge Gap (MEG). Our results show that positional effects are domain- and model-dependent: signed directional bias can be small even when middle positions underperform, and increasing visual or generation budget does not uniformly remove the imbalance. We further analyze prompt-level mitigation methods. Together, the results show that multi-video summarization remains sensitive to input protocol and position, motivating more robust order-invariant multimodal systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑