MESH——像人类一样理解视频：大型视频模型中的幻觉测量
MESH -- Understanding Videos Like Human: Measuring Hallucinations in Large Video Models
AI总结：
针对大型视频模型易产生幻觉的问题，提出基准MESH，采用问答框架自下而上评估对象、特征和动作对，发现模型在处理细节和长视频多主体动作时幻觉显著增加。
摘要：
大型视频模型（LVMs）建立在大型语言模型（LLMs）和视觉模块的语义能力之上，通过整合时间信息来更好地理解动态视频内容。尽管取得了进展，LVMs 容易产生幻觉——生成不准确或无关的描述。当前的视频幻觉基准严重依赖对视频内容的人工分类，忽视了人类自然解读视频时所依赖的基于感知的过程。我们引入了 MESH，一个旨在系统评估 LVMs 中幻觉的基准。MESH 采用问答框架，包含二元和多项选择格式，并纳入目标实例和陷阱实例。它遵循自下而上的方法，评估基本对象、从粗到细的主体特征以及主体-动作对，与人类视频理解方式对齐。我们证明 MESH 为识别视频中的幻觉提供了一种有效且全面的方法。我们的评估显示，尽管 LVMs 在识别基本对象和特征方面表现出色，但在处理细节或对齐较长视频中涉及多个主体的多个动作时，它们产生幻觉的倾向显著增加。
英文摘要：
Large Video Models (LVMs) build on the semantic capabilities of Large Language Models (LLMs) and vision modules by integrating temporal information to better understand dynamic video content. Despite their progress, LVMs are prone to hallucinations-producing inaccurate or irrelevant descriptions. Current benchmarks for video hallucination depend heavily on manual categorization of video content, neglecting the perception-based processes through which humans naturally interpret videos. We introduce MESH, a benchmark designed to evaluate hallucinations in LVMs systematically. MESH uses a Question-Answering framework with binary and multi-choice formats incorporating target and trap instances. It follows a bottom-up approach, evaluating basic objects, coarse-to-fine subject features, and subject-action pairs, aligning with human video understanding. We demonstrate that MESH offers an effective and comprehensive approach for identifying hallucinations in videos. Our evaluations show that while LVMs excel at recognizing basic objects and features, their susceptibility to hallucinations increases markedly when handling fine details or aligning multiple actions involving various subjects in longer videos.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
