基于两阶段前缀增强多模态大语言模型的电影事件导向归因生成
Generating Event-oriented Attribution for Movies via Two-Stage Prefix-Enhanced Multimodal LLM
AI总结：
针对现有电影事件研究缺乏全片因果分析、多模态大模型受上下文长度限制的问题，提出两阶段前缀增强TSPE方法，通过局部分段总结与全局关联事件聚焦实现精准事件归因，性能优于SOTA方法。
摘要：
社交媒体平台的繁荣催生了对富语义服务（如事件与剧情归因）的迫切需求。然而，现有研究大多聚焦于片段级事件理解，主要通过基础字幕任务实现，并未针对整部电影分析事件的成因。这是一项重大挑战，因为即便是先进的多模态大语言模型（MLLMs），也会受限于上下文长度，难以处理海量多模态信息。为解决该问题，我们提出一种用于电影视频事件归因（即关联事件与其因果语义建立联系）的两阶段前缀增强MLLM（TSPE）方法。在局部阶段，我们引入交互感知前缀，引导模型关注单个片段内的相关多模态信息，对单个事件进行简要总结。相应地，在全局阶段，我们利用推理知识图谱强化关联事件间的联系，并设计事件感知前缀，引导模型聚焦关联事件而非所有前置片段，从而实现精准的事件归因。在两个真实数据集上的综合评估表明，我们的框架性能优于现有最优方法。
英文摘要：
The prosperity of social media platforms has raised the urgent demand for semantic-rich services, e.g., event and storyline attribution. However, most existing research focuses on clip-level event understanding, primarily through basic captioning tasks, without analyzing the causes of events across an entire movie. This is a significant challenge, as even advanced multimodal large language models (MLLMs) struggle with extensive multimodal information due to limited context length. To address this issue, we propose a Two-Stage Prefix-Enhanced MLLM (TSPE) approach for event attribution, i.e., connecting associated events with their causal semantics, in movie videos. In the local stage, we introduce an interaction-aware prefix that guides the model to focus on the relevant multimodal information within a single clip, briefly summarizing the single event. Correspondingly, in the global stage, we strengthen the connections between associated events using an inferential knowledge graph, and design an event-aware prefix that directs the model to focus on associated events rather than all preceding clips, resulting in accurate event attribution. Comprehensive evaluations of two real-world datasets demonstrate that our framework outperforms state-of-the-art methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
