MovieTeller: 借助工具的电影概要与ID一致的渐进抽象
MovieTeller: Tool-augmented Movie Synopsis with ID Consistent Progressive Abstraction
AI总结：
MovieTeller通过工具增强的渐进抽象生成电影概要，解决传统VLM在长视频摘要中的身份一致性和叙事连贯性问题，提升事实准确性与一致性。
摘要：
随着数字娱乐的爆炸性增长，自动化视频摘要已成为内容索引、个性化推荐和高效媒体归档等应用中不可或缺的工具。自动为长视频（如电影和电视剧）生成摘要对现有视觉-语言模型（VLMs）构成了重大挑战。虽然这些通用模型在单图像描述上表现良好，但它们在长持续时间场景中常出现关键失败，主要问题在于缺乏ID一致的角色识别和破碎的叙事连贯性。为克服这些限制，我们提出MovieTeller，一种通过工具增强的渐进抽象生成电影概要的新框架。我们的核心贡献是免训练、工具增强、事实基础的生成过程。与需要昂贵模型微调不同，我们的框架直接利用现成模型进行即插即用操作。我们首先调用专门的面部识别模型作为外部工具，建立事实基础——精确的角色身份及其对应的边界框。这些基础随后被注入提示中，引导VLM的推理，确保生成的场景描述基于可验证的事实。此外，我们的渐进抽象流程将整部电影的摘要分解为多阶段过程，有效缓解了当前VLMs的上下文长度限制。实验表明，与端到端基线相比，我们的方法在事实准确性、角色一致性和整体叙事连贯性方面取得了显著提升。
英文摘要：
With the explosive growth of digital entertainment, automated video summarization has become indispensable for applications such as content indexing, personalized recommendation, and efficient media archiving. Automatic synopsis generation for long-form videos, such as movies and TV series, presents a significant challenge for existing Vision-Language Models (VLMs). While proficient at single-image captioning, these general-purpose models often exhibit critical failures in long-duration contexts, primarily a lack of ID-consistent character identification and a fractured narrative coherence. To overcome these limitations, we propose MovieTeller, a novel framework for generating movie synopses via tool-augmented progressive abstraction. Our core contribution is a training-free, tool-augmented, fact-grounded generation process. Instead of requiring costly model fine-tuning, our framework directly leverages off-the-shelf models in a plug-and-play manner. We first invoke a specialized face recognition model as an external "tool" to establish Factual Groundings--precise character identities and their corresponding bounding boxes. These groundings are then injected into the prompt to steer the VLM's reasoning, ensuring the generated scene descriptions are anchored to verifiable facts. Furthermore, our progressive abstraction pipeline decomposes the summarization of a full-length movie into a multi-stage process, effectively mitigating the context length limitations of current VLMs. Experiments demonstrate that our approach yields significant improvements in factual accuracy, character consistency, and overall narrative coherence compared to end-to-end baselines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
