基于智能体的视频剪辑（Agent-based Video Trimming）
Agent-based Video Trimming
AI总结：
针对用户生成视频冗余度高、现有片段选取方法忽略片段关联与排序的问题，提出三阶段AVT方法实现连贯叙事的视频剪辑，构建新基准数据集，在多数据集及用户研究中表现更优。
摘要：
随着信息获取愈发便捷，用户生成视频的时长不断增加，给观众从海量内容中筛选有价值信息带来了负担。这一趋势凸显了开发高效提取视频关键信息算法的必要性。尽管高光检测、时刻检索和视频摘要领域已取得显著进展，但现有方法主要聚焦于选取特定时间区间，往往忽略了片段间的关联性以及片段排序的潜力。本文提出了一项名为Video Trimming（VT，视频剪辑）的新任务，核心是检测冗余片段、筛选有价值片段，并将其组合成具有连贯叙事性的最终视频。为解决该任务，我们提出了Agent-based Video Trimming（AVT，基于智能体的视频剪辑）方法，分为三个阶段：视频结构化、片段过滤和叙事构建。具体而言，我们采用Video Captioning Agent（视频字幕生成智能体）将视频切片转换为结构化文本描述，通过过滤模块根据每个片段的结构化信息动态丢弃低质量片段，再由Video Arrangement Agent（视频编排智能体）选取有效片段并整合成连贯的最终叙事。在评估方面，我们开发了Video Evaluation Agent（视频评估智能体）对剪辑后的视频进行评估，并与人工评估同步开展。此外，我们利用互联网上的原始用户视频，构建了一个用于视频剪辑任务的新基准数据集。实验结果表明，AVT在用户研究中获得了更优的评价，并且在YouTube Highlights、TVSum以及我们自建的数据集上，高光检测任务的mAP和精度均表现更出色。代码和模型可访问https://ylingfeng.github.io/AVT获取。
英文摘要：
As information becomes more accessible, user-generated videos are increasing in length, placing a burden on viewers to sift through vast content for valuable insights. This trend underscores the need for an algorithm to extract key video information efficiently. Despite significant advancements in highlight detection, moment retrieval, and video summarization, current approaches primarily focus on selecting specific time intervals, often overlooking the relevance between segments and the potential for segment arranging. In this paper, we introduce a novel task called Video Trimming (VT), which focuses on detecting wasted footage, selecting valuable segments, and composing them into a final video with a coherent story. To address this task, we propose Agent-based Video Trimming (AVT), structured into three phases: Video Structuring, Clip Filtering, and Story Composition. Specifically, we employ a Video Captioning Agent to convert video slices into structured textual descriptions, a Filtering Module to dynamically discard low-quality footage based on the structured information of each clip, and a Video Arrangement Agent to select and compile valid clips into a coherent final narrative. For evaluation, we develop a Video Evaluation Agent to assess trimmed videos, conducting assessments in parallel with human evaluations. Additionally, we curate a new benchmark dataset for video trimming using raw user videos from the internet. As a result, AVT received more favorable evaluations in user studies and demonstrated superior mAP and precision on the YouTube Highlights, TVSum, and our own dataset for the highlight detection task. The code and models are available at https://ylingfeng.github.io/AVT.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
