使用基于文本的查询和条件建模的个性化视频摘要
Personalized Video Summarization using Text-Based Queries and Conditional Modeling
AI总结：
本论文提出一种结合文本查询与视觉信息的多模态深度学习方法，并引入条件建模与片段级伪标签技术，有效解决数据稀缺问题，生成更符合用户需求的个性化视频摘要。
摘要：
YouTube和Vimeo等平台上视频内容的激增给高效定位相关信息带来了重大挑战。自动视频摘要旨在通过以浓缩形式提取和呈现关键内容来解决这一问题。本论文探讨了通过整合基于文本的查询和条件建模来增强视频摘要，以根据用户需求定制摘要。传统方法通常生成固定摘要，可能不符合个人需求。为克服这一点，我们提出了一种多模态深度学习方法，结合文本查询和视觉信息，并在模型架构的不同层次进行融合。使用准确率和F1分数等评估指标来评估生成摘要的质量。论文还研究了利用上下文词嵌入和专门的注意力网络来改进基于文本的查询表示。这增强了对查询的语义理解，从而产生更好的视频摘要。为了模拟既考虑视觉连贯性又考虑故事情节一致性等抽象因素的人类摘要方式，我们引入了一种条件建模方法。该方法使用多个随机变量和联合分布来捕获关键摘要组件，从而生成更像人类且可解释的摘要。针对全监督学习中的数据稀缺问题，论文提出了一种片段级伪标签方法。这种自监督方法生成额外数据，即使在人工标注数据集有限的情况下也能提升模型性能。总之，本研究旨在通过纳入基于文本的查询、改进查询表示、引入条件建模以及解决数据稀缺问题，来增强自动视频摘要，从而创建更有效和个性化的视频摘要。
英文摘要：
The proliferation of video content on platforms like YouTube and Vimeo presents significant challenges in efficiently locating relevant information. Automatic video summarization aims to address this by extracting and presenting key content in a condensed form. This thesis explores enhancing video summarization by integrating text-based queries and conditional modeling to tailor summaries to user needs. Traditional methods often produce fixed summaries that may not align with individual requirements. To overcome this, we propose a multi-modal deep learning approach that incorporates both textual queries and visual information, fusing them at different levels of the model architecture. Evaluation metrics such as accuracy and F1-score assess the quality of the generated summaries. The thesis also investigates improving text-based query representations using contextualized word embeddings and specialized attention networks. This enhances the semantic understanding of queries, leading to better video summaries. To emulate human-like summarization, which accounts for both visual coherence and abstract factors like storyline consistency, we introduce a conditional modeling approach. This method uses multiple random variables and joint distributions to capture key summarization components, resulting in more human-like and explainable summaries. Addressing data scarcity in fully supervised learning, the thesis proposes a segment-level pseudo-labeling approach. This self-supervised method generates additional data, improving model performance even with limited human-labeled datasets. In summary, this research aims to enhance automatic video summarization by incorporating text-based queries, improving query representations, introducing conditional modeling, and addressing data scarcity, thereby creating more effective and personalized video summaries.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
