Prompt2LVideos：探索用于理解长时程多模态视频的提示
Prompt2LVideos: Exploring Prompts for Understanding Long-Form Multimodal Videos
AI总结：
本文提出Prompt2LVideos数据集，包含长时程讲座与新闻视频，并通过基线方法分析现有模型局限，倡导利用提示工程结合ASR和OCR来全面理解长时程多模态视频。
摘要：
学习多模态视频理解通常依赖于由视频片段与人工标注字幕配对组成的数据集。然而，当处理教育和新闻领域中从几分钟到几小时不等的长时程视频时，由于需要更多具备学科专业知识的人工标注者，这一任务变得更加困难。因此，亟需自动化解决方案。大语言模型（LLMs）的最新进展有望通过利用自动语音识别（ASR）和光学字符识别（OCR）技术，捕获简洁且信息丰富的内容，从而理解整个视频。ASR从音频中提供文本内容，而OCR从特定帧中提取文本内容。本文引入了一个包含长时程讲座和新闻视频的数据集。我们提出了基线方法来理解其在该数据集上的局限性，并倡导探索提示工程技术以全面理解长时程多模态视频数据集。
英文摘要：
Learning multimodal video understanding typically relies on datasets comprising video clips paired with manually annotated captions. However, this becomes even more challenging when dealing with long-form videos, lasting from minutes to hours, in educational and news domains due to the need for more annotators with subject expertise. Hence, there arises a need for automated solutions. Recent advancements in Large Language Models (LLMs) promise to capture concise and informative content that allows the comprehension of entire videos by leveraging Automatic Speech Recognition (ASR) and Optical Character Recognition (OCR) technologies. ASR provides textual content from audio, while OCR extracts textual content from specific frames. This paper introduces a dataset comprising long-form lectures and news videos. We present baseline approaches to understand their limitations on this dataset and advocate for exploring prompt engineering techniques to comprehend long-form multimodal video datasets comprehensively.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
