X-LeBench：一个用于极长第一人称视频理解的基准数据集
X-LeBench: A Benchmark for Extremely Long Egocentric Video Understanding
AI总结：
X-LeBench通过模拟真实日常生活场景，构建了首个极长第一人称视频理解基准数据集，揭示了长视频理解中的关键挑战。
摘要：
长形式的第一人称视频理解提供了丰富的上下文信息和对长期人类行为的独特见解，具有在具身智能、长期活动分析和个性化辅助技术中的重大应用潜力。然而，现有基准数据集主要关注单个、短（例如，分钟到几十分钟）到中等长度的视频，留下了在评估极长的第一人称视频记录方面的显著空白。为此，我们引入了X-LeBench，一个全新的基准数据集，通过专注于需要全面理解极长第一人称视频记录的任务来填补这一空白。我们的X-LeBench开发了一条生活日程模拟管道，生成与现实世界视频数据一致的现实、连贯的日程。这种方法使得合成日程与现实世界 footage 从Ego4D大规模第一人称视频数据集中的广泛日常生活场景相结合，从而产生432个模拟视频日志，从23分钟到16.4小时不等。对几种基线系统和多模态大语言模型（MLLMs）的评估揭示了它们在各个方面表现不佳，突显了长形式第一人称视频理解固有的挑战，如时间定位和推理、上下文聚合和记忆保持，并强调了需要更先进的模型。
英文摘要：
Long-form egocentric video understanding provides rich contextual information and unique insights into long-term human behaviors, holding significant potential for applications in embodied intelligence, long-term activity analysis, and personalized assistive technologies. However, existing benchmark datasets primarily focus on single, short (\eg, minutes to tens of minutes) to moderately long videos, leaving a substantial gap in evaluating extensive, ultra-long egocentric video recordings. To address this, we introduce X-LeBench, a novel benchmark dataset meticulously designed to fill this gap by focusing on tasks requiring a comprehensive understanding of extremely long egocentric video recordings. Our X-LeBench develops a life-logging simulation pipeline that produces realistic, coherent daily plans aligned with real-world video data. This approach enables the flexible integration of synthetic daily plans with real-world footage from Ego4D-a massive-scale egocentric video dataset covers a wide range of daily life scenarios-resulting in 432 simulated video life logs spanning from 23 minutes to 16.4 hours. The evaluations of several baseline systems and multimodal large language models (MLLMs) reveal their poor performance across the board, highlighting the inherent challenges of long-form egocentric video understanding, such as temporal localization and reasoning, context aggregation, and memory retention, and underscoring the need for more advanced models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
