CaptainCook4D: 一个用于理解程序性活动中错误的数据集
CaptainCook4D: A Dataset for Understanding Errors in Procedural Activities
AI总结：
该研究针对程序性活动错误理解的视觉推理难题，构建了含384段共94.5小时的第一视角4D食谱制作数据集CaptainCook4D，提供细粒度标注并设置三类任务基准。
摘要：
遵循分步流程是人们日常开展各类活动的核心组成部分。无论是组装家具还是制作食谱，这些流程都是指导框架，助力高效达成目标。然而，程序性活动的复杂性与时长特性本身会提升出错概率。从帧序列中理解这类程序性活动是一项颇具挑战的任务，需要精准解读视觉信息，并具备推理活动结构的能力。为此，我们采集了全新的第一视角4D数据集CaptainCook4D，包含384段（94.5小时）真实厨房环境下人们制作食谱的录制视频。该数据集涵盖两类截然不同的活动：一类是参与者严格遵循给定食谱说明的操作，另一类是参与者偏离流程、主动诱发错误的操作。我们提供了5300条步骤标注与1万条细粒度动作标注，并针对以下任务在该数据集上设置了基准测试：有监督错误识别、多步定位以及流程学习。
英文摘要：
Following step-by-step procedures is an essential component of various activities carried out by individuals in their daily lives. These procedures serve as a guiding framework that helps to achieve goals efficiently, whether it is assembling furniture or preparing a recipe. However, the complexity and duration of procedural activities inherently increase the likelihood of making errors. Understanding such procedural activities from a sequence of frames is a challenging task that demands an accurate interpretation of visual information and the ability to reason about the structure of the activity. To this end, we collect a new egocentric 4D dataset, CaptainCook4D, comprising 384 recordings (94.5 hours) of people performing recipes in real kitchen environments. This dataset consists of two distinct types of activity: one in which participants adhere to the provided recipe instructions and another in which they deviate and induce errors. We provide 5.3K step annotations and 10K fine-grained action annotations and benchmark the dataset for the following tasks: supervised error recognition, multistep localization, and procedure learning
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
