EgoOops：一个面向参照流程文本的第一人称视频错误动作检测数据集
EgoOops: A Dataset for Mistake Action Detection from Egocentric Videos referring to Procedural Texts
AI总结：
针对现有错误检测研究缺乏流程文本支撑、相关数据集覆盖领域有限的问题，本文提出跨领域第一人称错误动作检测数据集EgoOops及结合视频文本对齐与标签分类的检测方法，验证了流程文本对错误检测的重要性。
摘要：
错误动作检测对于开发可检测工人失误并提供反馈的智能档案系统至关重要。现有研究聚焦于自由风格活动中视觉上明显的错误，由此产生了仅基于视频的错误检测方法。然而，在遵循文本的活动中，模型不参照文本就无法判断部分动作的正确性。此外，当前除烹饪领域外，错误数据集很少在视频录制时使用流程文本。为填补这些空白，本文提出EgoOops数据集，其中第一人称视频记录了跨多个领域遵循流程文本时的错误活动。该数据集具备三类标注：视频-文本对齐、错误标签以及错误描述。我们还提出了一种错误检测方法，结合视频-文本对齐与错误标签分类以利用文本信息。我们的实验结果表明，纳入流程文本对错误检测至关重要。数据可通过https://y-haneji.github.io/EgoOops-project-page/获取。
英文摘要：
Mistake action detection is crucial for developing intelligent archives that detect workers' errors and provide feedback. Existing studies have focused on visually apparent mistakes in free-style activities, resulting in video-only approaches to mistake detection. However, in text-following activities, models cannot determine the correctness of some actions without referring to the texts. Additionally, current mistake datasets rarely use procedural texts for video recording except for cooking. To fill these gaps, this paper proposes the EgoOops dataset, where egocentric videos record erroneous activities when following procedural texts across diverse domains. It features three types of annotations: video-text alignment, mistake labels, and descriptions for mistakes. We also propose a mistake detection approach, combining video-text alignment and mistake label classification to leverage the texts. Our experimental results show that incorporating procedural texts is essential for mistake detection. Data is available through https://y-haneji.github.io/EgoOops-project-page/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
