错误归因：第一性错误理解在第一人称视频中
Mistake Attribution: Fine-Grained Mistake Understanding in Egocentric Videos
AI总结：
本文提出MATT任务，通过细粒度分析第一人称视频中的人类错误，开发MisEngine数据引擎生成带有归因注释的错误样本，并提出MisFormer模型在语义、时间和空间维度上实现统一的错误归因，实验表明其在视频语言理解等任务上优于现有方法。
摘要：
我们引入Mistake Attribution (MATT)，一种新的任务，用于细粒度理解第一人称视频中的人类错误。尽管先前工作检测是否发生错误，MATT归因于违反指令的哪一部分（语义角色），在视频中偏差变得不可逆的时间点（不可逆点，PNR），以及错误出现在PNR帧中的位置。我们开发了MisEngine，一个自动从现有数据集中构建错误样本的数据引擎，带有丰富的归因注释。应用于大规模第一人称语料库，MisEngine生成EPIC-KITCHENS-M和Ego4D-M两个数据集，规模比先前的错误数据集大两个数量级。然后我们提出MisFormer，一个统一的基于注意力的模型，用于跨语义、时间和空间维度的错误归因，通过MisEngine监督进行训练。一项人类研究验证了我们构建的错误样本的生态效度，确认EPIC-KITCHENS-M和Ego4D-M可以作为可靠的错误理解基准。在我们数据集和先前基准上的实验表明，MisFormer作为单一统一模型，在视频语言理解、时间定位、手-物体交互和错误检测任务上，分别优于任务特定的SOTA方法至少6.66%、21.81%、18.7%和3.00%。项目页面：https://yayuanli.github.io/MATT/
英文摘要：
We introduce Mistake Attribution (MATT), a new task for fine-grained understanding of human mistakes in egocentric videos. While prior work detects whether a mistake occurs, MATT attributes the mistake to what part of the instruction is violated (semantic role), when in the video the deviation becomes irreversible (the Point-of-No-Return, PNR), and where the mistake appears in the PNR frame. We develop MisEngine, a data engine that automatically constructs mistake samples from existing datasets with attribution-rich annotations. Applied to large egocentric corpora, MisEngine yields EPIC-KITCHENS-M and Ego4D-M -- two datasets up to two orders of magnitude larger than prior mistake datasets. We then present MisFormer, a unified attention-based model for mistake attribution across semantic, temporal, and spatial dimensions, trained with MisEngine supervision. A human study demonstrates the ecological validity of our MisEngine-constructed mistake samples, confirming that EPIC-KITCHENS-M and Ego4D-M can serve as reliable benchmarks for mistake understanding. Experiments on both our datasets and prior benchmarks show that MisFormer, as a single unified model, outperforms task-specific SOTA methods by at least 6.66%, 21.81%, 18.7%, and 3.00% in video-language understanding, temporal localization, hand-object interaction, and mistake detection, respectively. Project page: https://yayuanli.github.io/MATT/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
