ESTANet: 通过预测不一致性实现程序性视频中的高效在线错误检测
ESTANet: Efficient Online Error Detection in Procedural Videos via Prediction Inconsistency
AI总结:
提出ESTANet轻量框架,利用多个动作检测器在正确与错误执行时的预测不一致性,通过多数投票实现实时在线错误检测,在三个数据集上达到最优性能。
摘要:
一个高效准确的程序性任务错误检测系统对于支持人类日常生活中的需求至关重要,因为它可以提供即时通知并指导人们纠正错误。在这项工作中,我们从简单但被忽视的角度研究程序性视频中的实时在线错误检测:动作检测器本身的预测行为。我们没有设计复杂的架构或专门的监督,而是观察到动作检测器根据其对输入动态和时间上下文的敏感性自然表现出不同的预测特性。因此,我们提出了ESTANet(错误敏感和时间变化网络),这是一个轻量级框架,通过利用一小部分动作检测器产生的动作预测之间的不一致性来检测错误。我们构建了标准和错误敏感的动作检测器,它们在正确执行时表现相似,但在错误发生时反应不同。同时,当程序偏离预期序列时,具有不同时间上下文的检测器进一步放大了预测不一致性。在推理过程中,我们通过多数投票聚合标准和错误敏感预测之间的不匹配来检测错误,从而标记包含错误的帧。在EgoPER、Assembly-101-O和EPIC-Tent-O上的大量实验表明,ESTANet在在线错误检测中实现了最先进的性能,同时通过轻量级架构保持了实时效率。我们的结果强调,利用动作检测器的内在属性可以在不增加架构设计复杂性的情况下为在线错误检测提供强大且实用的解决方案。
英文摘要:
An efficient and accurate system for detecting errors in procedural tasks is crucial for supporting human needs in daily life, as it can provide instant notifications and guide people to correct mistakes. In this work, we study real-time online error detection in procedural videos from a simple but overlooked perspective: the prediction behavior of action detectors themselves. Instead of designing complex architectures or specialized supervision, we observe that action detectors naturally exhibit different prediction characteristics depending on their sensitivity to input dynamics and temporal context. We therefore propose ESTANet (Error-Sensitive and Temporally-vArying Network), a lightweight framework that detects errors by exploiting inconsistencies among action predictions produced by a small set of action detectors. We construct standard and error-sensitive action detectors that behave similarly on correct executions but respond differently when errors occur. Meanwhile, detectors operating with different temporal contexts further amplify prediction inconsistencies when the procedure deviates from the intended sequence. During inference, we detect errors by aggregating mismatches between standard and error-sensitive predictions through majority voting to flag frames that contain errors. Extensive experiments on EgoPER, Assembly-101-O, and EPIC-Tent-O demonstrate that ESTANet achieves state-of-the-art performance in online error detection while maintaining real-time efficiency with a lightweight architecture. Our results highlight that leveraging the intrinsic properties of action detectors can yield a powerful and practical solution for online error detection without increasing architectural design complexity.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。