PROVIA：面向第一视角视频在线错误检测的程序状态跟踪
PROVIA: Procedure State Tracking for Online Mistake Detection in Egocentric Videos
- Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）
- Hunan University（湖南大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
提出PROVIA，一种通过区分事实状态与接受进度来跟踪程序状态、在线检测第一视角视频中错误的方法，在多个基准上优于仅依赖时间的基线。
AI中文摘要：
一个观察第一视角视频的助手应当仅从过去的帧中注意到错误，在下一步开始之前，并在操作者恢复后继续工作。错误会改变工作状态，因此后续每一步都必须根据已完成的操作而非计划来解读。当前在线方法报告的首个错误协议会在每个记录的第一个错误处截断，因此一个从不查看视频的固定时间规则在每个案例上都是正确的。我们在完整的试验中进行评估，其中错误和恢复自然发生，在验证误报预算下，并与仅使用时间信息的对照组进行比较。PROVIA 区分两种记录：事实状态，即每个操作者所执行步骤的学习摘要（包括错误），以及已接受的进度，即由贝叶斯状态合并从正确演示中归纳出的自动机状态上的精确后验分布，以及每个操作者的执行状态。程序状态转换仅发生在正确状态分支中；错误和纠正分支保留源状态。一个序贯检验将逐帧的错误概率转化为警报。凭借一个过滤器和一条优化规则，PROVIA 在 CaptainCook4D、IndustReal、HoloAssist 和 IMPACT-ego 上，在评估的受控基线中排名最佳。在每分钟 0.1 次误报的验证预算下，它在 CaptainCook4D 上召回率为 .154 对比 .128，在 HoloAssist 上为 .034 对比 .015，并在每个预算下均领先。该流程以每秒 58-70 帧的速度运行。源代码可在以下 https URL 获取。
英文摘要：
An assistant watching egocentric video should notice a mistake from past frames alone, before the next step begins, and keep working once the person recovers. A mistake changes the state of the work, so every later step has to be read against what was done rather than against the plan. The first-mistake protocol that current online methods report on cuts each recording at its first mistake, so a fixed-time rule that never looks at the video is right on every case. We evaluate on complete trials, where mistakes and recoveries arise naturally, under a validation false-alarm budget and against controls that use timing alone. PROVIA keeps two records apart: a factual state, a learned summary of the steps each actor performed, mistakes included, and the accepted progress, an exact posterior over the state of an automaton induced from correct demonstrations by Bayesian state merging and over the execution status of each actor. Procedure-state transitions occur only in the correct-status branch; the mistake and correction branches retain the source state. A sequential test turns the per-frame mistake probability into alarms. With one filter and one optimization rule, PROVIA ranks mistakes best among the evaluated controlled baselines on CaptainCook4D, IndustReal, HoloAssist and IMPACT-ego. At a validation budget of 0.1 false alarms per minute it recalls .154 against .128 on CaptainCook4D and .034 against .015 on HoloAssist, where it leads at every budget. The pipeline runs at 58-70 frames per second. The source code is available at https://github.com/Kratos-Wen/PROVIA.
