用于教学视频中关键步骤识别的视频挖掘任务图
Video-Mined Task Graphs for Keystep Recognition in Instructional Videos
AI总结：
本文提出从教学视频中自动挖掘任务图以建模关键步骤间的概率关系，并利用该图正则化新视频中的关键步骤识别，在多个数据集上实现了超越现有方法的零样本定位与视频表示学习性能。
摘要：
程序性活动理解要求从更广泛的任务角度感知人类动作，其中多个关键步骤在一段长视频中按顺序执行以达到最终目标状态——例如食谱的步骤或DIY修理任务。先前的工作大多将关键步骤识别与这种更广泛的结构隔离开来，或者严格地将关键步骤限制为与预定义的顺序脚本对齐。我们提出从操作视频中自动发现任务图，以概率方式表示人们倾向于如何执行关键步骤，然后利用该图来规范新视频中的关键步骤识别。在多个真实世界教学视频数据集上，我们展示了其影响：更可靠的零样本关键步骤定位和改进的视频表示学习，超越了现有最先进水平。
英文摘要：
Procedural activity understanding requires perceiving human actions in terms of a broader task, where multiple keysteps are performed in sequence across a long video to reach a final goal state -- such as the steps of a recipe or a DIY fix-it task. Prior work largely treats keystep recognition in isolation of this broader structure, or else rigidly confines keysteps to align with a predefined sequential script. We propose discovering a task graph automatically from how-to videos to represent probabilistically how people tend to execute keysteps, and then leverage this graph to regularize keystep recognition in novel videos. On multiple datasets of real-world instructional videos, we show the impact: more reliable zero-shot keystep localization and improved video representation learning, exceeding the state of the art.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
