利用程序性知识与任务层级实现高效的教学视频预训练
Leveraging Procedural Knowledge and Task Hierarchies for Efficient Instructional Video Pre-training
AI总结：
该论文提出利用任务层级和程序性步骤作为先验知识，预训练模型Pivot以高效识别和预测教学视频中的任务与步骤，在数据与计算受限时优于基线，实现高效视频推荐。
摘要：
教学视频为学习新任务（如烹饪食谱或组装家具）提供了一种便捷的方式。观看者希望找到既反映其感兴趣的整体任务，又包含执行该任务所需相关步骤的相应视频。为实现这一点，教学视频模型应能够推断输入视频中发生的任务和步骤。在计算资源或用于训练该模型的相关视频主题有限的情况下，高效且可泛化地完成这一任务至关重要。为满足这些要求，我们明确挖掘了与教学视频相关的任务层级和程序性步骤。我们利用这些先验知识对模型$\ exttt{Pivot}$进行预训练，用于步骤和任务预测。在预训练期间，我们还提供了视频增强和早停策略，以最优地确定用于下游任务的模型。我们在两个下游数据集上测试了该预训练模型在任务识别、步骤识别和步骤预测方面的性能。当预训练数据和计算资源有限时，我们在这些任务上优于之前的基线方法。因此，利用先前的任务和步骤结构能够实现$\ exttt{Pivot}$的高效训练，用于教学视频推荐。
英文摘要：
Instructional videos provide a convenient modality to learn new tasks (ex. cooking a recipe, or assembling furniture). A viewer will want to find a corresponding video that reflects both the overall task they are interested in as well as contains the relevant steps they need to carry out the task. To perform this, an instructional video model should be capable of inferring both the tasks and the steps that occur in an input video. Doing this efficiently and in a generalizable fashion is key when compute or relevant video topics used to train this model are limited. To address these requirements we explicitly mine task hierarchies and the procedural steps associated with instructional videos. We use this prior knowledge to pre-train our model, $\texttt{Pivot}$, for step and task prediction. During pre-training, we also provide video augmentation and early stopping strategies to optimally identify which model to use for downstream tasks. We test this pre-trained model on task recognition, step recognition, and step prediction tasks on two downstream datasets. When pre-training data and compute are limited, we outperform previous baselines along these tasks. Therefore, leveraging prior task and step structures enables efficient training of $\texttt{Pivot}$ for instructional video recommendation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
