教学视频中的开放事件过程规划
Open-Event Procedure Planning in Instructional Videos
AI总结：
本文提出开放事件过程规划任务 OEPP 与 OpenEvent 基准，通过基础/新事件划分检验过程知识迁移，并给出通用框架和基线分析。
摘要：
给定当前视觉观察，教学视频中的传统过程规划任务要求模型在给定动作空间内生成目标导向的计划。以往所有针对该任务的方法都在相同动作空间下进行训练和推理，并且只能为训练集中预定义的事件进行规划。我们认为这一设定不适用于现实生活中的人类辅助，并旨在提出一种更通用、更实际的规划范式。具体而言，本文引入了一个名为开放事件过程规划（Open-event Procedure Planning，OEPP）的新任务，将传统过程规划扩展到开放事件设定。OEPP旨在验证规划器能否将学到的知识迁移到训练期间未曾见过的类似事件。我们基于现有数据集为该任务重建了一个新的 OpenEvent 基准，并将涉及的事件划分为基础部分和新事件部分。在数据收集过程中，我们通过多个阶段评估不同事件步骤描述之间的相似性，仔细确保基础事件和新事件之间过程知识的可迁移性。基于所收集的数据，我们进一步提出了一个专为 OEPP 设计的简单且通用的框架，并使用多种基线方法开展了广泛研究，针对该任务的结果提供了详细且富有洞见的分析。
英文摘要：
Given the current visual observations, the traditional procedure planning task in instructional videos requires a model to generate goal-directed plans within a given action space. All previous methods for this task conduct training and inference under the same action space, and they can only plan for pre-defined events in the training set. We argue this setting is not applicable for human assistance in real lives and aim to propose a more general and practical planning paradigm. Specifically, in this paper, we introduce a new task named Open-event Procedure Planning (OEPP), which extends the traditional procedure planning to the open-event setting. OEPP aims to verify whether a planner can transfer the learned knowledge to similar events that have not been seen during training. We rebuild a new benchmark of OpenEvent for this task based on existing datasets and divide the events involved into base and novel parts. During the data collection process, we carefully ensure the transfer ability of procedural knowledge for base and novel events by evaluating the similarity between the descriptions of different event steps with multiple stages. Based on the collected data, we further propose a simple and general framework specifically designed for OEPP, and conduct extensive study with various baseline methods, providing a detailed and insightful analysis on the results for this task.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
