arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2302.09173cs.AIcs.CLcs.LG
收藏

从教学视频转录文本中无监督生成任务图

Unsupervised Task Graph Generation from Instructional Video Transcripts

LG AI Research(LG人工智能研究院);University of Michigan, Ann Arbor(密歇根大学安娜堡分校)

Lajanugen Logeswaran, Sungryull Sohn, Yunseok Jang, Moontae Lee, Honglak Lee

AI总结:

本研究针对教学视频转录文本的真实活动任务图生成问题,提出结合指令调优语言模型推理与聚类排序的无监督方法,在ProceL、CrossTask数据集上表现优于有监督方法。

摘要:

本研究探讨真实世界活动的任务图生成问题。与现有建模方式不同,我们考虑的场景是:给定执行真实世界活动(例如制作咖啡)的教学视频的文本转录内容,目标是识别与任务相关的关键步骤,以及这些关键步骤之间的依赖关系。我们提出一种新颖的任务图生成方法,该方法结合了指令调优语言模型的推理能力,以及聚类和排序组件,以完全无监督的方式生成准确的任务图。我们在ProceL和CrossTask数据集的任务上开展实验,结果表明,与有监督学习方法相比,所提方法能生成更准确的任务图。

英文摘要:

This work explores the problem of generating task graphs of real-world activities. Different from prior formulations, we consider a setting where text transcripts of instructional videos performing a real-world activity (e.g., making coffee) are provided and the goal is to identify the key steps relevant to the task as well as the dependency relationship between these key steps. We propose a novel task graph generation approach that combines the reasoning capabilities of instruction-tuned language models along with clustering and ranking components to generate accurate task graphs in a completely unsupervised manner. We show that the proposed approach generates more accurate task graphs compared to a supervised learning approach on tasks from the ProceL and CrossTask datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑