arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2109.14744cs.CV
收藏

手边的物体:从手-物体交互中自动编辑混合现实视频指导

The Object at Hand: Automated Editing for Mixed Reality Video Guidance from Hand-Object Interactions

University of Bristol(布里斯托大学)

Yao Lu, Walterio W. Mayol-Cuevas

摘要:

在本文中,我们关注的问题,如何自动提取的步骤,组成现实生活中的手部活动。这是在混合现实系统中处理、监控和提供视频指导的关键能力。我们使用以自我为中心的视觉来观察现实任务中的手-物体交互,并自动将视频分解为其组成步骤。我们的方法结合了手对象交互(HOI)检测,对象相似性测量和有限状态机(FSM)表示自动编辑视频的步骤。我们使用卷积神经网络(CNN)和FSM的组合来发现,编辑剪切和合并片段,同时观察真实的手部活动。我们定量和定性评估我们的算法在两个数据集:GTEA \cite{li2015delving},和一个新的数据集,我们介绍了中国茶叶制作。实验结果表明,该方法能够以较高的精度将手-物交互视频分割成关键步骤段。

英文摘要:

In this paper, we concern with the problem of how to automatically extract the steps that compose real-life hand activities. This is a key competence towards processing, monitoring and providing video guidance in Mixed Reality systems. We use egocentric vision to observe hand-object interactions in real-world tasks and automatically decompose a video into its constituent steps. Our approach combines hand-object interaction (HOI) detection, object similarity measurement and a finite state machine (FSM) representation to automatically edit videos into steps. We use a combination of Convolutional Neural Networks (CNNs) and the FSM to discover, edit cuts and merge segments while observing real hand activities. We evaluate quantitatively and qualitatively our algorithm on two datasets: the GTEA\cite{li2015delving}, and a new dataset we introduce for Chinese Tea making. Results show our method is able to segment hand-object interaction videos into key step segments with high levels of precision.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑