arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2101.11249cs.CVcs.HCcs.NEq-bio.NC
收藏

基于EEG和眼动追踪信号的高效视频摘要框架

Efficient Video Summarization Framework using EEG and Eye-tracking Signals

Indian Institute of Technology, Delhi(印度理工学院德里分校)

Sai Sukruth Bezugam, Swatilekha Majumdar, Chetan Ralekar, Tapan Kumar Gandhi

摘要:

本文提出了一种有效的视频摘要框架,将整个视频的要点,在几个关键帧或视频略读。现有的视频摘要框架基于利用计算机视觉低级特征提取或高级域级提取的算法。然而,作为总结视频的最终用户,人类仍然是最被忽视的方面。因此,本文考虑了人类在摘要中的作用,并引入了基于人类视觉注意力的摘要技术。为了了解人类的注意力行为,我们设计并进行了实验与人类参与者使用脑电图(EEG)和眼动追踪技术。从实验中获得的EEG和眼动追踪数据被同时处理,并用于从相当大的视频量中分割包含有用信息的帧。因此,帧分割主要依赖于人类的认知判断。使用我们的方法,视频总结了96.5%,同时保持更高的准确率和高召回率。与国家的最先进的技术的比较表明,所提出的方法产生天花板级的性能,降低计算成本,总结的视频。

英文摘要:

This paper proposes an efficient video summarization framework that will give a gist of the entire video in a few key-frames or video skims. Existing video summarization frameworks are based on algorithms that utilize computer vision low-level feature extraction or high-level domain level extraction. However, being the ultimate user of the summarized video, humans remain the most neglected aspect. Therefore, the proposed paper considers human's role in summarization and introduces human visual attention-based summarization techniques. To understand human attention behavior, we have designed and performed experiments with human participants using electroencephalogram (EEG) and eye-tracking technology. The EEG and eye-tracking data obtained from the experimentation are processed simultaneously and used to segment frames containing useful information from a considerable video volume. Thus, the frame segmentation primarily relies on the cognitive judgments of human beings. Using our approach, a video is summarized by 96.5% while maintaining higher precision and high recall factors. The comparison with the state-of-the-art techniques demonstrates that the proposed approach yields ceiling-level performance with reduced computational cost in summarising the videos.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑