arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1801.09436cs.CV
收藏

本地可视麦克风:改进无声视频的声音提取

Local Visual Microphones: Improved Sound Extraction from Silent Video

Sharif University of Technology(谢里夫理工大学);University of Tehran(德黑兰大学)

Mohammad Amin Shabani, Laleh Samadfam, Mohammad Amin Sadeghi

摘要:

声波引起附近物体的微小振动。文献中存在一些可以从视频中提取声音的技术。在本文中,我们研究在不同的图像位置的局部振动模式。我们表明,在图像中的不同位置振动不同。我们仔细地聚合局部振动,并产生一个声音质量,提高国家的最先进的。我们表明,局部振动可能有一个时间延迟,因为声波需要时间在空气中传播。我们利用这种现象来估计声音的方向。我们还提出了一种新的算法,加快了两到三个数量级的声音提取,并在20KHz的视频达到实时性能。

英文摘要:

Sound waves cause small vibrations in nearby objects. A few techniques exist in the literature that can extract sound from video. In this paper we study local vibration patterns at different image locations. We show that different locations in the image vibrate differently. We carefully aggregate local vibrations and produce a sound quality that improves state-of-the-art. We show that local vibrations could have a time delay because sound waves take time to travel through the air. We use this phenomenon to estimate sound direction. We also present a novel algorithm that speeds up sound extraction by two to three orders of magnitude and reaches real-time performance in a 20KHz video.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑