Lipper：使用多视图唇读合成您的语音
Lipper: Synthesizing Thy Speech using Multi-View Lipreading
摘要：
唇读在监控和视频会议等领域有着广泛的应用前景。尽管如此，构建唇读系统的大部分工作仅限于将无声视频分类为表示文本短语的类。然而，唇读作为一种基于文本的分类任务存在着许多问题，比如它依赖于特定的语言和词汇映射。因此，在本文中，我们提出了一个多视角唇读音频系统，即Lipper，它的模型作为一个回归任务。该模型将无声视频作为输入，并产生语音作为输出。对于多视图无声视频，我们观察到单视图语音重建结果的改进。我们展示了这一点，提出了一套详尽的实验依赖于扬声器，词汇和扬声器独立的设置。此外，我们比较的延迟值的利珀与其他语音阅读系统，以显示实时性的音频产生。我们还对制作的音频进行了用户研究，以了解使用理柏制作的音频的可理解性水平。
英文摘要：
Lipreading has a lot of potential applications such as in the domain of surveillance and video conferencing. Despite this, most of the work in building lipreading systems has been limited to classifying silent videos into classes representing text phrases. However, there are multiple problems associated with making lipreading a text-based classification task like its dependence on a particular language and vocabulary mapping. Thus, in this paper we propose a multi-view lipreading to audio system, namely Lipper, which models it as a regression task. The model takes silent videos as input and produces speech as the output. With multi-view silent videos, we observe an improvement over single-view speech reconstruction results. We show this by presenting an exhaustive set of experiments for speaker-dependent, out-of-vocabulary and speaker-independent settings. Further, we compare the delay values of Lipper with other speechreading systems in order to show the real-time nature of audio produced. We also perform a user study for the audios produced in order to understand the level of comprehensibility of audios produced using Lipper.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
