基于多模态信息的语音处理（MISP）2022挑战赛：视听日记和识别
The Multimodal Information based Speech Processing (MISP) 2022 Challenge: Audio-Visual Diarization and Recognition
摘要：
多模态基于信息的语音处理（MISP）挑战赛旨在通过促进唤醒词、说话人日志化、语音识别等技术的研究，扩展信号处理技术在特定场景中的应用。MISP 2022挑战赛有两个轨道：1）视听扬声器日记（AVSD），旨在使用音频和视频数据解决“谁在什么时候说话”; 2）一个新的视听日记和识别（AVDR）任务，重点是解决“谁在什么时候说了什么”与视听扬声器日记结果。两首曲目都专注于中文，并在真实的家庭电视场景中使用远场音频和视频：2-6人在电视噪音的背景下相互交流。本文介绍了MISP 2022挑战赛的数据集、赛道设置和基线。我们对实验和实例的分析表明，AVDR基线系统具有良好的性能，但由于以下原因，远场视频质量、背景中存在TV噪声以及无法区分的扬声器。
英文摘要：
The Multi-modal Information based Speech Processing (MISP) challenge aims to extend the application of signal processing technology in specific scenarios by promoting the research into wake-up words, speaker diarization, speech recognition, and other technologies. The MISP2022 challenge has two tracks: 1) audio-visual speaker diarization (AVSD), aiming to solve ``who spoken when'' using both audio and visual data; 2) a novel audio-visual diarization and recognition (AVDR) task that focuses on addressing ``who spoken what when'' with audio-visual speaker diarization results. Both tracks focus on the Chinese language, and use far-field audio and video in real home-tv scenarios: 2-6 people communicating each other with TV noise in the background. This paper introduces the dataset, track settings, and baselines of the MISP2022 challenge. Our analyses of experiments and examples indicate the good performance of AVDR baseline system, and the potential difficulties in this challenge due to, e.g., the far-field video quality, the presence of TV noise in the background, and the indistinguishable speakers.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
