OLKAVS：一个开放的大规模韩语音视觉语音数据集
OLKAVS: An Open Large-Scale Korean Audio-Visual Speech Dataset
AI总结：
本文提出大规模开放韩语音视觉语音数据集 OLKAVS，含1,150小时九视角多噪声数据，并提供语音识别与唇读基线以支持多模态研究。
摘要：
受人类以多模态方式理解语音的启发，研究人员已经构建了各种音视觉数据集。然而，大多数现有数据集聚焦于英语，在数据集准备过程中会引入与各种预测模型的依赖，并且仅包含少量多视角视频。为缓解这些局限，我们近期开发了 Open Large-scale Korean Audio-Visual Speech（OLKAVS，开放大规模韩语音视觉语音）数据集，它是公开可用的音视觉语音数据集中规模最大的。该数据集包含来自1,107名韩语说话者的1,150小时带转写音频，采集于具有九个不同视角和多种噪声情境的演播室环境。我们还提供了面向音视觉语音识别和唇读两项任务的预训练基线模型。我们基于这些模型开展实验，以验证多模态和多视角训练相对于单模态以及仅正面视角训练的有效性。我们期望 OLKAVS 数据集能够推动更广泛领域的多模态研究，例如韩语语音识别、说话人识别、发音水平分类以及口部运动分析。
英文摘要：
Inspired by humans comprehending speech in a multi-modal manner, various audio-visual datasets have been constructed. However, most existing datasets focus on English, induce dependencies with various prediction models during dataset preparation, and have only a small number of multi-view videos. To mitigate the limitations, we recently developed the Open Large-scale Korean Audio-Visual Speech (OLKAVS) dataset, which is the largest among publicly available audio-visual speech datasets. The dataset contains 1,150 hours of transcribed audio from 1,107 Korean speakers in a studio setup with nine different viewpoints and various noise situations. We also provide the pre-trained baseline models for two tasks, audio-visual speech recognition and lip reading. We conducted experiments based on the models to verify the effectiveness of multi-modal and multi-view training over uni-modal and frontal-view-only training. We expect the OLKAVS dataset to facilitate multi-modal research in broader areas such as Korean speech recognition, speaker recognition, pronunciation level classification, and mouth motion analysis.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
