VSRo-200：用于研究监督和多模态鲁棒性的罗马尼亚语视觉语音识别数据集
VSRo-200: A Romanian Visual Speech Recognition Dataset for Studying Supervision and Multimodal Robustness
AI总结：
本文介绍罗马尼亚语视觉语音识别数据集VSRo-200，通过伪标签和人工转录注释样本，建立低资源视觉语音识别基准。研究监督质量影响，评估域转移鲁棒性及噪声下视听语音识别，证明其表示可转移至其他基准，为相关研究提供新测试平台。
摘要：
我们介绍了VSRo-200，这是首个用于罗马尼亚语视觉语音识别（唇读）的大规模数据集，包含200小时的真实播客视频。所有样本都用微调后的罗马尼亚语ASR模型生成的伪标签进行注释，同时100小时的子集由人工转录，可在统一框架下对监督质量进行可控分析。基于此数据集，我们建立了低资源环境下视觉语音识别的基准。系统研究了监督质量的影响，表明在固定数据规模下人工注释性能更好，伪标签则通过可扩展性实现持续改进。还使用精选的分布外测试集评估了域转移下的鲁棒性，并分析了噪声条件下的视听语音识别，其中多模态融合比仅音频模型显著提高了鲁棒性。最后证明，在VSRo-200上学习的表示可有效转移到LRRo孤立词识别基准上，大幅超越先前报告的结果。总体而言，VSRo-200为低资源视觉语音识别中的监督、域泛化和多模态融合研究提供了新的测试平台。
英文摘要：
We introduce VSRo-200, the first large-scale dataset for visual speech recognition (lip reading) in Romanian, comprising 200 hours of real-world podcast videos. All samples are annotated with pseudo-labels generated by a fine-tuned Romanian ASR model, while a subset of 100 hours is additionally transcribed by humans, enabling controlled analysis of supervision quality under a unified framework. Building on this dataset, we establish a benchmark for visual speech recognition in low-resource settings. We systematically study the impact of supervision quality, showing that while human annotations provide better performance at fixed data scales, pseudo-labels enable continued improvements through scalability. We further evaluate robustness under domain shift using curated out-of-distribution (OOD) test sets, and analyze audio-visual speech recognition (AVSR) under noisy conditions, where multimodal fusion significantly improves robustness compared to audio-only models. Finally, we demonstrate that representations learned on VSRo-200 transfer effectively to the LRRo benchmark for isolated word recognition, substantially outperforming previously reported results. Overall, VSRo-200 provides a new testbed for studying supervision, domain generalization, and multimodal fusion in low-resource visual speech recognition.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
