Whistle：通过弱音素监督实现数据高效的多语言和跨语言语音识别
Whistle: Data-Efficient Multilingual and Crosslingual Speech Recognition via Weakly Phonetic Supervision
AI总结：
本文提出Whistle，一种通过弱音素监督预训练实现数据高效的多语言和跨语言语音识别方法，实验表明其在已见语言识别、未见语言跨语言性能、抗遗忘和训练效率上优于其他方法。
摘要：
多语言和跨语言自动语音识别（MCL-ASR）存在三种方法：使用音素或字形转录的监督预训练，以及自监督预训练。我们发现，迄今为止，音素监督的预训练在MCL-ASR中未被充分重视，而概念上它更有利于不同语言之间的信息共享。本文探索了使用弱音素监督进行预训练以实现数据高效的MCL-ASR的方法，称为Whistle。我们放宽了金标准人工验证音素转录的要求，通过利用LanguageNet字素到音素（G2P）模型获得基于国际音标（IPA）的转录。我们基于CommonVoice数据集构建了一个共同的实验设置，称为CV-Lang10，包含10种已见语言和2种未见语言。在CV-Lang10上进行了一系列实验，以尽可能公平地比较这三种方法在MCL-ASR共同设置下的表现。实验证明了基于音素的模型（Whistle）在MCL-ASR中的优势，包括对已见语言的语音识别、对未见语言使用不同数量少样本数据的跨语言性能、克服灾难性遗忘以及训练效率。研究发现，当训练数据更有限时，音素监督相比子词监督和自监督可以获得更好的结果，从而提供更高的数据效率。为了支持可复现性并促进这一方向的未来研究，我们在https://github.com/thu-spmi/CAT/tree/master/egs/cv-lang10发布了Whistle整个流程的代码、模型和数据。
英文摘要：
There exist three approaches for multilingual and crosslingual automatic speech recognition (MCL-ASR) - supervised pretraining with phonetic or graphemic transcription, and self-supervised pretraining. We find that pretraining with phonetic supervision has been underappreciated so far for MCL-ASR, while conceptually it is more advantageous for information sharing between different languages. This paper explores the approach of pretraining with weakly phonetic supervision towards data-efficient MCL-ASR, which is called Whistle. We relax the requirement of gold-standard human-validated phonetic transcripts, and obtain International Phonetic Alphabet (IPA) based transcription by leveraging the LanguageNet grapheme-to-phoneme (G2P) models. We construct a common experimental setup based on the CommonVoice dataset, called CV-Lang10, with 10 seen languages and 2 unseen languages. A set of experiments are conducted on CV-Lang10 to compare, as fair as possible, the three approaches under the common setup for MCL-ASR. Experiments demonstrate the advantages of phoneme-based models (Whistle) for MCL-ASR, in terms of speech recognition for seen languages, crosslingual performance for unseen languages with different amounts of few-shot data, overcoming catastrophic forgetting, and training efficiency. It is found that when training data is more limited, phoneme supervision can achieve better results compared to subword supervision and self-supervision, thereby providing higher data-efficiency. To support reproducibility and promote future research along this direction, we release the code, models and data for the entire pipeline of Whistle at https://github.com/thu-spmi/CAT/tree/master/egs/cv-lang10.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
