Discovering Phonetic Inventories with Crosslingual Automatic Speech Recognition

Whistle：通过弱音素监督实现数据高效的多语言和跨语言语音识别

Whistle: Data-Efficient Multilingual and Crosslingual Speech Recognition via Weakly Phonetic Supervision

本文提出Whistle，一种通过弱音素监督预训练实现数据高效的多语言和跨语言语音识别方法，实验表明其在已见语言识别、未见语言跨语言性能、抗遗忘和训练效率上优于其他方法。

That Sounds Familiar: an Analysis of Phonetic Representations Transfer Across Languages

Only a handful of the world's languages are abundant with the resources that enable practical applications of speech processing technologies. One of the methods to overcome this problem is to use the resources existing in other languages to train a multilingual automatic speech recognition (ASR) model, which, intuitively, should learn some universal phonetic representations. In this work, we focus on gaining a deeper understanding of how general these representations might be, and how individual phones are getting improved in a multilingual setting. To that end, we select a phonetically divers…

Multilingual and crosslingual speech recognition using phonological-vector based phone embeddings

The use of phonological features (PFs) potentially allows language-specific phones to remain linked in training, which is highly desirable for information sharing for multilingual and crosslingual speech recognition methods for low-resourced languages. A drawback suffered by previous methods in using phonological features is that the acoustic-to-PF extraction in a bottom-up way is itself difficult. In this paper, we propose to join phonology driven phone embedding (top-down) and deep neural network (DNN) based acoustic feature extraction (bottom-up) to calculate phone probabilities. The new me…

无需音素词典的音素基跨语言语音识别联合随机近似训练

Pronunciation-Lexicon Free Training for Phoneme-based Crosslingual ASR via Joint Stochastic Approximation

本研究提出无需音素词典的跨语言语音识别方法，通过联合随机近似算法提升模型性能，实现误差率降低。

How Phonotactics Affect Multilingual and Zero-shot ASR Performance

The idea of combining multiple languages' recordings to train a single automatic speech recognition (ASR) model brings the promise of the emergence of universal speech representation. Recently, a Transformer encoder-decoder model has been shown to leverage multilingual data well in IPA transcriptions of languages presented during training. However, the representations it learned were not successful in zero-shot transfer to unseen languages. Because that model lacks an explicit factorization of the acoustic model (AM) and language model (LM), it is unclear to what degree the performance suffere…

Allophant：基于发音属性的跨语言音素识别

Allophant: Cross-lingual Phoneme Recognition with Articulatory Attributes

本文提出多语言音素识别器 Allophant，利用发音属性多任务学习与 PHOIBLE 扩展资源，仅依赖音素清单即可跨语言迁移，并显著降低低资源与零样本场景的音素错误率。