Only a handful of the world's languages are abundant with the resources that enable practical applications of speech processing technologies. One of the methods to overcome this problem is to use the resources existing in other languages to train a multilingual automatic speech recognition (ASR) model, which, intuitively, should learn some universal phonetic representations. In this work, we focus on gaining a deeper understanding of how general these representations might be, and how individual phones are getting improved in a multilingual setting. To that end, we select a phonetically diverse set of languages, and perform a series of monolingual, multilingual and crosslingual (zero-shot) experiments. The ASR is trained to recognize the International Phonetic Alphabet (IPA) token sequences. We observe significant improvements across all languages in the multilingual setting, and stark degradation in the crosslingual setting, where the model, among other errors, considers Javanese as a tone language. Notably, as little as 10 hours of the target language training data tremendously reduces ASR error rates. Our analysis uncovered that even the phones that are unique to a single language can benefit greatly from adding training data from other languages - an encouraging result for the low-resource speech community.
From Semi-supervised to Almost-unsupervised Speech Recognition with Very-low Resource by Jointly Learning Phonetic Structures from Audio and Text Embeddings
UniGlyph: A Seven-Segment Script for Universal Language Representation
St. Xavier's Catholic College of Engineering（圣泽维尔天主教工程学院）；Loyola College（洛约拉学院）；University of Madras（马德拉斯大学）；Bell Matriculation Higher Secondary School（贝尔预科高中）；St. Xavier’s College（圣泽维尔学院）；Manonmaniyam Sundaranar University（马农马尼亚姆·孙达拉纳尔大学）
Discovering Phonetic Inventories with Crosslingual Automatic Speech Recognition
Center of Language and Speech Processing, The Johns Hopkins University（约翰斯·霍普金斯大学语言与语音处理中心）；Human Language Technology Center of Excellence, The Johns Hopkins University（约翰斯·霍普金斯大学人类语言技术卓越中心）；Multimedia Computing Group, Delft University of Technology（代尔夫特理工大学多媒体计算组）；Department of Electrical and Computer Engineering, University of Illinois（伊利诺伊大学电气与计算机工程系）
Deep Sound Change: Deep and Iterative Learning, Convolutional Neural Networks, and Language Change
Department of Linguistics, University of California, Berkeley（加利福尼亚大学伯克利分校语言学系）
本文提出了一种结合深度学习和迭代学习的声音变化建模框架。语音的获取和传输通过在未注释的原始语音数据上训练生成对抗网络（GANs）来建模。本文认为，几个属性的声音变化出现从建议的架构。GANs（Goodfellow et al. 2014 arXiv：1406.2661，Donahue et al. 2019 arXiv：1705.07904）是唯一适合建模语言变化的网络，因为网络是在原始的无监督声学数据上训练的，不包含语言特定的特征，并且正如Beguplant（2020 arXiv：2006.03965）所认为的那样，在其潜在空间中编码语音和音位表示，并生成语言信息丰富的创新数据。第一代网络是在TIMIT的人类语音中的相关序列上训练的。随后的几代不是在TIMIT上训练的，而是在上一代生成的输出上训练的，因此在迭代学习任务中开始相互学习。初始的allophonic分布随着每一代逐渐丢失，这可能是由于来自训练数据中的吸气的全局分布的压力。这些网络显示出语音目标逐渐转移的迹象，这是语音逐渐变化的特征。在端点，输出表面上类似于语音变化--规则丢失。
Inductive biases, pretraining and fine-tuning jointly account for brain responses to speech
Facebook AI Research（Facebook AI研究院）；Université de Paris（巴黎大学）；CoML, ENS/CNRS/EHESS/INRIA/PSL University（计算语言学实验室，巴黎高等师范学院/法国国家科学研究中心/法国社会科学高等研究院/法国国家信息与自动化研究所/巴黎文理研究大学）；CRI, Département Frontières du Vivant et de l’Apprendre, IIFR, Université de Paris（认知研究所，生命与学习前沿系，法国医学研究院，巴黎大学）；École normale supérieure, PSL University, CNRS（巴黎高等师范学院，巴黎文理研究大学，法国国家科学研究中心）
Evaluating computational models of infant phonetic learning across languages
School of Informatics, University of Edinburgh（爱丁堡大学信息学院）；Department of Linguistics & UMIACS, University of Maryland（马里兰大学语言学系与UMIACS）；E&E Engineering, Stellenbosch University（斯泰伦博斯大学电气与电子工程系）
Beyond Phones: Structured Phonemic Modeling for Vietnamese Automatic Speech Recognition
Faculty of Information Science and Engineering（信息科学与工程学院）；Faculty of Computer Science（计算机科学学院）；University of Information Technology（信息技术大学）；Vietnam National University, Ho Chi Minh city（越南国家大学，胡志明市）
Enhancing Quranic Learning: A Multimodal Deep Learning Approach for Arabic Phoneme Recognition
Department of Electronics and Communication Engineering, Kocaeli University, 41001, Kocaeli, Turkey（电子与通信工程系，科拉尔大学）；Department of Information Systems Engineering, Kocaeli University, 41001, Kocaeli, Turkey（信息系统工程系，科拉尔大学）；Maviay Consultancy Company, Kocaeli University Technopark, 41275, Kocaeli, Turkey（Maviay咨询公司，科拉尔大学技术园区）
Towards stable AI systems for Evaluating Arabic Pronunciations
Bio-Medical Imaging Core, Core Technology Platforms, New York University Abu Dhabi（生物医学成像核心、核心技术平台、纽约大学阿布扎比分校）；Institut de recherche technologique SystemX（SystemX技术研究所）；NYUAD Research Institute, New York University Abu Dhabi（NYUAD研究学院、纽约大学阿布扎比分校）；Courant Institute of Mathematical Sciences, New York University（Courant数学科学研究所、纽约大学）
Learning Disentangled Speech Representations with Contrastive Learning and Time-Invariant Retrieval
Ping An Technology (Shenzhen) Co., Ltd.（平安科技（深圳）有限公司）；University of Science and Technology of China（中国科学技术大学）；Huya Inc (Shenzhen) Co., Ltd.（虎牙信息科技（深圳）有限公司）
VoxCog: Towards End-to-End Multilingual Cognitive Impairment Classification through Dialectal Knowledge
Ming Hsieh Department of Electrical and Computer Engineering, University of Southern California（明斯希德电气与计算机工程系，南加州大学）；Dornsife College of Letters, Arts and Sciences, University of Southern California（多恩斯菲学院，南加州大学）
