This study investigates how learners organize perceptual space in early phonetic acquisition by advancing previous studies in two key aspects. Firstly, it examines the shape of the learned hidden representation as well as its ability to categorize phonetic categories. Secondly, it explores the impact of training models on context-free acoustic information, without involving contextual cues, on phonetic acquisition, closely mimicking the early language learning stage. Using a cross-linguistic modeling approach, autoencoder models are trained on English and Mandarin and evaluated in both native and non-native conditions, following experimental conditions used in infant language perception studies. The results demonstrate that unsupervised bottom-up training on context-free acoustic information leads to comparable learned representations of perceptual space between native and non-native conditions for both English and Mandarin, resembling the early stage of universal listening in infants. These findings provide insights into the organization of perceptual space during early phonetic acquisition and contribute to our understanding of the formation and representation of phonetic categories.
Many mispronunciation detection and diagnosis (MD&D) research approaches try to exploit both the acoustic and linguistic features as input. Yet the improvement of the performance is limited, partially due to the shortage of large amount annotated training data at the phoneme level. Phonetic embeddings, extracted from ASR models trained with huge amount of word level annotations, can serve as a good representation of the content of input speech, in a noise-robust and speaker-independent manner. These embeddings, when used as implicit phonetic supplementary information, can alleviate the data sh…
ENS（巴黎高等师范学院）；CNRS（法国国家科学研究中心）；EHESS（社会科学高等学院）；INRIA（法国国家信息与自动化研究所）；University of Paris（巴黎大学）；University of Toronto（多伦多大学）
Our native language influences the way we perceive speech sounds, affecting our ability to discriminate non-native sounds. We compare two ideas about the influence of the native language on speech perception: the Perceptual Assimilation Model, which appeals to a mental classification of sounds into native phoneme categories, versus the idea that rich, fine-grained phonetic representations tuned to the statistics of the native language, are sufficient. We operationalize this idea using representations from two state-of-the-art speech models, a Dirichlet process Gaussian mixture model and the mo…
Self-Supervised Speech Representations are More Phonetic than Semantic
Carnegie Mellon University（卡内基梅隆大学）；Toyota Technological Institute at Chicago（芝加哥丰田理工学院）；National Institute of Advanced Industrial Science and Technology (AIST)（日本国立先进工业科学技术研究院（AIST））
Pun Intended: Multi-Agent Translation of Wordplay with Contrastive Learning and Phonetic-Semantic Embeddings for CLEF JOKER 2025 Task 2
Georgia Institute of Technology（佐治亚理工学院）；Peoples' Friendship University of Russia (RUDN University)（俄罗斯人民友谊大学）；Joint Institute for Nuclear Research（联合核子研究中心）；Vrije Universiteit Amsterdam（阿姆斯特丹自由大学）；University of Skövde（斯德哥尔摩大学）
School of Informatics, University of Edinburgh（爱丁堡大学信息学院）；Department of Linguistics & UMIACS, University of Maryland（马里兰大学语言学系与UMIACS）；E&E Engineering, Stellenbosch University（斯泰伦博斯大学电气与电子工程系）
In the first year of life, infants' speech perception becomes attuned to the sounds of their native language. Many accounts of this early phonetic learning exist, but computational models predicting the attunement patterns observed in infants from the speech input they hear have been lacking. A recent study presented the first such model, drawing on algorithms proposed for unsupervised learning from naturalistic speech, and tested it on a single phone contrast. Here we study five such algorithms, selected for their potential cognitive relevance. We simulate phonetic learning with each algorith…
An essential design decision for multilingual Neural Text-To-Speech (NTTS) systems is how to represent input linguistic features within the model. Looking at the wide variety of approaches in the literature, two main paradigms emerge, unified and separate representations. The former uses a shared set of phonetic tokens across languages, whereas the latter uses unique phonetic tokens for each language. In this paper, we conduct a comprehensive study comparing multilingual NTTS systems models trained with both representations. Our results reveal that the unified approach consistently achieves be…
UniGlyph: A Seven-Segment Script for Universal Language Representation
St. Xavier's Catholic College of Engineering（圣泽维尔天主教工程学院）；Loyola College（洛约拉学院）；University of Madras（马德拉斯大学）；Bell Matriculation Higher Secondary School（贝尔预科高中）；St. Xavier’s College（圣泽维尔学院）；Manonmaniyam Sundaranar University（马农马尼亚姆·孙达拉纳尔大学）
University of Washington（华盛顿大学）；University of California, Berkeley（加利福尼亚大学伯克利分校）
Training deep neural networks on well-understood dependencies in speech data can provide new insights into how they learn internal representations. This paper argues that acquisition of speech can be modeled as a dependency between random space and generated speech data in the Generative Adversarial Network architecture and proposes a methodology to uncover the network's internal representations that correspond to phonetic and phonological properties. The Generative Adversarial architecture is uniquely appropriate for modeling phonetic and phonological learning because the network is trained o…
Facebook AI Research（Facebook人工智能研究院）；EHESS（社会科学高等学院）；CNRS（法国国家科学研究中心）；INRIA（法国国家信息与自动化研究所）
We introduce a new unsupervised task, spoken language modeling: the learning of linguistic representations from raw audio signals without any labels, along with the Zero Resource Speech Benchmark 2021: a suite of 4 black-box, zero-shot metrics probing for the quality of the learned models at 4 linguistic levels: phonetics, lexicon, syntax and semantics. We present the results and analyses of a composite baseline made of the concatenation of three unsupervised systems: self-supervised contrastive representation learning (CPC), clustering (k-means) and language modeling (LSTM or BERT). The langu…
Individual Biases, Cultural Evolution, and the Statistical Nature of Language Universals: The Case of Colour Naming Systems
City University London（伦敦城市大学）；Sapienza Università di Roma（罗马大学）；Institute for Scientific Interchange (ISI)（科学交流研究所（ISI））；SONY Computer Science Lab (SONY-CSL)（索尼计算机科学实验室（SONY-CSL））；CNR-ISC（意大利国家研究委员会复杂系统研究所（CNR-ISC））
