Several variants of deep neural networks have been successfully employed for building parametric models that project variable-duration spoken word segments onto fixed-size vector representations, or acoustic word embeddings (AWEs). However, it remains unclear to what degree we can rely on the distance in the emerging AWE space as an estimate of word-form similarity. In this paper, we ask: does the distance in the acoustic embedding space correlate with phonological dissimilarity? To answer this question, we empirically investigate the performance of supervised approaches for AWEs with different neural architectures and learning objectives. We train AWE models in controlled settings for two languages (German and Czech) and evaluate the embeddings on two tasks: word discrimination and phonological similarity. Our experiments show that (1) the distance in the embedding space in the best cases only moderately correlates with phonological distance, and (2) improving the performance on the word discrimination task does not necessarily yield models that better reflect word phonological similarity. Our findings highlight the necessity to rethink the current intrinsic evaluations for AWEs.
Multilingual and crosslingual speech recognition using phonological-vector based phone embeddings
Tsinghua University（清华大学）；Beijing National Research Center for Information Science and Technology（北京信息科学与技术国家研究中心）
语音特征（PF）的使用潜在地允许特定语言的音素在训练中保持链接，这对于用于低资源语言的多语言和跨语言语音识别方法的信息共享是非常期望的。以前的方法在使用语音特征时的缺点是，以自底向上的方式提取声学到PF本身是困难的。在本文中，我们建议将音位驱动的音素嵌入（自上而下）和基于深度神经网络（DNN）的声学特征提取（自下而上）结合起来，以计算音素概率。这种新方法被称为JoinAP（Joining of Acoustics and Phonology）。值得注意的是，语音识别不需要从声学特征到音位特征的反演。对于IPA（International Phonetic Alphabet）表中的每个音素，我们将其音素特征编码为一个音素向量，然后对该音素向量进行线性或非线性变换，得到音素嵌入。在CommonVoice数据集（德语、法语、西班牙语和意大利语）和AISHLL-1数据集（普通话）上进行了一系列多语种和跨语种（包括zero-shot和Few-Shot）语音识别实验，证明了非线性音素嵌入的JoinAP方法优于线性音素嵌入的JoinAP方法和传统的平面音素嵌入方法。
Are words easier to learn from infant- than adult-directed speech? A quantitative corpus-based investigation
Laboratoire de Sciences Cognitives et Psycholinguistique ENS/EHESS/CNRS/PSL（认知科学与心理语言学实验室（巴黎高等师范学院/社会科学高等研究院/法国国家科学研究中心/巴黎文理大学））；Department of English Literature and Language Faculty of Letters Konan University（甲南大学文学院英语文学与语言系）；Laboratory for Language Development RIKEN Brain Science Institute（理化学研究所脑科学中心语言发展实验室）；Department of Psychology and Neuroscience Duke University（杜克大学心理学与神经科学系）
The False Resonance: A Critical Examination of Emotion Embedding Similarity for Speech Generation Evaluation
Graduate Institute of Communication Engineering, National Taiwan University, Taiwan（国立台湾大学通讯工程研究所）；Ming Hsieh Department of Electrical and Computer Engineering, University of Southern California, USA（美国南加州大学明希电气与计算机工程系）；Gilbert AI Lab, USA（美国Gilbert人工智能实验室）；Graduate Institute of Electrical Engineering, National Taiwan University, Taiwan（国立台湾大学电气工程研究所）；Artificial Intelligence Center of Research Excellence, National Taiwan University, Taiwan（国立台湾大学人工智能研究中心）
