Learning to Understand Child-directed and Adult-directed Speech
Tilburg University（蒂尔堡大学）
Lieke Gelderloos, Grzegorz Chrupała, Afra Alishahi
备注：
Authors found an error in preprocessing of transcriptions before they were fed to SBERT. After correction, the experiments were rerun. The updated results can be found in this version. Importantly, - Most scores were affected to a small degree (performance was slightly worse). - The effect was consistent across conditions. Therefore, the general patterns remain the same
Speech directed to children differs from adult-directed speech in linguistic aspects such as repetition, word choice, and sentence length, as well as in aspects of the speech signal itself, such as prosodic and phonemic variation. Human language acquisition research indicates that child-directed speech helps language learners. This study explores the effect of child-directed speech when learning to extract semantic information from speech directly. We compare the task performance of models trained on adult-directed speech (ADS) and child-directed speech (CDS). We find indications that CDS helps in the initial stages of learning, but eventually, models trained on ADS reach comparable task performance, and generalize better. The results suggest that this is at least partially due to linguistic rather than acoustic properties of the two registers, as we see the same pattern when looking at models trained on acoustically comparable synthetic speech.
Massachusetts Institute of Technology（麻省理工学院）；Duke University（杜克大学）；University of Chicago（芝加哥大学）
Children's early speech often bears little resemblance to that of adults, and yet parents and other caregivers are able to interpret that speech and react accordingly. Here we investigate how these adult inferences as listeners reflect sophisticated beliefs about what children are trying to communicate, as well as how children are likely to pronounce words. Using a Bayesian framework for modeling spoken word recognition, we find that computational models can replicate adult interpretations of children's speech only when they include strong, context-specific prior expectations about the message…
MIT（麻省理工学院）；Duke University（杜克大学）；University of Chicago（芝加哥大学）
How do adults understand children's speech? Children's productions over the course of language development often bear little resemblance to typical adult pronunciations, yet caregivers nonetheless reliably recover meaning from them. Here, we employ a suite of Bayesian models of spoken word recognition to understand how adults overcome the noisiness of child language, showing that communicative success between children and adults relies heavily on adult inferential processes. By evaluating competing models on phonetically-annotated corpora, we show that adults' recovered meanings are best predi…
Can Generic LLMs Help Analyze Child-adult Interactions Involving Children with Autism in Clinical Observation?
University of Southern California（南加州大学）；Boston University（波士顿大学）；Korea University（高丽大学）；University of California, San Francisco（加利福尼亚大学旧金山分校）；University of California, Los Angeles（加利福尼亚大学洛杉矶分校）
University of Washington（华盛顿大学）；University of California Los Angeles（加州大学洛杉矶分校）
Disfluencies are prevalent in spontaneous speech, as shown in many studies of adult speech. Less is understood about children's speech, especially in pre-school children who are still developing their language skills. We present a novel dataset with annotated disfluencies of spontaneous explanations from 26 children (ages 5--8), interviewed twice over a year-long period. Our preliminary analysis reveals significant differences between children's speech in our corpus and adult spontaneous speech from two corpora (Switchboard and CallHome). Children have higher disfluency and filler rates, tend …
CAIT: A Syntactic Parsing Toolkit for Child-Adult InTeractions
Center for Language and Cognition (CLCG), University of Groningen（格罗宁根大学语言与认知中心）；Georgetown University（乔治城大学）；Computational Linguistics, Department of Linguistics, Bielefeld University（比勒菲尔德大学语言学系计算语言学）
Children's speech recognition is a vital, yet largely overlooked domain when building inclusive speech technologies. The major challenge impeding progress in this domain is the lack of adequate child speech corpora; however, recent advances in self-supervised learning have created a new opportunity for overcoming this problem of data scarcity. In this paper, we leverage self-supervised adult speech representations and use three well-known child speech corpora to build models for children's speech recognition. We assess the performance of fine-tuning on both native and non-native children's spe…
Child-Adapted Structured Phonological Representations for Interpretable Speech Sound Analysis
Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg（弗里德里希-亚历山大-埃尔朗根-纽伦堡大学模式识别实验室）；GITA Lab. Facultad de Ingeniería. Universidad de Antioquia UdeA（安蒂奥基亚大学工程学院GITA实验室）；Chair for AI in Healthcare and Medicine, Technical University of Munich（慕尼黑工业大学医疗与医学人工智能教席）
In earlier work, we have shown that articulation rate in Swedish child-directed speech (CDS) increases as a function of the age of the child, even when utterance length and differences in articulation rate between subjects are controlled for. In this paper we show on utterance level in spontaneous Swedish speech that i) for the youngest children, articulation rate in CDS is lower than in adult-directed speech (ADS), ii) there is a significant negative correlation between articulation rate and surprisal (the negative log probability) in ADS, and iii) the increase in articulation rate in Swedish…
Automatic Speech Recognition (ASR) systems are known to exhibit difficulties when transcribing children's speech. This can mainly be attributed to the absence of large children's speech corpora to train robust ASR models and the resulting domain mismatch when decoding children's speech with systems trained on adult data. In this paper, we propose multiple enhancements to alleviate these issues. First, we propose a data augmentation technique based on the source-filter model of speech to close the domain gap between adult and children's speech. This enables us to leverage the data availability …
University of Edinburgh（爱丁堡大学）；Stanford University（斯坦福大学）
The impressive ability of children to acquire language is a widely studied phenomenon, and the factors influencing the pace and patterns of word learning remains a subject of active research. Although many models predicting the age of acquisition of words have been proposed, little emphasis has been directed to the raw input children achieve. In this work we present a comparatively large-scale multi-modal corpus of prosody-text aligned child directed speech. Our corpus contains automatically extracted word-level prosodic features, and we investigate the utility of this information as predictor…
Children speech recognition is challenging mainly due to the inherent high variability in children's physical and articulatory characteristics and expressions. This variability manifests in both acoustic constructs and linguistic usage due to the rapidly changing developmental stage in children's life. Part of the challenge is due to the lack of large amounts of available children speech data for efficient modeling. This work attempts to address the key challenges using transfer learning from adult's models to children's models in a Deep Neural Network (DNN) framework for children's Automatic …
Recent progress in grammar induction has shown that grammar induction is possible without explicit assumptions of language-specific knowledge. However, evaluation of induced grammars usually has ignored phrasal labels, an essential part of a grammar. Experiments in this work using a labeled evaluation metric, RH, show that linguistically motivated predictions about grammar sparsity and use of categories can only be revealed through labeled evaluation. Furthermore, depth-bounding as an implementation of human memory constraints in grammar inducers is still effective with labeled evaluation on m…
University of Trento（特伦托大学）；University of Groningen（格罗宁根大学）；Fondazione Bruno Kessler (FBK)（布鲁诺·凯斯勒基金会（FBK））
Pronunciation is one of the fundamentals of language learning, and it is considered a primary factor of spoken language when it comes to an understanding and being understood by others. The persistent presence of high error rates in speech recognition domains resulting from mispronunciations motivates us to find alternative techniques for handling mispronunciations. In this study, we develop a mispronunciation assessment system that checks the pronunciation of non-native English speakers, identifies the commonly mispronounced phonemes of Italian learners of English, and presents an evaluation …
Laboratoire de Sciences Cognitives et Psycholinguistique ENS/EHESS/CNRS/PSL（认知科学与心理语言学实验室（巴黎高等师范学院/社会科学高等研究院/法国国家科学研究中心/巴黎文理大学））；Department of English Literature and Language Faculty of Letters Konan University（甲南大学文学院英语文学与语言系）；Laboratory for Language Development RIKEN Brain Science Institute（理化学研究所脑科学中心语言发展实验室）；Department of Psychology and Neuroscience Duke University（杜克大学心理学与神经科学系）
We investigate whether infant-directed speech (IDS) could facilitate word form learning when compared to adult-directed speech (ADS). To study this, we examine the distribution of word forms at two levels, acoustic and phonological, using a large database of spontaneous speech in Japanese. At the acoustic level we show that, as has been documented before for phonemes, the realizations of words are more variable and less discriminable in IDS than in ADS. At the phonological level, we find an effect in the opposite direction: the IDS lexicon contains more distinctive words (such as onomatopoeias…
A statistical model for segmentation and word discovery in child directed speech is presented. An incremental unsupervised learning algorithm to infer word boundaries based on this model is described and results of empirical tests showing that the algorithm is competitive with other models that have been used for similar tasks are also presented.
Korean aegyo speech shows systematic F1 increase to signal childlike qualities
Department of Korean Language and Literature, Duksung Women’s University（韩国语言文学系，杜庆女子大学）；Department of Computational Linguistics, University of Zurich（计算语言学系，苏黎世大学）
University of Virginia（弗吉尼亚大学）；New York University（纽约大学）；Colgate University（科尔盖特大学）；University of California, Los Angeles（加利福尼亚大学洛杉矶分校）
Many households include children who use voice personal assistants (VPA) such as Amazon Alexa. Children benefit from the rich functionalities of VPAs and third-party apps but are also exposed to new risks in the VPA ecosystem. In this paper, we first investigate "risky" child-directed voice apps that contain inappropriate content or ask for personal information through voice interactions. We build SkillBot - a natural language processing (NLP)-based system to automatically interact with VPA apps and analyze the resulting conversations. We find 28 risky child-directed apps and maintain a growin…
Can large audio language models understand child stuttering speech? speech summarization, and source separation
Department of Computer Science, University of Florida, Gainesville, FL, USA（佛罗里达大学计算机科学系）；Department of Computer Science, Lebanese American University（黎巴嫩美国大学计算机科学系）
ChildGuard: A Specialized Dataset for Combatting Child-Targeted Hate Speech
Macquarie University（麦考瑞大学）；MBZUAI（穆罕默德·本·拉希德人工智能研究所）；DSEU（德里国家理工学院）；Department of Information and Computer Science, KFUPM（科威特石油大学信息与计算机科学系）；Stanford University（斯坦福大学）
