How do humans learn language, and can the first language be learned at all? These fundamental questions are still hotly debated. In contemporary linguistics, there are two major schools of thought that give completely opposite answers. According to Chomsky's theory of universal grammar, language cannot be learned because children are not exposed to sufficient data in their linguistic environment. In contrast, usage-based models of language assume a profound relationship between language structure and language use. In particular, contextual mental processing and mental representations are assumed to have the cognitive capacity to capture the complexity of actual language use at all levels. The prime example is syntax, i.e., the rules by which words are assembled into larger units such as sentences. Typically, syntactic rules are expressed as sequences of word classes. However, it remains unclear whether word classes are innate, as implied by universal grammar, or whether they emerge during language acquisition, as suggested by usage-based approaches. Here, we address this issue from a machine learning and natural language processing perspective. In particular, we trained an artificial deep neural network on predicting the next word, provided sequences of consecutive words as input. Subsequently, we analyzed the emerging activation patterns in the hidden layers of the neural network. Strikingly, we find that the internal representations of nine-word input sequences cluster according to the word class of the tenth word to be predicted as output, even though the neural network did not receive any explicit information about syntactic rules or word classes during training. This surprising result suggests, that also in the human brain, abstract representational categories such as word classes may naturally emerge as a consequence of predictive coding and processing during language acquisition.
Scaling laws in human speech, decreasing emergence of new words and a generalized model
Nanjing University（南京大学）；Boston University（波士顿大学）；Beth Israel Deaconess Medical Center（贝斯以色列女执事医疗中心）；Harvard Medical School（哈佛医学院）；Nanjing University of Posts and Telecommunications（南京邮电大学）
Word-Class and Construction-Like Structure Emerges in Neural Successor Representations Trained on Natural Language
Cognitive Computational Neuroscience Group（认知计算神经科学组）；Friedrich-Alexander-Universität Erlangen–Nürnberg (FAU)（弗赖堡-亚历山大-大学埃尔兰根-纽伦堡（FAU））；Mannheim Center for Neuromodulation and Neuroprosthetics (MCNN)（曼海姆神经调制与神经假体中心（MCNN））；University Hospital Mannheim, University Heidelberg（曼海姆大学医院，海德堡大学）
Where New Words Are Born: Distributional Semantic Analysis of Neologisms and Their Semantic Neighborhoods
Language Technologies Institute, Carnegie Mellon University（卡内基梅隆大学语言技术研究所）；Department of Computer Science, University of Toronto（多伦多大学计算机科学系）；Computer Science and Engineering, University of California, San Diego（加利福尼亚大学圣地亚哥分校计算机科学与工程系）
Emergent morpho-phonological representations in self-supervised speech models
Department of Neurological Surgery, University of California, San Francisco（加州大学旧金山分校神经外科系）；Department of Linguistics, University of Southern California（南加州大学语言学系）；Center for Computational Language Sciences, University of Southern California（南加州大学计算语言科学中心）
Multiplex model of mental lexicon reveals explosive learning in humans
Institute for Complex Systems Simulation, University of Southampton（南安普顿大学复杂系统模拟研究所）；Fondazione Bruno Kessler（布鲁诺·凯斯勒基金会）；Department of Electrical Engineering and Computer Science, University of Kansas（堪萨斯大学电气工程与计算机科学系）；School of Computer Science and Mathematics, Universitat Rovira i Virgili（罗维拉-威尔吉利大学计算机科学与数学学院）
