Large language models possess general linguistic abilities but acquire language less efficiently than humans. This study proposes a method for integrating the developmental characteristics of working memory during the critical period, a stage when human language acquisition is particularly efficient, into the training process of language models. The proposed method introduces a mechanism that initially constrains working memory during the early stages of training and gradually relaxes this constraint in an exponential manner as learning progresses. Targeted syntactic evaluation shows that the proposed method outperforms conventional methods without memory constraints or with static memory constraints. These findings not only provide new directions for designing data-efficient language models but also offer indirect evidence supporting the role of the developmental characteristics of working memory as the underlying mechanism of the critical period in language acquisition.
Findings of the BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corpora
ETH Zürich（苏黎世联邦理工学院）；Northeastern University（东北大学）；Technion（以色列理工学院）；MIT（麻省理工学院）；IBM Research（IBM研究院）；MLCommons；Meta AI (FAIR)（元人工智能研究院（FAIR））；University of Washington（华盛顿大学）；New York University（纽约大学）
BabyBabelLM: A Multilingual Benchmark of Developmentally Plausible Training Data
University of Groningen（Groningen 大学）；Aix Marseille University（阿尔勒-马赛大学）；Nara Institute of Science and Technology（神户科学技术大学）；Bielefeld University（比勒菲尔德大学）；University of Colorado Boulder（科罗拉多大学波德分校）；University of Cambridge（剑桥大学）；KAIST（韩国科学技术院）；University of Cape Town（开普敦大学）；City University of Hong Kong（香港城市大学）；HiTZ, University of the Basque Country（基督教大学）；Columbia University（大都会大学）；SomosNLP；EPFL（瑞士联邦理工学院）；University of Tehran（德黑兰大学）；University of Texas at Austin（德克萨斯大学奥斯汀分校）；Ukrainian Catholic University（乌克兰天主教大学）；Shanghai Jiao Tong University（上海交通大学）；University of California San Diego（加州大学圣地亚哥分校）；MIT（麻省理工学院）；MIT-IBM Watson AI Lab（麻省理工-IBM Watson AI 实验室）
Findings of the Second BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corpora
New York University（纽约大学）；Northeastern University（东北大学）；Technion（以色列理工学院）；Meta AI (FAIR)（元人工智能研究院（FAIR））；IBM Research（IBM研究院）；MIT（麻省理工学院）；ETH Zürich（苏黎世联邦理工学院）；Georgetown University（乔治城大学）
BabySLM: language-acquisition-friendly benchmark of self-supervised spoken language models
LSCP；ENS（巴黎高等师范学院）；EHESS（高等社会科学研究学院）；CNRS（法国国家科学研究中心）；PSL University（PSL大学）；Meta AI Research（Meta人工智能研究院）；Tampere University（坦佩雷大学）；IRIT；INRIA（法国国家信息与自动化研究院）
Lexicon-Level Contrastive Visual-Grounding Improves Language Modeling
McGovern Institute for Brain Research, MIT（麻省理工学院麦戈文脑科学研究所）；Department of Brain and Cognitive Sciences, MIT（麻省理工学院脑与认知科学系）；The Program in Speech and Hearing Bioscience and Technology, Harvard University（哈佛大学言语与听觉生物科学与技术项目）；CSAIL, MIT（麻省理工学院计算机科学与人工智能实验室）
