Ethan Roland, Murat Cubuktepe, Erick Martinez, Stijn Servaes, Keenan Pepper, Mike Vaiana, Diogo Schwerz de Lucena, Judd Rosenblatt, Addie Foote, Cem Anil, Alex Cloud
AI developers face a dual-use dilemma. An AI capability that helps one user cure a disease can help another synthesize one. This dilemma could be resolved with access control, limiting dual-use AI capabilities to trusted deployments with a legitimate need. A gold standard for access control would be to serve separate models with different capabilities to different users. However, training and deploying multiple models is prohibitively expensive. To address this challenge, we propose gradient-routed auxiliary modules (GRAM), a pre-training method that adds modules to a neural network and selectively updates them to induce specialization. Ablating a module at inference time removes its capability from the network, approximating a model trained on filtered data. We evaluate GRAM on synthetic stories and realistic dual-use data spanning virology, cybersecurity, nuclear physics, and specialized code. These experiments show that GRAM disables targeted capabilities while preserving the rest, and resists their recovery under finetuning better than post-hoc unlearning. Most importantly, a Chinchilla-optimal scaling analysis from 50M to 5B parameters shows that the gap between data-filtered and full-data models widens with scale on removed capabilities but stays small on retained ones, and that GRAM closely tracks data filtering. GRAM's training cost is independent of the number of supported capability profiles, yielding a 5x reduction over data filtering in our 5-profile setting.
Information Flow Control in Machine Learning through Modular Model Architecture
Cornell University(康奈尔大学);University of Washington(华盛顿大学);FAIR at Meta(Meta FAIR实验室);Google DeepMind(谷歌DeepMind);Georgia Institute of Technology(佐治亚理工学院);Virginia Tech(弗吉尼亚理工大学);Pennsylvania State University(宾夕法尼亚州立大学);Intel(英特尔);NVIDIA(英伟达);Meta
A Multi-Center Study on the Adaptability of a Shared Foundation Model for Electronic Health Records
Program in Child Health Evaluative Sciences, The Hospital for Sick Children, Toronto, ON, 1 Canada(儿童健康评估科学计划,多伦多医院儿童医学中心,多伦多,加拿大);Stanford Center for Biomedical Informatics Research, Stanford University, Palo Alto, CA, USA(斯坦福大学生物医学信息学研究中心,斯坦福大学,帕洛阿尔托,美国);Universidad del Norte, Barranquilla, Columbia(北方大学,巴兰基利亚,哥伦比亚);Division of Haematology/Oncology, The Hospital for Sick Children, Toronto, ON, Canada(血液肿瘤科,多伦多医院儿童医学中心,多伦多,加拿大)
Position: Modular Memory is the Key to Continual Learning Agents
University of Bremen(不莱梅大学);Seoul National University(首尔国立大学);Computer Vision Center Barcelona(巴塞罗那计算机视觉中心);University of Florence(佛罗伦萨大学);Microsoft Research(微软研究院);HEC Montreal, Mila--Quebec AI Institute, Canada CIFAR AI Chair(蒙特利尔HEC学院、魁北克人工智能研究所、加拿大CIFAR人工智能 chair);Bielefeld University(比勒海姆大学);Georgia Institute of Technology(佐治亚理工学院);University of Rochester(罗切斯特大学);University of Texas at San Antonio(德克萨斯大学圣安东尼奥分校);Nankai University(南开大学);LUISS University(卢西亚诺大学);Stony Brook University(石溪大学);University of Tübingen(图宾根大学);University of Trento, FBK(特伦托大学,FBK);Tsinghua University(清华大学);University of Groningen(Groningen大学)