Pseudo-Asynchronous Local SGD: Robust and Efficient Data-Parallel Training
机构 * Mila ; Université de Montréal(蒙特利尔大学) ; University of Washington(华盛顿大学) ; The University of Hong Kong(香港大学) ; Microsoft(微软) ; NVIDIA(英伟达)
Comments Accepted to TMLR
期刊&会议
Transactions on Machine Learning Research · 期刊 · Machine Learning
机构 * Mila ; Université de Montréal(蒙特利尔大学) ; University of Washington(华盛顿大学) ; The University of Hong Kong(香港大学) ; Microsoft(微软) ; NVIDIA(英伟达)
Comments Accepted to TMLR
机构 * Department of Computer Science, Aalto University(艾尔沃斯大学计算机科学系) ; Department of Computer Science, Toronto Metropolitan University(多伦多 Metropolitan 大学计算机科学系) ; Department of Computer Science, University of Manchester(曼彻斯特大学计算机科学系) ; Intel Corporation(英特尔公司)
Comments 14 pages in the main text, 22 pages including references and supplementary materials. 3 figures and 3 tables in the main text, 6 figures and 3 tables in supplementary materials
Journal ref Transactions on Machine Learning Research (TMLR) (02/2025)
机构 * Mila - Quebec AI Institute(魁北克AI研究所) ; DIRO, Université de Montréal(蒙特利尔大学计算研究所) ; Université Laval(拉瓦尔大学) ; IID(智能与数据研究所) ; DEEL, IRT Saint Exupéry(DEEL,圣埃克苏佩里研究所) ; CIFAR AI Chair(CIFAR人工智能席位)
Comments Accepted to Transactions on Machine Learning Research (TMLR) 2025. 12 pages, 5 figures, 7 tables