We introduce novel communication strategies in synchronous distributed Deep Learning consisting of decentralized gradient reduction orchestration and computational graph-aware grouping of gradient tensors. These new techniques produce an optimal overlap between computation and communication and result in near-linear scaling (0.93) of distributed training up to 27,600 NVIDIA V100 GPUs on the Summit Supercomputer. We demonstrate our gradient reduction techniques in the context of training a Fully Convolutional Neural Network to approximate the solution of a longstanding scientific inverse problem in materials imaging. The efficient distributed training on a dataset size of 0.5 PB, produces a model capable of an atomically-accurate reconstruction of materials, and in the process reaching a peak performance of 2.15(4) EFLOPS$_{16}$.
SRN-SZ: Deep Leaning-Based Scientific Error-bounded Lossy Compression with Super-resolution Neural Networks
University of California, Riverside(加州大学河滨分校);Argonne National Laboratory(阿贡国家实验室);University of Kentucky(肯塔基大学);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Florida State University(佛罗里达州立大学);Indiana University Bloomington(印第安纳大学伯明顿分校)
Breaking the Training Barrier of Billion-Parameter Universal Machine Learning Interatomic Potentials
State Key Lab of Processors, Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所 processors 国家重点实验室);University of Chinese Academy of Sciences(中国科学院大学);Independent Researcher(独立研究者);Sun Yat-Sen University(中山大学)
Uncertainties in Physics-informed Inverse Problems: The Hidden Risk in Scientific AI
Graduate School of Social and Data Sciences(社会科学与数据科学研究生院);Hitotsubashi University(立命馆大学);Department of Electrical and Electronic Engineering(电子工程系);Nihon University(日立大学)
D-Flow SGLD: Source-Space Posterior Sampling for Scientific Inverse Problems with Flow Matching
Sibley School of Mechanical and Aerospace Engineering, Cornell University(科尼尔大学机械与航空航天工程学院);Department of Aerospace and Mechanical Engineering, University of Notre Dame(诺丁汉大学航空航天与机械工程系)
Multimodal Scientific Learning Beyond Diffusions and Flows
Graduate Group in Applied Mathematics and Computational Science(应用数学与计算科学联合研究生组);Department of Computer and Information Science(计算机与信息科学系);Department of Mechanical Engineering and Applied Mechanics(机械工程与应用力学系)
Inverse Knowledge Search over Verifiable Reasoning: Synthesizing a Scientific Encyclopedia from a Long Chains-of-Thought Knowledge Base
Lanzhou Center for Theoretical Physics, Key Laboratory of Theoretical Physics of Gansu Province, Key Laboratory of Quantum Theory and Applications of MoE, Gansu Provincial Research Center for Basic Disciplines of Quantum Physics(兰州理论物理中心、甘肃省理论物理重点实验室、教育部量子理论与应用重点实验室、甘肃省量子物理基础学科研究省重点中心);Institute of Theoretical Physics, Chinese Academy of Sciences(中国科学院理论物理研究所);DP Technology(DP技术);Institute of Physics, Chinese Academy of Sciences(中国科学院物理研究所);Département d’Informatique, École normale supérieure(巴黎高等师范大学计算机系)
Applications of ML-Based Surrogates in Bayesian Approaches to Inverse Problems
Izmir University of Economics(伊兹密尔经济大学);Carnegie Melon University(卡内基梅隆大学);University of Warwick(华威大学);Case Western Reserve University(凯斯西储大学);University of California, Los Angeles(加利福尼亚大学洛杉矶分校);Advanced Micro Devices, Inc.(超威半导体公司)
Learning-guided Kansa collocation for forward and inverse PDEs beyond linearity
Department of Computer Science and Technology(计算机科学与技术系);Department of Applied Mathematics and Theoretical Physics(应用数学与理论物理系);University of Cambridge(剑桥大学)
Scientific Machine Learning of Flow Resistance Using Universal Shallow Water Equations with Differentiable Programming
Department of Civil and Environmental Engineering, Pennsylvania State University, University Park, Pennsylvania, USA 16802(宾夕法尼亚州立大学土木与环境工程系,大学公园,宾夕法尼亚州,美国 16802);Institute of Computational and Data Sciences, Pennsylvania State University, University Park, Pennsylvania, USA 16802(宾夕法尼亚州立大学计算与数据科学研究所,大学公园,宾夕法尼亚州,美国 16802)
Physics-Constrained Fine-Tuning of Flow-Matching Models for Generation and Inverse Problems
German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心);Department of Computer Science, University of Kaiserslautern–Landau (RPTU)(凯泽斯劳滕-兰道大学计算机科学系);Department of Mathematics, Imperial College London(伦敦帝国理工学院数学系)
A Stability Benchmark of Generative Regularizers for Inverse Problems
Helmholtz Imaging, Deutsches Elektronen-Synchrotron DESY, Germany(德意志电子同步辐射研究中心(Helmholtz Imaging));ENS Paris & Inria, France(巴黎高等师范学院与法国国家信息与自动化研究所);Chemnitz University of Technology, Germany(化学工业大学)