Foundation models have revolutionized language modeling, while whether this success is replicated in scientific computing remains unexplored. We present OmniArch, the first prototype aiming at solving multi-scale and multi-physics scientific computing problems with physical alignment. We addressed all three challenges with one unified architecture. Its pre-training stage contains a Fourier Encoder-decoder fading out the disharmony across separated dimensions and a Transformer backbone integrating quantities through temporal dynamics, and the novel PDE-Aligner performs physics-informed fine-tuning under flexible conditions. As far as we know, we first conduct 1D-2D-3D united pre-training on the PDEBench, and it sets not only new performance benchmarks for 1D, 2D, and 3D PDEs but also demonstrates exceptional adaptability to new physics via in-context and zero-shot learning approaches, which supports realistic engineering applications and foresight physics discovery.
University of California Santa Barbara（加州大学圣巴巴拉分校）
There has been a lot of recent interest in adopting machine learning methods for scientific and engineering applications. This has in large part been inspired by recent successes and advances in the domains of Natural Language Processing (NLP) and Image Classification (IC). However, scientific and engineering problems have their own unique characteristics and requirements raising new challenges for effective design and deployment of machine learning approaches. There is a strong need for further mathematical developments on the foundations of machine learning methods to increase the level of r…
STEP: Scientific Time-Series Encoder Pretraining via Cross-Domain Distillation
Shanghai Aritificial Intelligence Laboratory（上海人工智能实验室）；Tsinghua University（清华大学）；Harbin Institute of Technology（哈尔滨工业大学）；Sichuan University（四川大学）；University of Cambridge（剑桥大学）
A foundation model for energy and radiation systems built on heterogeneous scientific interfaces
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Indian Institute of Technology Delhi（印度理工学院德里分校）；Yardi School of Artificial Intelligence (ScAI)（雅迪人工智能学院）；National Center for Supercomputing Applications (NCSA)（国家超级计算应用中心）
Scalable Cross-Facility Federated Learning for Scientific Foundation Models on Multiple Supercomputers
Mathematics and Computer Science Division, Argonne National Laboratory（阿贡国家实验室数学与计算机科学 division）；Data Science and Learning Division, Argonne National Laboratory（阿贡国家实验室数据科学与学习 division）；Department of Computer Science, University of Chicago（芝加哥大学计算机科学系）
SAFE-SVD: Sensitivity-Aware Fidelity-Enforcing SVD for Physics Foundation Models
AI Centre, University College London（伦敦大学学院人工智能中心）；University College London（伦敦大学学院）；Central South University（中南大学）；University of Massachusetts at Amherst（马萨诸塞大学阿姆赫斯特分校）
Physics-Guided Multimodal Transformers are the Necessary Foundation for the Next Generation of Meteorological Science
School of Artificial Intelligence, Beijing University of Posts；Huawei Noah's Ark Lab；Department of Earth System Science, Tsinghua University；College of Computing \& Data Science, Nanyang Technological University；State Key Laboratory of Networking；Switching Technology, Beijing University of Posts
Earth Science Foundation Models: From Perception to Reasoning and Discovery
Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University（数据科学与人工智能系，香港理工大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）
HGNet: Scalable Foundation Model for Automated Knowledge Graph Generation from Scientific Literature
BASIRA Lab, Imperial-X (I-X) and Department of Computing（BASIRA实验室、Imperial-X（I-X）和计算系）；Imperial College London, London, United Kingdom（伦敦帝国理工学院，伦敦，英国）
Deep Learning Foundation and Pattern Models: Challenges in Hydrological Time Series
Computer Science（计算机科学）；University of Virginia（弗吉尼亚大学）；College of Computing（计算学院）；Georgia Institute of Technology（佐治亚理工学院）；Biocomplexity Institute（生物复杂性研究所）
Defining Foundation Models for Computational Science: A Call for Clarity and Rigor
Center for Applied Scientific Computing（应用科学计算中心）；Lawrence Livermore National Laboratory（劳伦斯利弗莫尔国家实验室）；Intelligence and Interaction Research Center（智能交互研究中心）；Korea Institute of Science and Technology（韩国科学技术院）；Department of Mathematics（数学系）；Virginia Tech（弗吉尼亚理工大学）；Quantitative Modeling and Software Engineering（定量建模与软件工程）；Sandia National Laboratories（桑地亚国家实验室）；Department of Aerospace Engineering（航空航天工程系）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Dept. Computational Applied Mathematics and Operations Research（计算应用数学与运筹学系）；Rice University（莱斯大学）
Statistically Meaningful Geometry and Gauge Symmetry Breaking: A Geometric Foundation for Scientific Discovery and Intelligence Emergence
Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）；AMSS Center for Forecasting Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院预测科学研究中心）；Beijing Institute of Mathematical Sciences and Applications (BIMSA)（北京国际数学研究中心）；Department of Statistics, London School of Economics and Political Science（伦敦政治经济学院统计系）；Department of Statistics and Data Science, Tsinghua University（清华大学统计与数据科学系）；Paula and Gregory Chow Institute for the Studies in Economics, Xiamen University（厦门大学经济研究所）；Yau Mathematical Sciences Center, Tsinghua University（清华大学丘成桐数学科学中心）
UniGenX: a unified generative foundation model that couples sequence, structure and function to accelerate scientific design across proteins, molecules and materials
Microsoft Research AI for Science（微软研究院人工智能科学部门）；School of Electronic and Computer Engineering, Peking University（北京大学电子与计算机工程学院）；School of AI4S, Shenzhen Graduate School, Peking University（北京大学深圳研究生院人工智能学院）；Beijing Institute of Mathematical Sciences and Applications（北京数学科学研究院）；School of Computer Science and Technology, Huazhong University of Science and Technology（华中科技大学计算机科学与技术学院）；School of Artificial Intelligence and Automation, Huazhong University of Science and Technology（华中科技大学人工智能与自动化学院）；Department of Automation, Tsinghua University（清华大学自动化系）；Yau Mathematical Sciences Center, Tsinghua University（清华大学数学科学中心）
SSL4EO-L: Datasets and Foundation Models for Landsat Imagery
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Technical University of Munich（慕尼黑工业大学）；University of Texas at San Antonio（德克萨斯大学圣安东尼奥分校）；German Aerospace Center（德国航空航天中心）；Microsoft AI for Good Research Lab（微软人工智能向善研究实验室）
