Full-complexity Earth system models (ESMs) are computationally very expensive, limiting their use in exploring the climate outcomes of multiple emission pathways. More efficient emulators that approximate ESMs can directly map emissions onto climate outcomes, and benchmarks are being used to evaluate their accuracy on standardized tasks and datasets. We investigate a popular benchmark in data-driven climate emulation, ClimateBench, on which deep learning-based emulators are currently achieving the best performance. We compare these deep learning emulators with a linear regression-based emulator, akin to pattern scaling, and show that it outperforms the incumbent 100M-parameter deep learning foundation model, ClimaX, on 3 out of 4 regionally-resolved climate variables, notably surface temperature and precipitation. While emulating surface temperature is expected to be predominantly linear, this result is surprising for emulating precipitation. Precipitation is a much more noisy variable, and we show that deep learning emulators can overfit to internal variability noise at low frequencies, degrading their performance in comparison to a linear emulator. We address the issue of overfitting by increasing the number of climate simulations per emission pathway (from 3 to 50) and updating the benchmark targets with the respective ensemble averages from the MPI-ESM1.2-LR model. Using the new targets, we show that linear pattern scaling continues to be more accurate on temperature, but can be outperformed by a deep learning-based technique for emulating precipitation. We publish our code and data at github.com/blutjens/climate-emulator.
Assessing the Robustness of Climate Foundation Models under No-Analog Distribution Shifts
Centre for Artificial Intelligence(人工智能中心);Department of Computer Science(计算机科学系);University College London(伦敦大学学院);Division of Emerging Interdisciplinary Areas(新兴交叉学科领域);Hong Kong University of Science and Technology(香港科技大学);University of Oxford(牛津大学)
ClimateSet: A Large-Scale Climate Model Dataset for Machine Learning
Mila Quebec AI Institute(魁北克人工智能研究所);McGill University(麦吉尔大学);University of Osnabrück(奥斯纳布吕克大学);University of Montreal(蒙特利尔大学);Intel Labs(英特尔实验室);German Aerospace Center(德国航空航天中心);Technische Universität Berlin(柏林工业大学)
UHI-Bench: Benchmarking Dual-Source Urban Heat Island Modeling Across Cities in Diverse Climate Regimes
Technical University of Munich(慕尼黑工业大学);Hong Kong Institute of Science & Innovation, Chinese Academy of Sciences(中国科学院香港创新研究院);RWTH Aachen University(亚琛工业大学);Hainan Bielefeld University of Applied Sciences(海南比勒费尔德应用科学大学);Heilbronn Data Science Center(海尔布隆数据科学中心);Munich Data Science Institute(慕尼黑数据科学研究所)
How Can Machine Learning Emulators Best Support Climate Science?
Cluster of Excellence Machine Learning University of Tübingen(图宾根大学机器学习卓越中心);Institute for Atmospheric and Climate Science ETH Zurich(大气与气候科学研究所,苏黎世联邦理工学院)
ClimateBench v2.0: Probabilistic Climate Model Benchmarking
Scripps Institution of Oceanography, University of California San Diego(加州大学圣地亚哥分校斯克里普斯海洋研究所);Deutsches Zentrum für Luft- und Raumfahrt e.V. (DLR)(德国航空航天中心);Schmidt Sciences(施密特科学中心);Jet Propulsion Laboratory, California Institute of Technology(加州理工学院喷气推进实验室);Allen Institute for Artificial Intelligence(艾伦人工智能研究所);Lawrence Livermore National Laboratory(劳伦斯利弗莫尔国家实验室);University of Colorado Boulder(科罗拉多大学博尔德分校);Lawrence Berkeley National Laboratory(劳伦斯伯克利国家实验室);University of East Anglia(东英吉利大学);Columbia University(哥伦比亚大学);Met Office(英国气象局);Google Research(谷歌研究院);Alfred Wegener Institute(阿尔弗雷德·魏格纳研究所);National Center for Atmospheric Research(美国国家大气研究中心);Otto von Guericke University Magdeburg(马格德堡奥托·冯·格里克大学);University of Bristol(布里斯托尔大学);University of California, Los Angeles(加州大学洛杉矶分校);NVIDIA Research(英伟达研究院);NASA Goddard Institute for Space Studies(美国国家航空航天局戈达德空间研究所);California Institute of Technology(加州理工学院)
Building Interpretable Climate Emulators for Economics
Institute of Computing, Università della Svizzera italiana(瑞士大学计算研究所);Institute for Atmospheric and Climate Science, ETHZ(大气与气候科学研究所);ifo Institute, Ludwig-Maximilian University of Munich(ifo研究所,慕尼黑莱布尼茨大学);Department for Banking and Finance, University of Zürich(金融学院,苏黎世大学);Department of Economics, University of Lausanne(经济学系,洛桑大学);Grantham Research Institute, London School of Economics(格兰塔姆研究所,伦敦政治经济学院)
High-Resolution Climate Projections Using Diffusion-Based Downscaling of a Lightweight Climate Emulator
Information Sciences and Technology, The Pennsylvania State University(宾夕法尼亚州立大学信息科学与技术系);Department of Applied Mathematics, University of California, Santa Cruz(加州大学圣克鲁兹分校应用数学系);Allen Institute for Artificial Intelligence (AI2)(人工智能研究所);School of Mechanical Engineering, Purdue University(普渡大学机械工程学院)
MERCURY: A fast and versatile multi-resolution based global emulator of compound climate hazards
University of Oxford(牛津大学);Montreal Polytechnique(蒙特利尔理工学院);Columbia Climate School(哥伦比亚气候学院);International Institute for Applied Systems Analysis(国际应用系统分析研究所);Universität Leipzig(莱比锡大学);IRI THESys;Humboldt-Universität zu Berlin(柏林洪堡大学);Institute Pierre-Simone-Laplace(皮埃尔-西蒙·拉普拉斯研究所)
MobiFlow: Real-World Mobile Agent Benchmarking through Trajectory Fusion
Institute of Parallel and Distributed Systems (IPADS), Shanghai Jiao Tong University(上海交通大学并行与分布式系统研究所(IPADS));Department of Precision Instrument, Tsinghua University(清华大学精密仪器系);National Innovation Institute of High-end Smart Appliances(国家高端智能家电创新研究院)
Department of Aeronautics and Astronautics, Massachusetts Institute of Technology(航空与航天系,麻省理工学院);Center for Sustainability Science and Strategy, Massachusetts Institute of Technology(可持续科学与战略中心,麻省理工学院);Department of Earth, Atmospheric, and Planetary Sciences, Massachusetts Institute of Technology(地球、大气与行星科学系,麻省理工学院);Brahmal Vasudevan Institute for Sustainable Aviation, Department of Aeronautics, Imperial College London(可持续航空研究所,帝国理工学院伦敦校区);Institute for Data, Systems, and Society, Massachusetts Institute of Technology(数据、系统与社会研究所,麻省理工学院)
LUCIE-3D: A three-dimensional climate emulator for forced responses
Pennsylvania State University(宾夕法尼亚州立大学);Allen Insitute of Artificial Intelligence(人工智能 Allen 机构);Argonne National Laboratory(阿贡国家实验室);University of California, Santa Cruz(加州大学圣克鲁兹分校)
Decision-oriented benchmarking to transform AI weather forecast access: Application to the Indian monsoon
Department of the Geophysical Sciences, University of Chicago(地球科学系,芝加哥大学);Development Innovation Lab, University of Chicago(创新发展实验室,芝加哥大学);Development Innovation Lab India, University of Chicago Trust(印度创新发展实验室,芝加哥大学信托);Data Science Institute, University of Chicago(数据科学研究所,芝加哥大学);Department of Earth and Planetary Science, University of California, Berkeley(地球与行星科学系,加州大学伯克利分校);NorthWest Research Associates, Boulder, CO(北西部研究协会,博尔德,科罗拉多);Department of Economics, University of Chicago(经济学系,芝加哥大学);Harris School of Public Policy, University of Chicago(哈里斯公共政策学院,芝加哥大学);Climate and Ecosystem Sciences Division, Lawrence Berkeley National Laboratory(气候与生态系统科学部,伯克利国家实验室)
GUI-CC: Benchmarking Contextual Consistency of GUI World Models as Agent Environments
Yale University(耶鲁大学);Zhejiang University(浙江大学);China University of Geosciences(中国地质大学);DAMO Academy, Alibaba Group(阿里巴巴达摩院);Tongji University(同济大学);University of California, San Diego(加利福尼亚大学圣迭戈分校)
ThousandWorlds: A benchmark for climate emulation of potentially habitable exoplanets
University of Cambridge(剑桥大学);University of Oxford(牛津大学);University of Colorado Boulder(科罗拉多大学博尔德分校);University of Bristol(布里斯托大学);Purdue University(普渡大学);University of Exeter(埃克塞特大学)
AI-ready Snow Radar Echogram Dataset (SRED) for climate change monitoring
Center for Remote Sensing and Integrated Systems (CReSIS) University of Kansas(远程感知与集成系统中心(CReSIS)大学堪萨斯大学);Department of Information Systems University of Maryland Baltimore County(信息系统系马里兰大学巴尔的摩县分校);Department of Computer Science and Engineering Lehigh University(计算机科学与工程系莱恩大学)
Digital Typhoon: Long-term Satellite Image Dataset for the Spatio-Temporal Modeling of Tropical Cyclones
National Institute of Informatics(国立情报学研究所);Yokohama National University(横滨国立大学);University of Southern California(南加州大学);Grenoble-INP, Ensimag(格勒诺布尔综合理工学院,恩西马格);Université Clermont Auvergne, ISIMA(克莱蒙奥弗涅大学,ISIMA);Google DeepMind(谷歌DeepMind)
Samudra 2: Scaling Ocean Emulators across Resolutions
Courant Institute School of Mathematics, Computing, and Data Science, New York University(Courant学院数学、计算与数据科学系,纽约大学);Open Athena AI Foundation, Inc.(开放Athena人工智能基金会);Program in Atmospheric and Oceanic Sciences, Princeton University(大气与海洋科学项目,普林斯顿大学)
Building high accuracy emulators for scientific simulations with deep neural architecture search
University of Oxford(牛津大学);University of Rochester(罗切斯特大学);Yale University(耶鲁大学);University of Seville(塞维利亚大学);Max-Planck-Institut für Plasmaphysik(马克斯·普朗克等离子体物理研究所)
Towards Causal Representations of Climate Model Data
Mila –Quebec AI Institute(米拉-魁北克人工智能研究所);McGill University(麦吉尔大学);Osnabrück University(奥斯纳布吕克大学);Université de Montréal(蒙特利尔大学);Intel Labs(英特尔实验室);Karlsruhe Institute of Technology(卡尔斯鲁厄理工学院);Columbia University(哥伦比亚大学);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)
Stochastic Emulation of a Fully Coupled Preindustrial E3SMv3 Simulation
Allen Institute for Artificial Intelligence (Ai2)(艾伦人工智能研究所);Pacific Northwest National Laboratory(太平洋西北国家实验室);Los Alamos National Laboratory(洛斯阿拉莫斯国家实验室);Brookhaven National Laboratory(布鲁克海文国家实验室);Lawrence Livermore National Laboratory(劳伦斯利弗莫尔国家实验室)