Nanyang Technological University（南洋理工大学）；University of Cambridge（剑桥大学）；Loyola Marymount University（洛约拉马利蒙特大学）；Mila（米拉人工智能研究所）；University of Montréal（蒙特利尔大学）；National University of Singapore（新加坡国立大学）
Vijay Prakash Dwivedi, Chaitanya K. Joshi, Anh Tuan Luu, Thomas Laurent, Yoshua Bengio, Xavier Bresson
In the last few years, graph neural networks (GNNs) have become the standard toolkit for analyzing and learning from data on graphs. This emerging field has witnessed an extensive growth of promising techniques that have been applied with success to computer science, mathematics, biology, physics and chemistry. But for any successful field to become mainstream and reliable, benchmarks must be developed to quantify progress. This led us in March 2020 to release a benchmark framework that i) comprises of a diverse collection of mathematical and real-world graphs, ii) enables fair model comparison with the same parameter budget to identify key architectures, iii) has an open-source, easy-to-use and reproducible code infrastructure, and iv) is flexible for researchers to experiment with new theoretical ideas. As of December 2022, the GitHub repository has reached 2,000 stars and 380 forks, which demonstrates the utility of the proposed open-source framework through the wide usage by the GNN community. In this paper, we present an updated version of our benchmark with a concise presentation of the aforementioned framework characteristics, an additional medium-sized molecular dataset AQSOL, similar to the popular ZINC, but with a real-world measured chemical target, and discuss how this framework can be leveraged to explore new GNN designs and insights. As a proof of value of our benchmark, we study the case of graph positional encoding (PE) in GNNs, which was introduced with this benchmark and has since spurred interest of exploring more powerful PE for Transformers and GNNs in a robust experimental setting.
How much progress have we made in neural network training? A New Evaluation Protocol for Benchmarking Optimizers
Department of Computer Science, UCLA（加州大学洛杉矶分校计算机科学系）；Department of Computer Science, National University of Singapore（新加坡国立大学计算机科学系）；Google Research（谷歌研究院）
RWTH Aachen University（亚琛RWTH大学）；University of Oxford（牛津大学）；Mila – Quebec AI Institute（魁北克AI研究所）；Technion - Israel Institute of Technology（技术学院-以色列理工学院）；ETAS Research University of Stuttgart（斯图加特大学ETAS研究所）；Google Research（谷歌研究）；Microsoft Research（微软研究院）
University of Kansas（堪萨斯大学）；Polytechnic University of Catalonia（加泰罗尼亚理工大学）；Indian Institute of Technology Bhilai（比莱印度理工学院）；University of Florida（佛罗里达大学）；University of Alabama（阿拉巴马大学）；Software Engineering Institute, Carnegie Mellon University（卡内基梅隆大学软件工程研究所）
B-XAIC Dataset: Benchmarking Explainable AI for Graph Neural Networks Using Chemical Data
University of Edinburgh（爱丁堡大学）；Jagiellonian University（雅盖隆大学）；Jagiellonian University, Faculty of Mathematics and Computer Science（雅盖隆大学数学与计算机科学学院）；Jagiellonian University, Faculty of Chemistry（雅盖隆大学化学学院）；Ardigen SA（Ardigen公司）
Benchmarking Graph Neural Networks in Solving Hard Constraint Satisfaction Problems
Department of Computing Sciences, Bocconi University（博科尼大学计算机科学系）；Bocconi Institute for Data Science and Analytics (BIDSA)（博科尼数据科学与分析研究所）；Department of Physics and Astronomy and INFN, University of Florence（佛罗伦萨大学物理与天文学系及意大利国家核物理研究所）；Dipartimento di Fisica, Sapienza Università di Roma（罗马萨皮恩扎大学物理系）；CNR - Nanotec, unità di Roma, P.le Aldo Moro 5, 00185 Rome, Italy（意大利罗马国家科研委员会纳米技术单元，Aldo Moro大道5号，00185罗马，意大利）；Department of Theoretical Physics, Physics Faculty, University of Havana（古巴哈瓦那大学理论物理系）；INFN, sezione di Roma1, P.le Aldo Moro 5, 00185 Rome, Italy（意大利罗马1研究所，Aldo Moro大道5号，00185罗马，意大利）
Benchmarking Graph Representations and Graph Neural Networks for Multivariate Time Series Classification
School of Computer Science & Software Engineering, Shenzhen University（深圳大学计算机与软件学院）；Shenzhen Institute of Artificial Intelligence and Robotics for Society（深圳市人工智能与机器人研究院）；Guangdong Provincial Key Laboratory of Intelligent Information Processing（广东省智能信息处理重点实验室）；HBUG Lab, University of Exeter（埃克塞特大学HBUG实验室）
Benchmarking Probabilistic Time Series Forecasting Models on Neural Activity
Department of Applied Mathematics, University of Washington（应用数学系，华盛顿大学）；Department of Neurobiology and Biophysics, University of Washington（神经生物学与生物物理学系，华盛顿大学）；Allen Institute for Brain Science（脑科学研究所）；Department of Electrical and Computer Engineering, University of Washington（电气与计算机工程系，华盛顿大学）
Benchmarking Long Roll-outs of Auto-regressive Neural Operators for the Compressible Navier-Stokes Equations with Conserved Quantity Correction
Graduate Researcher, Computer Science and Engineering（计算机科学与工程研究生）；Postdoctoral Researcher, Mechanical and Aerospace Engineering（机械与航空航天工程博士后）；Glenn Chair Professor, Mechanical and Aerospace Engineering（机械与航空航天工程 Glenn 教授）；Professor, Computer Science and Engineering（计算机科学与工程教授）
An extensible Benchmarking Graph-Mesh dataset for studying Steady-State Incompressible Navier-Stokes Equations
Sorbonne Université（索邦大学）；CNRS（法国国家科学研究中心）；École Normale Supérieure Paris Saclay（巴黎萨克雷高等师范学院）；Extrality（Extrality公司）；Criteo AI Lab（Criteo人工智能实验室）
\emph{Geometric Deep Learning}（GDL）的最新进展显示了其提供强大的数据驱动模型的潜力。这为探索从Graph-Mesh数据学习由\emph{Partial Differential Equations}（PDE）管理的物理系统的新方法提供了动力。然而，尽管做出了努力并取得了最新成就，但仍有几个研究方向尚未探索，进展仍然远远不能满足现实世界现象的物理要求。主要障碍之一是缺乏基准数据集和共同的物理评估协议。在本文中，我们提出了一个2-D图形网格数据集，以研究在高雷诺数制度的翼型气流（从$10^6$和超越）。我们还介绍了在翼型上的应力的度量，以评估GDL模型的重要物理量。此外，我们提供广泛的GDL基线。
OpenBreastUS: Benchmarking Neural Operators for Wave Imaging Using Breast Ultrasound Computed Tomography
Peking University（北京大学）；King Abdullah University of Science and Technology（阿卜杜拉国王科技大学）；Institute of Acoustics, Chinese Academy of Sciences（中国科学院声学研究所）；Yanqi Lake Beijing Institute of Mathematical Sciences and Applications（雁栖湖北京数学科学与应用研究院）；DP Technology（深势科技）；National Biomedical Imaging Center, Peking University（北京大学国家生物医学成像中心）
