Properties of interest for crystals and molecules, such as band gap, elasticity, and solubility, are generally related to each other: they are governed by the same underlying laws of physics. However, when state-of-the-art graph neural networks attempt to predict multiple properties simultaneously (the multi-task learning (MTL) setting), they frequently underperform a suite of single property predictors. This suggests graph networks may not be fully leveraging these underlying similarities. Here we investigate a potential explanation for this phenomenon: the curvature of each property's loss surface significantly varies, leading to inefficient learning. This difference in curvature can be assessed by looking at spectral properties of the Hessians of each property's loss function, which is done in a matrix-free manner via randomized numerical linear algebra. We evaluate our hypothesis on two benchmark datasets (Materials Project (MP) and QM8) and consider how these findings can inform the training of novel multi-task learning models.
Lightweight Geometric Adaptation for Training Physics-Informed Neural Networks
Department of Computational Applied Mathematics and Operations Research(计算应用数学与运筹学部门);School of Data Science(数据科学学院);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳))
Label-informed Graph Structure Learning for Node Classification
Center for Research on Intelligent Perception and Computing, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所智能感知与计算研究中心);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院)
Informed along the road: roadway capacity driven graph convolution network for network-wide traffic prediction
Tandon School of Engineering, New York University(纽约大学工程学院);State Key Laboratory of Internet of Things for Smart City, University of Macau(澳门大学物联网智能城市国家重点实验室)
Physics-Informed Graph Neural Networks to Reconstruct Local Fields Considering Finite Strain Hyperelasticity
Univ. Bordeaux, CNRS, Bordeaux INP, I2M, UMR 5295(波尔多大学、国家科学研究中心、波尔多INP、I2M、UMR 5295);Univ. Bordeaux, CNRS, Bordeaux INP, LaBRI, UMR 5800(波尔多大学、国家科学研究中心、波尔多INP、LaBRI、UMR 5800);Arts et Metiers Institute of Technology, CNRS, Bordeaux INP, I2M, UMR 5295(艺术与技术学院技术学院、国家科学研究中心、波尔多INP、I2M、UMR 5295)
FIMP: Foundation Model-Informed Message Passing for Graph Neural Networks
Yale University(耶鲁大学);Brown University(布朗大学);New York University(纽约大学);MIT(麻省理工学院);ETH Zurich(苏黎世联邦理工学院);Idaho State University(爱达荷州立大学);University of Massachusetts Boston(马萨诸塞大学波士顿分校);Baylor College of Medicine(贝勒医学院);EPFL(洛桑联邦理工学院);University of Southern California(南加州大学)
Visualizing High-Dimensional Graph Embeddings via Informed Multi-View Projections
Khoury College of Computer Sciences, Northeastern University(东北大学库里计算机科学学院);School of Computation, Information and Technology, Technical University of Munich(慕尼黑工业大学计算、信息与技术学院)
Reducing Oversmoothing through Informed Weight Initialization in Graph Neural Networks
School of Electrical Engineering and Computer Science, National Technical University of Athens(雅典国立技术大学电气与计算机工程学院);National Center for Scientific Research “Demokritos”(德谟克利特国家科学研究中心)
RBF-MGN:Solving spatiotemporal PDEs with Physics-informed Graph Neural Network
College of Aerospace Science and Engineering, National University of Defense Technology(国防科技大学航天科学与工程学院);National Innovation Institute of Defense Technology, Chinese Academy of Military Science(中国人民解放军军事科学院国防科技创新研究院)
Aragon Institute of Engineering Research (I3A)(阿拉贡工程研究所);University of Zaragoza(萨拉戈萨大学);Higher Technical School of Industrial Engineering, Polytechnic University of Madrid(马德里理工大学工业工程高等技术学院);Polytechnic University of Madrid(马德里理工大学);ENSAM Institute of Technology(法国国立高等工程技术学院)
Physics-Informed Heterogeneous Graph Neural Networks for DC Blocker Placement
Argonne National Laboratory(阿贡国家实验室);Oak Ridge National Laboratory(橡树岭国家实验室);Lawrence Berkeley National Laboratory(劳伦斯伯克利国家实验室);University of California, Berkeley(加州大学伯克利分校);Los Alamos National Laboratory(洛斯阿拉莫斯国家实验室)
Curvature-Aware Optimization for High-Accuracy Physics-Informed Neural Networks
Department of Information Engineering and Computer Science, University of Trento(特伦托大学信息工程与计算机科学系);Division of Applied Mathematics, Brown University(布朗大学应用数学部);Center for Biomedical Engineering, Brown University(布朗大学生物医学工程中心);Institute of Mathematics, TU Berlin(柏林工业大学数学研究所);ETH Zurich(苏黎世联邦理工学院)
HypNO: A Graph-Based Neural Operator with Physics-Informed Message Passing for Hyperbolic Conservation Laws
Laboratory of Computer Science (LIX), École Polytechnique(计算机科学实验室(LIX),巴黎综合理工学院);Department of Electrical Engineering and Computer Sciences, University of California, Berkeley(电气工程与计算机科学系,加利福尼亚大学伯克利分校)