Graph neural networks (GNNs) have demonstrated their effectiveness in various tasks supported by their generalization capabilities. However, the current analysis of GNN generalization relies on the assumption that training and testing data are independent and identically distributed (i.i.d). This imposes limitations on the cases where a model mismatch exists when generating testing data. In this paper, we examine GNNs that operate on geometric graphs generated from manifold models, explicitly focusing on scenarios where there is a mismatch between manifold models generating training and testing data. Our analysis reveals the robustness of the GNN generalization in the presence of such model mismatch. This indicates that GNNs trained on graphs generated from a manifold can still generalize well to unseen nodes and graphs generated from a mismatched manifold. We attribute this mismatch to both node feature perturbations and edge perturbations within the generated graph. Our findings indicate that the generalization gap decreases as the number of nodes grows in the training graph while increasing with larger manifold dimension as well as larger mismatch. Importantly, we observe a trade-off between the generalization of GNNs and the capability to discriminate high-frequency components when facing a model mismatch. The most important practical consequence of this analysis is to shed light on the filter design of generalizable GNNs robust to model mismatch. We verify our theoretical findings with experiments on multiple real-world datasets.
PACMAN: PAC-style bounds accounting for the Mismatch between Accuracy and Negative log-loss
CSC-CONICET(阿根廷国家科学与技术研究理事会);Universidad de Buenos Aires(布宜诺斯艾利斯大学);Laboratoire des Signaux et Systèmes (L2S)(信号与系统实验室);Université Paris-Saclay(巴黎萨克雷大学);CNRS(法国国家科学研究中心);CentraleSupélec(中央理工-高等电力学院)
Split-KalmanNet: A Robust Model-Based Deep Learning Approach for SLAM
POSTECH(浦项科技大学);Kyungpook National University(庆北国立大学);Ben-Gurion University of the Negev(内盖夫本-古里安大学);Weizmann Institute of Science(魏茨曼科学研究所);Korea University(高丽大学)
Improved Diffusion-based Generative Model with Better Adversarial Robustness
Harbin Institute of Technology(哈尔滨工业大学);Renmin University of China(中国人民大学);Academy of Mathematics and Systems Science, Chinese Academy of Sciences(中国科学院数学与系统科学研究院);University of Chinese Academy of Sciences(中国科学院大学)
Adaptive Learning with Robust Generalization Guarantees
California Institute of Technology(加州理工学院);Hebrew University of Jerusalem(希伯来大学);Ben-Gurion University(本-古里安大学);Harvard University(哈佛大学);University of Pennsylvania(宾夕法尼亚大学)