Graph Neural Networks (GNNs) have achieved remarkable success across various graph-based tasks but remain highly sensitive to distribution shifts. In this work, we focus on a prevalent yet under-explored phenomenon in graph generalization, Minimal Shift Flip (MSF),where test samples that slightly deviate from the training distribution are abruptly misclassified. To interpret this phenomenon, we revisit MSF through the lens of Sharpness-Aware Minimization (SAM), which characterizes the local stability and sharpness of the loss landscape while providing a theoretical foundation for modeling generalization error. To quantify loss sharpness, we introduce the concept of Local Robust Radius, measuring the smallest perturbation required to flip a prediction and establishing a theoretical link between local stability and generalization. Building on this perspective, we further observe a continual decrease in the robust radius during training, indicating weakened local stability and an increasingly sharp loss landscape that gives rise to MSF. To jointly solve the MSF phenomenon and the intractability of radius, we develop an energy-based formulation that is theoretically proven to be monotonically correlated with the robust radius, offering a tractable and principled objective for modeling flatness and stability. Building on these insights, we propose an energy-driven generative augmentation framework (E2A) that leverages energy-guided latent perturbations to generate pseudo-OOD samples and enhance model generalization. Extensive experiments across multiple benchmarks demonstrate that E2A consistently improves graph OOD generalization, outperforming state-of-the-art baselines.
Fix the Loss, Not the Radius: Rethinking the Adversarial Perturbation of Sharpness-Aware Minimization
CSMT, Wenzhou-Kean University（温州肯恩大学计算机科学与技术学院）；International Frontier Interdisciplinary Research Institute, Wenzhou-Kean University（温州肯恩大学国际前沿交叉学科研究院）
University of Cambridge（剑桥大学）；Imperial College London（帝国理工学院）
Sharpness-aware minimization (SAM) aims to improve the generalisation of gradient-based learning by seeking out flat minima. In this work, we establish connections between SAM and Mean-Field Variational Inference (MFVI) of neural network parameters. We show that both these methods have interpretations as optimizing notions of flatness, and when using the reparametrisation trick, they both boil down to calculating the gradient at a perturbed version of the current mean parameter. This thinking motivates our study of algorithms that combine or interpolate between SAM and MFVI. We evaluate the pr…
Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Hong Kong University of Science and Technology（香港科技大学）；Stanford University（斯坦福大学）
Graph Neural Networks (GNNs) are widely applied for graph anomaly detection. As one of the key components for GNN design is to select a tailored spectral filter, we take the first step towards analyzing anomalies via the lens of the graph spectrum. Our crucial observation is the existence of anomalies will lead to the `right-shift' phenomenon, that is, the spectral energy distribution concentrates less on low frequencies and more on high frequencies. This fact motivates us to propose the Beta Wavelet Graph Neural Network (BWGNN). Indeed, BWGNN has spectral and spatial localized band-pass filte…
CMoS: Rethinking Time Series Prediction Through the Lens of Chunk-wise Spatial Correlations
Computer Network Information Center, CAS（中国科学院计算机网络信息中心）；University of Chinese Academy of Sciences（中国科学院大学）；Nanjing University（南京大学）；Tsinghua University（清华大学）
National Key Laboratory of Space Integrated Information System（空间信息集成国家重点实验室）；Institute of Software Chinese Academy of Sciences（中国科学院软件研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Thrust of Artificial Intelligence（人工智能研究所）；Hong Kong University of Science and Technology（香港科技大学）；Department of Computer Science and Engineering（计算机科学与工程系）；Amazon.com, Inc.（亚马逊公司）
SHAPO: Sharpness-Aware Policy Optimization for Safe Exploration
Université de Montréal（蒙特利尔大学）；Mila - Québec AI Institute（蒙特利尔人工智能研究所）；Université Laval（拉瓦尔大学）；IID - Institut Intelligence et Données（智能与数据研究所）；LawZero；CIFAR AI Chair（CIFAR人工智能主席）
Neural networks appear to have mysterious generalization properties when using parameter counting as a proxy for complexity. Indeed, neural networks often have many more parameters than there are data points, yet still provide good generalization performance. Moreover, when we measure generalization as a function of parameters, we see double descent behaviour, where the test error decreases, increases, and then again decreases. We show that many of these properties become understandable when viewed through the lens of effective dimensionality, which measures the dimensionality of the parameter…
Rethinking Robustness in Machine Learning: A Posterior Agreement Approach
Department of Computer Science, ETH Zurich（苏黎世联邦理工学院计算机科学系）；Faculty of Engineering, Free University of Bozen-Bolzano（博兹纳-博尔扎诺自由大学工程学院）；Department of Computer Science, University of Genoa（热那亚大学计算机科学系）；CISPA Helmholtz Center for Information Security（信息安全赫尔姆霍茨研究中心）
University of Cambridge（剑桥大学）；Ecole Polytechnique de Montréal（蒙特利尔理工学院）；Loyola Marymount University（洛约拉马利蒙特大学）
End-to-end training of neural network solvers for graph combinatorial optimization problems such as the Travelling Salesperson Problem (TSP) have seen a surge of interest recently, but remain intractable and inefficient beyond graphs with few hundreds of nodes. While state-of-the-art learning-driven approaches for TSP perform closely to classical solvers when trained on trivially small sizes, they are unable to generalize the learnt policy to larger instances at practical scales. This work presents an end-to-end neural combinatorial optimization pipeline that unifies several recent papers in o…
Models trained in federated settings often suffer from degraded performances and fail at generalizing, especially when facing heterogeneous scenarios. In this work, we investigate such behavior through the lens of geometry of the loss and Hessian eigenspectrum, linking the model's lack of generalization capacity to the sharpness of the solution. Motivated by prior studies connecting the sharpness of the loss surface and the generalization gap, we show that i) training clients locally with Sharpness-Aware Minimization (SAM) or its adaptive version (ASAM) and ii) averaging stochastic weights (SW…
