Scientific machine learning has seen significant progress with the emergence of operator learning. However, existing methods encounter difficulties when applied to problems on unstructured grids and irregular domains. Spatial graph neural networks utilize local convolution in a neighborhood to potentially address these challenges, yet they often suffer from issues such as over-smoothing and over-squashing in deep architectures. Conversely, spectral graph neural networks leverage global convolution to capture extensive features and long-range dependencies in domain graphs, albeit at a high computational cost due to Eigenvalue decomposition. In this paper, we introduce a novel approach, referred to as Spatio-Spectral Graph Neural Operator (Sp$^2$GNO) that integrates spatial and spectral GNNs effectively. This framework mitigates the limitations of individual methods and enables the learning of solution operators across arbitrary geometries, thus catering to a wide range of real-world problems. Sp$^2$GNO demonstrates exceptional performance in solving both time-dependent and time-independent partial differential equations on regular and irregular domains. Our approach is validated through comprehensive benchmarks and practical applications drawn from computational mechanics and scientific computing literature.
ASTGI: Adaptive Spatio-Temporal Graph Interactions for Irregular Multivariate Time Series Forecasting
School of Data Science and Engineering, East China Normal University, Shanghai, China（东华大学数据科学与工程学院，上海，中国）；Engineering Research Center of Blockchain Data Management, Ministry of Education, China（教育部区块链数据管理工程研究中心，中国）
Center for Artificial Intelligence Research and Innovation（人工智能研究与创新中心）
Graph-based spatio-temporal neural networks are effective to model the spatial dependency among discrete points sampled irregularly from unstructured grids, thanks to the great expressiveness of graph neural networks. However, these models are usually spatially-transductive -- only fitting the signals for discrete spatial nodes fed in models but unable to generalize to `unseen' spatial points with zero-shot. In comparison, for forecasting tasks on continuous space such as temperature prediction on the earth's surface, the \textit{spatially-inductive} property allows the model to generalize to …
Center of Artificial Intelligence for Research and Innovation, Westlake University（西湖大学人工智能研究与创新中心）；Eco-Environmental Research Laboratory, Westlake University（西湖大学生态环境研究实验室）；Zhejiang University（浙江大学）
Spatio-temporal forecasting is challenging attributing to the high nonlinearity in temporal dynamics as well as complex location-characterized patterns in spatial domains, especially in fields like weather forecasting. Graph convolutions are usually used for modeling the spatial dependency in meteorology to handle the irregular distribution of sensors' spatial location. In this work, a novel graph-based convolution for imitating the meteorological flows is proposed to capture the local spatial patterns. Based on the assumption of smoothness of location-characterized patterns, we propose condit…
University of Iowa（爱荷华大学）；Northern Illinois University（北伊利诺伊大学）；Worcester Polytechnic Institute（伍斯特理工学院）
Traffic accident forecasting is a significant problem for transportation management and public safety. However, this problem is challenging due to the spatial heterogeneity of the environment and the sparsity of accidents in space and time. The occurrence of traffic accidents is affected by complex dependencies among spatial and temporal features. Recent traffic accident prediction methods have attempted to use deep learning models to improve accuracy. However, most of these methods either focus on small-scale and homogeneous areas such as populous cities or simply use sliding-window-based ens…
Machine learning for modelling unstructured grid data in computational physics: a review
School of Artificial Intelligence and Computer Science, Nantong University（南通大学人工智能与计算机科学学院）；Department of Computer Science and Engineering, Hong Kong University of Science and Technology（香港科技大学计算机科学与工程系）；School of Microelectronics, Tianjin University（天津大学微电子学院）；Centre for Health Informatics, Cumming School of Medicine, University of Calgary（卡尔加里大学卡明医学院健康信息学中心）；Department of Community Health Sciences, Cumming School of Medicine, University of Calgary（卡尔加里大学卡明医学院社区健康科学系）；Grantham Institute for Climate Change and the Environment, Imperial College London（伦敦帝国理工学院格兰瑟姆气候变化与环境研究所）；Concordia Institute for Information Systems Engineering, Concordia University（康考迪亚大学康科迪亚信息系统工程学院）；Department of Chemical Engineering, University College London（伦敦大学学院化学工程系）
Generative Spatio-temporal GraphNet for Transonic Wing Pressure Distribution Forecasting
Faculty of Engineering and Physical Sciences, University of Southampton（南安普顿大学工程与物理科学学院）；School of Engineering, Zurich University of Applied Sciences ZHAW（苏黎世应用科技大学工程学院）；Federal Institute of Technology Zurich ETHZ（苏黎世联邦理工学院）
Meshless solutions of PDE inverse problems on irregular geometries
School of Engineering and Applied Sciences, Harvard University, Cambridge MA 02138（工程与应用科学学院，哈佛大学，康科德马萨诸塞州02138）；School of Engineering（工程学院）；Applied Sciences, Harvard University, Cambridge MA 02138（应用科学，哈佛大学，康科德马萨诸塞州02138）；Department of Physics, Harvard University, Cambridge MA 02138（物理系，哈佛大学，康科德马萨诸塞州02138）
G-PARC: Graph-Physics Aware Recurrent Convolutional Neural Networks for Spatiotemporal Dynamics on Unstructured Meshes
School of Data Science, University of Virginia（数据科学学院，弗吉尼亚大学）；University of Virginia（弗吉尼亚大学）；Department of Engineering Systems and Environment, University of Virginia（工程系统与环境系，弗吉尼亚大学）；Department of Mechanical and Aerospace Engineering, University of Virginia（机械与航空航天工程系，弗吉尼亚大学）；University of Iowa（爱荷华大学）
RBF-MGN:Solving spatiotemporal PDEs with Physics-informed Graph Neural Network
College of Aerospace Science and Engineering, National University of Defense Technology（国防科技大学航天科学与工程学院）；National Innovation Institute of Defense Technology, Chinese Academy of Military Science（中国人民解放军军事科学院国防科技创新研究院）
GRIFDIR: Graph Resolution-Invariant Diffusion Models over Irregular Domains
University of Cambridge（剑桥大学）；Technical University of Denmark（丹麦技术大学）；Simon Fraser University（西蒙弗雷泽大学）；Helmholtz Imaging（亥姆霍兹成像中心）；Deutsches Elektronen-Synchrotron DESY（德国电子同步加速器研究所）
Tsinghua University（清华大学）；Institute for Brain and Cognitive Science, Tsinghua University (THUIBCS)（清华大学脑与认知科学研究所）；Chinese PLA General Hospital（中国人民解放军总医院）
Causal discovery from time-series data has been a central task in machine learning. Recently, Granger causality inference is gaining momentum due to its good explainability and high compatibility with emerging deep neural networks. However, most existing methods assume structured input data and degenerate greatly when encountering data with randomly missing entries or non-uniform sampling frequencies, which hampers their applications in real scenarios. To address this issue, here we present CUTS, a neural Granger causal discovery algorithm to jointly impute unobserved data points and build cau…
University of Oslo（奥斯陆大学）；Institute for Energy Systems (IFE)（能源系统研究所（IFE））
With an ever-increasing number of sensors in modern society, spatio-temporal time series forecasting has become a de facto tool to make informed decisions about the future. Most spatio-temporal forecasting models typically comprise distinct components that learn spatial and temporal dependencies. A common methodology employs some Graph Neural Network (GNN) to capture relations between spatial locations, while another network, such as a recurrent neural network (RNN), learns temporal correlations. By representing every recorded sample as its own node in a graph, rather than all measurements for…
Iowa State University（爱荷华州立大学）；New York University（纽约大学）；Johnson Control Inc.（江森自控公司）
Neural network-based approaches for solving partial differential equations (PDEs) have recently received special attention. However, the large majority of neural PDE solvers only apply to rectilinear domains, and do not systematically address the imposition of Dirichlet/Neumann boundary conditions over irregular domain boundaries. In this paper, we present a framework to neurally solve partial differential equations over domains with irregularly shaped (non-rectilinear) geometric boundaries. Our network takes in the shape of the domain as an input (represented using an unstructured point cloud…
Deep learning approaches for spatio-temporal prediction problems such as crowd-flow prediction assumes data to be of fixed and regular shaped tensor and face challenges of handling irregular, sparse data tensor. This poses limitations in use-case scenarios such as predicting visit counts of individuals' for a given spatial area at a particular temporal resolution using raster/image format representation of the geographical region, since the movement patterns of an individual can be largely restricted and localized to a certain part of the raster. Additionally, current deep-learning approaches …
Carnegie Mellon University（卡内基梅隆大学）；University of Pittsburgh（匹兹堡大学）
Many scientific and engineering processes produce spatially unstructured data. However, most data-driven models require a feature matrix that enforces both a set number and order of features for each sample. They thus cannot be easily constructed for an unstructured dataset. Therefore, a graph based data-driven model to perform inference on fields defined on an unstructured mesh, using a Graph Convolutional Neural Network (GCNN) is presented. The ability of the method to predict global properties from spatially irregular measurements with high accuracy is demonstrated by predicting the drag fo…
PhyMPGN: Physics-encoded Message Passing Graph Network for spatiotemporal PDE systems
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）；School of Engineering Science, University of Chinese Academy of Sciences（中国科学院大学工程科学学院）；Huawei Technologies（华为技术有限公司）
More Sensors Only One Field: Rethinking Continual Spatio-Temporal Forecasting
City University of Hong Kong (Dongguan)（香港城市大学（东莞））；City University of Hong Kong（香港城市大学）；Zhuhai College of Science and Technology（珠海科技学院）；Northwestern Polytechnical University（西北工业大学）
The Ohio State University（俄亥俄州立大学）；Argonne National Laboratory（阿贡国家实验室）；Los Alamos National Laboratory（洛斯阿拉莫斯国家实验室）
We propose GNN-Surrogate, a graph neural network-based surrogate model to explore the parameter space of ocean climate simulations. Parameter space exploration is important for domain scientists to understand the influence of input parameters (e.g., wind stress) on the simulation output (e.g., temperature). The exploration requires scientists to exhaust the complicated parameter space by running a batch of computationally expensive simulations. Our approach improves the efficiency of parameter space exploration with a surrogate model that predicts the simulation outputs accurately and efficien…
Spatio-temporal graph learning is a key method for urban computing tasks, such as traffic flow, taxi demand and air quality forecasting. Due to the high cost of data collection, some developing cities have few available data, which makes it infeasible to train a well-performed model. To address this challenge, cross-city knowledge transfer has shown its promise, where the model learned from data-sufficient cities is leveraged to benefit the learning process of data-scarce cities. However, the spatio-temporal graphs among different cities show irregular structures and varied features, which lim…
We present a novel deep learning framework for flow field predictions in irregular domains when the solution is a function of the geometry of either the domain or objects inside the domain. Grid vertices in a computational fluid dynamics (CFD) domain are viewed as point clouds and used as inputs to a neural network based on the PointNet architecture, which learns an end-to-end mapping between spatial positions and CFD quantities. Using our approach, (i) the network inherits desirable features of unstructured meshes (e.g., fine and coarse point spacing near the object surface and in the far fie…
JD Intelligent Cities Research（京东智能城市研究院）；JD Intelligent Cities Business Unit, JD Digits（京东数字科技智能城市业务部）；Institute of Artificial Intelligence, Southwest Jiaotong University（西南交通大学人工智能研究院）；Xidian University（西安电子科技大学）；National University of Singapore（新加坡国立大学）
Being able to predict the crowd flows in each and every part of a city, especially in irregular regions, is strategically important for traffic control, risk assessment, and public safety. However, it is very challenging because of interactions and spatial correlations between different regions. In addition, it is affected by many factors: i) multiple temporal correlations among different time intervals: closeness, period, trend; ii) complex external influential factors: weather, events; iii) meta features: time of the day, day of the week, and so on. In this paper, we formulate crowd flow for…
Traditional methods for demand forecasting only focus on modeling the temporal dependency. However, forecasting on spatio-temporal data requires modeling of complex nonlinear relational and spatial dependencies. In addition, dynamic contextual information can have a significant impact on the demand values, and therefore needs to be captured. For example, in a bike-sharing system, bike usage can be impacted by weather. Existing methods assume the contextual impact is fixed. However, we note that the contextual impact evolves over time. We propose a novel context integrated relational model, Con…
Ludwig Maximilian Universität（路德维希-马克西米利安大学）；Helmholtz Zentrum München – German Research Center for Environmental Health（亥姆霍兹慕尼黑中心——德国环境健康研究中心）
During 2020, the infection rate of COVID-19 has been investigated by many scholars from different research fields. In this context, reliable and interpretable forecasts of disease incidents are a vital tool for policymakers to manage healthcare resources. Several experts have called for the necessity to account for human mobility to explain the spread of COVID-19. Existing approaches are often applying standard models of the respective research field. This habit, however, often comes along with certain restrictions. For instance, most statistical or epidemiological models cannot directly incor…
