High-dimensional spatio-temporal dynamics can often be encoded in a low-dimensional subspace. Engineering applications for modeling, characterization, design, and control of such large-scale systems often rely on dimensionality reduction to make solutions computationally tractable in real-time. Common existing paradigms for dimensionality reduction include linear methods, such as the singular value decomposition (SVD), and nonlinear methods, such as variants of convolutional autoencoders (CAE). However, these encoding techniques lack the ability to efficiently represent the complexity associated with spatio-temporal data, which often requires variable geometry, non-uniform grid resolution, adaptive meshing, and/or parametric dependencies. To resolve these practical engineering challenges, we propose a general framework called Neural Implicit Flow (NIF) that enables a mesh-agnostic, low-rank representation of large-scale, parametric, spatial-temporal data. NIF consists of two modified multilayer perceptrons (MLPs): (i) ShapeNet, which isolates and represents the spatial complexity, and (ii) ParameterNet, which accounts for any other input complexity, including parametric dependencies, time, and sensor measurements. We demonstrate the utility of NIF for parametric surrogate modeling, enabling the interpretable representation and compression of complex spatio-temporal dynamics, efficient many-spatial-query tasks, and improved generalization performance for sparse reconstruction.
Deep learning has become a one-size-fits-all solution for technical and business domains thanks to its flexibility and adaptability. It is implemented using opaque models, which unfortunately undermines the outcome trustworthiness. In order to have a better understanding of the behavior of a system, particularly one driven by time series, a look inside a deep learning model so-called posthoc eXplainable Artificial Intelligence (XAI) approaches, is important. There are two major types of XAI for time series data, namely model-agnostic and model-specific. Model-specific approach is considered in…
Modeling neural population dynamics underlying noisy single-trial spiking activities is essential for relating neural observation and behavior. A recent non-recurrent method - Neural Data Transformers (NDT) - has shown great success in capturing neural dynamics with low inference latency without an explicit dynamical model. However, NDT focuses on modeling the temporal evolution of the population activity while neglecting the rich covariation between individual neurons. In this paper we introduce SpatioTemporal Neural Data Transformer (STNDT), an NDT-based architecture that explicitly models r…
We propose MASTAF, a Model-Agnostic Spatio-Temporal Attention Fusion network for few-shot video classification. MASTAF takes input from a general video spatial and temporal representation,e.g., using 2D CNN, 3D CNN, and Video Transformer. Then, to make the most of such representations, we use self- and cross-attention models to highlight the critical spatio-temporal region to increase the inter-class variations and decrease the intra-class variations. Last, MASTAF applies a lightweight fusion network and a nearest neighbor classifier to classify each query video. We demonstrate that MASTAF imp…
Mila - Québec AI Institute（米拉-魁北克人工智能研究所）；Université de Montréal（蒙特利尔大学）；Université du Québec à Rimouski（魁北克大学里穆斯基分校）
The computational complexity of classical numerical methods for solving Partial Differential Equations (PDE) scales significantly as the resolution increases. As an important example, climate predictions require fine spatio-temporal resolutions to resolve all turbulent scales in the fluid simulations. This makes the task of accurately resolving these scales computationally out of reach even with modern supercomputers. As a result, current numerical modelers solve PDEs on grids that are too coarse (3km to 200km on each side), which hinders the accuracy and usefulness of the predictions. In this…
Traffic time series forecasting is challenging due to complex spatio-temporal dynamics time series from different locations often have distinct patterns; and for the same time series, patterns may vary across time, where, for example, there exist certain periods across a day showing stronger temporal correlations. Although recent forecasting models, in particular deep learning based models, show promising results, they suffer from being spatio-temporal agnostic. Such spatio-temporal agnostic models employ a shared parameter space irrespective of the time series locations and the time periods a…
G-PARC: Graph-Physics Aware Recurrent Convolutional Neural Networks for Spatiotemporal Dynamics on Unstructured Meshes
School of Data Science, University of Virginia（数据科学学院，弗吉尼亚大学）；University of Virginia（弗吉尼亚大学）；Department of Engineering Systems and Environment, University of Virginia（工程系统与环境系，弗吉尼亚大学）；Department of Mechanical and Aerospace Engineering, University of Virginia（机械与航空航天工程系，弗吉尼亚大学）；University of Iowa（爱荷华大学）
University of California, Berkeley（加州大学伯克利分校）；Stanford University（斯坦福大学）；California Institute of Technology（加州理工学院）；Lawrence Berkeley National Laboratory（劳伦斯伯克利国家实验室）；NVIDIA（英伟达）
We propose MeshfreeFlowNet, a novel deep learning-based super-resolution framework to generate continuous (grid-free) spatio-temporal solutions from the low-resolution inputs. While being computationally efficient, MeshfreeFlowNet accurately recovers the fine-scale quantities of interest. MeshfreeFlowNet allows for: (i) the output to be sampled at all spatio-temporal resolutions, (ii) a set of Partial Differential Equation (PDE) constraints to be imposed, and (iii) training on fixed-size inputs on arbitrarily sized spatio-temporal domains owing to its fully convolutional encoder. We empiricall…
University of Tübingen（图宾根大学）；University of Hohenheim（霍恩海姆大学）
We introduce a distributed spatio-temporal artificial neural network architecture (DISTANA). It encodes mesh nodes using recurrent, neural prediction kernels (PKs), while neural transition kernels (TKs) transfer information between neighboring PKs, together modeling and predicting spatio-temporal time series dynamics. As a consequence, DISTANA assumes that generally applicable causes, which may be locally modified, generate the observed data. DISTANA learns in a parallel, spatially distributed manner, scales to large problem spaces, is capable of approximating complex dynamics, and is particul…
PhyMPGN: Physics-encoded Message Passing Graph Network for spatiotemporal PDE systems
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）；School of Engineering Science, University of Chinese Academy of Sciences（中国科学院大学工程科学学院）；Huawei Technologies（华为技术有限公司）
Learning 3D geometry directly from raw data, such as point clouds, triangle soups, or unoriented meshes is still a challenging task that feeds many downstream computer vision and graphics applications. In this paper, we introduce SALD: a method for learning implicit neural representations of shapes directly from raw data. We generalize sign agnostic learning (SAL) to include derivatives: given an unsigned distance function to the input raw data, we advocate a novel sign agnostic regression loss, incorporating both pointwise values and gradients of the unsigned distance function. Optimizing thi…
Carnegie Mellon University（卡内基梅隆大学）；University of Toronto（多伦多大学）；LIX, École Polytechnique（巴黎综合理工学院计算机科学实验室）
We introduce a new general-purpose approach to deep learning on 3D surfaces, based on the insight that a simple diffusion layer is highly effective for spatial communication. The resulting networks are automatically robust to changes in resolution and sampling of a surface -- a basic property which is crucial for practical applications. Our networks can be discretized on various geometric representations such as triangle meshes or point clouds, and can even be trained on one representation then applied to another. We optimize the spatial support of diffusion as a continuous network parameter r…
In this paper, we address the problem of predicting complex, nonlinear spatiotemporal dynamics when available data is recorded at irregularly-spaced sparse spatial locations. Most of the existing deep learning models for modeling spatiotemporal dynamics are either designed for data in a regular grid or struggle to uncover the spatial relations from sparse and irregularly-spaced data sites. We propose a deep learning model that learns to predict unknown spatiotemporal dynamics using data from sparsely-distributed data sites. We base our approach on Radial Basis Function (RBF) collocation method…
Universidad Nacional de Educación a Distancia — UNED（西班牙国家远程教育大学）
When confronting a spatio-temporal regression, it is sensible to feed the model with any available prior information about the spatial dimension. For example, it is common to define the architecture of neural networks based on spatial closeness, adjacency, or correlation. A common alternative, if spatial information is not available or is too costly to introduce it in the model, is to learn it as an extra step of the model. While the use of prior spatial knowledge, given or learnt, might be beneficial, in this work we question this principle by comparing spatial agnostic neural networks with s…
Spatio-temporal graph learning is a key method for urban computing tasks, such as traffic flow, taxi demand and air quality forecasting. Due to the high cost of data collection, some developing cities have few available data, which makes it infeasible to train a well-performed model. To address this challenge, cross-city knowledge transfer has shown its promise, where the model learned from data-sufficient cities is leveraged to benefit the learning process of data-scarce cities. However, the spatio-temporal graphs among different cities show irregular structures and varied features, which lim…
Munich Data Science Institute（慕尼黑数据科学研究所）；Technical University of Munich（慕尼黑工业大学）
We propose a new method for spatio-temporal forecasting on arbitrarily distributed points. Assuming that the observed system follows an unknown partial differential equation, we derive a continuous-time model for the dynamics of the data via the finite element method. The resulting graph neural network estimates the instantaneous effects of the unknown dynamics on each cell in a meshing of the spatial domain. Our model can incorporate prior knowledge via assumptions on the form of the unknown PDE, which induce a structural bias towards learning specific processes. Through this mechanism, we de…
School of Computing and Data Science（计算与数据科学学院）；The University of Hong Kong（香港大学）；School of Electrical Engineering and Computer Science（电气工程与计算机科学学院）；The University of Queensland（昆士兰大学）；Department of Computing（计算学院）
P$^2$C$^2$Net: PDE-Preserved Coarse Correction Network for efficient prediction of spatiotemporal dynamics
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）；Northeastern University（东北大学）；University of Notre Dame（圣母大学）；Huawei Technologies（华为技术有限公司）；School of Engineering Science, University of Chinese Academy of Sciences（中国科学院大学工程科学学院）
COPPE/Federal University of Rio de Janeiro（里约热内卢联邦大学COPPE学院）；Gran Sasso Science Institute（格兰萨索科学研究所）；Università di Pavia（帕维亚大学）
Dynamic Mode Decomposition (DMD) is a powerful data-driven method used to extract spatio-temporal coherent structures that dictate a given dynamical system. The method consists of stacking collected temporal snapshots into a matrix and mapping the nonlinear dynamics using a linear operator. The standard procedure considers that snapshots possess the same dimensionality for all the observable data. However, this often does not occur in numerical simulations with adaptive mesh refinement/coarsening schemes (AMR/C). This paper proposes a strategy to enable DMD to extract features from observation…
Multiscale Training of Convolutional Neural Networks
Department of Physics & Astronomy, University of British Columbia（物理与天文学系，不列颠哥伦比亚大学）；Department of Earth, Ocean and Atmospheric Sciences, University of British Columbia（地球、海洋和大气科学系，不列颠哥伦比亚大学）；Department of Applied Mathematics and Theoretical Physics, University of Cambridge（应用数学与理论物理系，剑桥大学）
College of Systems Engineering, National University of Defense Technology（国防科技大学系统工程学院）；Bristol Centre for Functional Nanomaterials, University of Bristol（布里斯托大学布里斯托功能纳米材料中心）
Urban ride-hailing demand prediction is a crucial but challenging task for intelligent transportation system construction. Predictable ride-hailing demand can facilitate more reasonable vehicle scheduling and online car-hailing platform dispatch. Conventional deep learning methods with no external structured data can be accomplished via hybrid models of CNNs and RNNs by meshing plentiful pixel-level labeled data, but spatial data sparsity and limited learning capabilities on temporal long-term dependencies are still two striking bottlenecks. To address these limitations, we propose a new virtu…
Mesh-free sparse identification of nonlinear dynamics
Paul G. Allen School of Computer Science & Engineering University of Washington（保罗·G·艾伦计算机科学与工程学院华盛顿大学）；Applied Mathematics and Electrical and Computer Engineering University of Washington（应用数学与电气与计算机工程华盛顿大学）；Faculty of Mechanical Engineering Delft University of Technology（机械工程学院代尔夫特理工大学）
Pruning AMR: Efficient Visualization of Implicit Neural Representations via Weight Matrix Analysis
Center for Applied Mathematics, Cornell University（应用数学中心，康奈尔大学）；Center for Applied Scientific Computing, Lawrence Livermore National Laboratory（应用科学计算中心，劳伦斯·利弗莫尔国家实验室）
Residual-based Attention Physics-informed Neural Networks for Spatio-Temporal Ageing Assessment of Transformers Operated in Renewable Power Plants
Mondragon University（蒙dragon大学）；University of the Basque Country (UPV/EHU)（巴斯克大学）；Basque Centre for Applied Mathematics（巴斯克应用数学中心）；KTH, Royal Institute of Technology（皇家理工学院）；Ikerbasque, Basque Foundation for Science（巴斯克科学基金会）
