Neural networks for structured data like graphs have been studied extensively in recent years. To date, the bulk of research activity has focused mainly on static graphs. However, most real-world networks are dynamic since their topology tends to change over time. Predicting the evolution of dynamic graphs is a task of high significance in the area of graph mining. Despite its practical importance, the task has not been explored in depth so far, mainly due to its challenging nature. In this paper, we propose a model that predicts the evolution of dynamic graphs. Specifically, we use a graph neural network along with a recurrent architecture to capture the temporal evolution patterns of dynamic graphs. Then, we employ a generative model which predicts the topology of the graph at the next time step and constructs a graph instance that corresponds to that topology. We evaluate the proposed model on several artificial datasets following common network evolving dynamics, as well as on real-world datasets. Results demonstrate the effectiveness of the proposed model.
Zhejiang University（浙江大学）；Alibaba Group（阿里巴巴集团）；Emory University（埃默里大学）；University of Southern California（南加州大学）
The accurate and interpretable prediction of future events in time-series data often requires the capturing of representative patterns (or referred to as states) underpinning the observed data. To this end, most existing studies focus on the representation and recognition of states, but ignore the changing transitional relations among them. In this paper, we present evolutionary state graph, a dynamic graph structure designed to systematically represent the evolving relations (edges) among states (nodes) along time. We conduct analysis on the dynamic graphs constructed from the time-series dat…
Bike sharing systems often suffer from poor capacity management as a result of variable demand. These bike sharing systems would benefit from models to predict demand in order to moderate the number of bikes stored at each station. In this paper, we attempt to apply a graph neural network model to predict bike demand in the New York City, Citi Bike dataset.
Rutgers University（罗格斯大学）；HPI, Univ. of Potsdam（波茨坦大学哈索普拉特纳研究所 (HPI)）
Software development is becoming increasingly open and collaborative with the advent of platforms such as GitHub. Given its crucial role, there is a need to better understand and model the dynamics of GitHub as a social platform. Previous work has mostly considered the dynamics of traditional social networking sites like Twitter and Facebook. We propose GitEvolve, a system to predict the evolution of GitHub repositories and the different ways by which users interact with them. To this end, we develop an end-to-end multi-task sequential deep neural network that given some seed events, simultane…
Evolution-based Feature Selection for Predicting Dissolved Oxygen Concentrations in Lakes
University of Pittsburgh（匹兹堡大学）；Aarhus University（奥胡斯大学）；University of Science and Technology of China（中国科学技术大学）；Georgia Institute of Technology（佐治亚理工学院）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；University of Maryland（马里兰大学）
Communities in social networks evolve over time as people enter and leave the network and their activity behaviors shift. The task of predicting structural changes in communities over time is known as community evolution prediction. Existing work in this area has focused on the development of frameworks for defining events while using traditional classification methods to perform the actual prediction. We present a novel graph neural network for predicting community evolution events from structural and temporal information. The model (GNAN) includes a group-node attention component which enabl…
State Key Laboratory of Multimodal Artificial Intelligence Systems Institute of Automation, Chinese Academy of Sciences（多模态人工智能系统国家重点实验室 中国科学院自动化研究所）；School of Artificial Intelligence University of Chinese Academy of Sciences（中国科学院自动化研究所人工智能学院）
Physics-based simulation for fluid flow in porous media is a computational technology to predict the temporal-spatial evolution of state variables (e.g. pressure) in porous media, and usually requires high computational expense due to its nonlinearity and the scale of the study domain. This letter describes a deep learning (DL) workflow to predict the pressure evolution as fluid flows in large-scale 3D heterogeneous porous media. In particular, we apply feature coarsening technique to extract the most representative information and perform the training and prediction of DL at the coarse scale,…
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences（多模态人工智能系统国家重点实验室，自动化研究所，中国科学院）；School of Artificial Intelligence, University of Chinese Academy of Sciences（人工智能学院，中国科学院大学）
Predicting Infant Brain Connectivity with Federated Multi-Trajectory GNNs using Scarce Data
Imperial College London（伦敦帝国理工学院）；University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；ShanghaiTech University（上海科技大学）；Shanghai United Imaging Intelligence Co., Ltd.（上海联影智能医疗科技有限公司）；Shanghai Clinical Research and Trial Center（上海临床研究中心）
Istanbul Technical University（伊斯坦布尔技术大学）；University of Manouba（马努巴大学）
Several brain disorders can be detected by observing alterations in the brain's structural and functional connectivities. Neurological findings suggest that early diagnosis of brain disorders, such as mild cognitive impairment (MCI), can prevent and even reverse its development into Alzheimer's disease (AD). In this context, recent studies aimed to predict the evolution of brain connectivities over time by proposing machine learning models that work on brain images. However, such an approach is costly and time-consuming. Here, we propose to use brain connectivities as a more efficient alternat…
Lawrence Livermore National Laboratory（劳伦斯利弗莫尔国家实验室）；Texas A & M University（德克萨斯农工大学）；University of Notre Dame（圣母大学）
In this work, sequence-to-sequence (seq2seq) models, originally developed for language translation, are used to predict the temporal evolution of complex, multi-physics computer simulations. The predictive performance of seq2seq models is compared to state transition models for datasets generated with multi-physics codes with varying levels of complexity - from simple 1D diffusion calculations to simulations of inertial confinement fusion implosions. Seq2seq models demonstrate the ability to accurately emulate complex systems, enabling the rapid estimation of the evolution of quantities of int…
An emerging way of tackling the dimensionality issues arising in the modeling of a multivariate process is to assume that the inherent data structure can be captured by a graph. Nevertheless, though state-of-the-art graph-based methods have been successful for many learning tasks, they do not consider time-evolving signals and thus are not suitable for prediction. Based on the recently introduced joint stationarity framework for time-vertex processes, this letter considers multivariate models that exploit the graph topology so as to facilitate the prediction. The resulting method yields simila…
Istanbul Technical University（伊斯坦布尔技术大学）；University of Dundee（邓迪大学）
Predicting the evolution of the brain network, also called connectome, by foreseeing changes in the connectivity weights linking pairs of anatomical regions makes it possible to spot connectivity-related neurological disorders in earlier stages and detect the development of potential connectomic anomalies. Remarkably, such a challenging prediction problem remains least explored in the predictive connectomics literature. It is a known fact that machine learning (ML) methods have proven their predictive abilities in a wide variety of computer vision problems. However, ML techniques specifically …
Princess Margaret Cancer Centre, University Health Network（大学健康网络玛格丽特公主癌症中心）；CIFAR AI Chair（CIFAR AI 主席）
Cancer treatment is an arduous process for patients and causes many side-effects during and post-treatment. The treatment can affect almost all body systems and result in pain, fatigue, sleep disturbances, cognitive impairments, etc. These conditions are often under-diagnosed or under-treated. In this paper, we use patient data to predict the evolution of their symptoms such that treatment-related impairments can be prevented or effects meaningfully ameliorated. The focus of this study is on predicting the pain and tiredness level of a patient post their diagnosis. We implement an interpretabl…
Siemens AG（西门子股份公司）；Ludwig Maximilian University of Munich（慕尼黑大学）
Embedding learning, a.k.a. representation learning, has been shown to be able to model large-scale semantic knowledge graphs. A key concept is a mapping of the knowledge graph to a tensor representation whose entries are predicted by models using latent representations of generalized entities. Knowledge graphs are typically treated as static: A knowledge graph grows more links when more facts become available but the ground truth values associated with links is considered time invariant. In this paper we address the issue of knowledge graphs where triple states depend on time. We assume that c…
Managing patients with chronic diseases is a major and growing healthcare challenge in several countries. A chronic condition, such as diabetes, is an illness that lasts a long time and does not go away, and often leads to the patient's health gradually getting worse. While recent works involve raw electronic health record (EHR) from hospitals, this work uses only financial records from health plan providers (medical claims) to predict diabetes disease evolution with a self-attentive recurrent neural network. The use of financial data is due to the possibility of being an interface to internat…
Claremont Graduate University（克莱蒙特研究生大学）；Pomona College（波莫纳学院）；San Diego State University（圣地亚哥州立大学）；Los Alamos National Laboratory（洛斯阿拉莫斯国家实验室）；Broadridge Financial Solutions（百瑞吉金融解决方案公司）；Institute of Mathematical Sciences（数学科学研究所）；Mathematics Department（数学系）；Computational Science Research Center（计算科学研究中心）；Earth and Environmental Sciences Division（地球与环境科学部）
We propose a machine learning approach to address a key challenge in materials science: predicting how fractures propagate in brittle materials under stress, and how these materials ultimately fail. Our methods use deep learning and train on simulation data from high-fidelity models, emulating the results of these models while avoiding the overwhelming computational demands associated with running a statistically significant sample of simulations. We employ a graph convolutional network that recognizes features of the fracturing material and a recurrent neural network that models the evolution…
Keldysh Institute of Applied Mathematics of the Russian Academy of Sciences（俄罗斯科学院克尔德什应用数学研究所）；Skolkovo Institute of Science and Technology（斯科尔科沃科学技术学院）；Marchuk Institute of Numerical Mathematics of the Russian Academy of Sciences（俄罗斯科学院马尔丘克数值数学研究所）
Forecasting time series and time-dependent data is a common problem in many applications. One typical example is solving ordinary differential equation (ODE) systems $\dot{x}=F(x)$. Oftentimes the right hand side function $F(x)$ is not known explicitly and the ODE system is described by solution samples taken at some time points. Hence, ODE solvers cannot be used. In this paper, a data-driven approach to learning the evolution of dynamical systems is considered. We show how by training neural networks with ResNet-like architecture on the solution samples, models can be developed to predict the…
The significance of heating load (HL) accurate approximation is the primary motivation of this research to distinguish the most efficient predictive model among several neural-metaheuristic models. The proposed models are through synthesizing multi-layer perceptron network (MLP) with ant lion optimization (ALO), biogeography-based optimization (BBO), dragonfly algorithm (DA), evolutionary strategy (ES), invasive weed optimization (IWO), and league champion optimization (LCA) hybrid algorithms. Each ensemble is optimized in terms of the operating population. Accordingly, the ALO-MLP, BBO-MLP, D…
Cambridge Cancer Genomics（剑桥癌症基因组学公司）；The University of Edinburgh（爱丁堡大学）
The majority of cancer treatments end in failure due to Intra-Tumor Heterogeneity (ITH). ITH in cancer is represented by clonal evolution where different sub-clones compete with each other for resources under conditions of Darwinian natural selection. Predicting the growth of these sub-clones within a tumour is among the key challenges of modern cancer research. Predicting tumor behavior enables the creation of risk profiles for patients and the optimisation of their treatment by therapeutically targeting sub-clones more likely to grow. Current research efforts in this space are focused on mat…
Istanbul Technical University（伊斯坦布尔理工大学）；University of Sousse（苏塞大学）
While existing predictive frameworks are able to handle Euclidean structured data (i.e, brain images), they might fail to generalize to geometric non-Euclidean data such as brain networks. Besides, these are rooted the sample selection step in using Euclidean or learned similarity measure between vectorized training and testing brain networks. Such sample connectomic representation might include irrelevant and redundant features that could mislead the training sample selection step. Undoubtedly, this fails to exploit and preserve the topology of the brain connectome. To overcome this major dra…
Learning how to estimate a connectional brain template(CBT) from a population of brain multigraphs, where each graph (e.g., functional) quantifies a particular relationship between pairs of brain regions of interest (ROIs), allows to pin down the unique connectivity patterns shared across individuals. Specifically, a CBT is viewed as an integral representation of a set of highly heterogeneous graphs and ideally meeting the centeredness (i.e., minimum distance to all graphs in the population) and discriminativeness (i.e., distinguishes the healthy from the disordered population) criteria. So fa…
Provable learning separation for predicting time-evolution of quantum many-body systems
Leiden University（莱顿大学）；Leiden Institute of Advanced Computer Science（莱顿高级计算机科学研究所）；Dahlem Center for Complex Quantum Systems（达赫姆复杂量子系统中心）；Fraunhofer Heinrich Hertz Institute（弗劳恩霍夫海因里希·赫兹研究所）
ETH Zurich（苏黎世联邦理工学院）；Swiss Data Science Center（瑞士数据科学中心）；EPF Lausanne（洛桑联邦理工学院）；EAWAG（瑞士联邦水科学与技术研究所）；University of Zurich（苏黎世大学）
In this technical report we compare different deep learning models for prediction of water depth rasters at high spatial resolution. Efficient, accurate, and fast methods for water depth prediction are nowadays important as urban floods are increasing due to higher rainfall intensity caused by climate change, expansion of cities and changes in land use. While hydrodynamic models models can provide reliable forecasts by simulating water depth at every location of a catchment, they also have a high computational burden which jeopardizes their application to real-time prediction in large urban ar…
Stanford University（斯坦福大学）；Georgia Institute of Technology（佐治亚理工学院）；University of Illinois, Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Modeling sequential interactions between users and items/products is crucial in domains such as e-commerce, social networking, and education. Representation learning presents an attractive opportunity to model the dynamic evolution of users and items, where each user/item can be embedded in a Euclidean space and its evolution can be modeled by an embedding trajectory in this space. However, existing dynamic embedding methods generate embeddings only when users take actions and do not explicitly model the future trajectory of the user/item in the embedding space. Here we propose JODIE, a couple…
Dartmouth College（达特茅斯学院）；ByteDance AI Lab（字节跳动人工智能实验室）
Predicting the behaviors of Hamiltonian systems has been drawing increasing attention in scientific machine learning. However, the vast majority of the literature was focused on predicting separable Hamiltonian systems with their kinematic and potential energy terms being explicitly decoupled while building data-driven paradigms to predict nonseparable Hamiltonian systems that are ubiquitous in fluid dynamics and quantum mechanics were rarely explored. The main computational challenge lies in the effective embedding of symplectic priors to describe the inherently coupled evolution of position …
Okinawa Institute of Science and Technology（冲绳科学技术大学院大学）；Research Center for the Early Universe, School of Science, The University of Tokyo（东京大学理学部早期宇宙研究中心）
Understanding the long-term evolution of hierarchical triple systems is challenging due to its inherent chaotic nature, and it requires computationally expensive simulations. Here we propose a convolutional neural network model to predict the stability of hierarchical triples by looking at their evolution during the first $5 \times 10^5$ inner binary orbits. We employ the regularized few-body code TSUNAMI to simulate $5\times 10^6$ hierarchical triples, from which we generate a large training and test dataset. We develop twelve different network configurations that use different combinations o…
University of Houston（休斯顿大学）；Los Alamos National Laboratory（洛斯阿拉莫斯国家实验室）
In this paper we demonstrate that reservoir computing can be used to learn the dynamics of the shallow-water equations. In particular, while most previous applications of reservoir computing have required training on a particular trajectory to further predict the evolution along that trajectory alone, we show the capability of reservoir computing to predict trajectories of the shallow-water equations with initial conditions not seen in the training process. However, in this setting, we find that the performance of the network deteriorates for initial conditions with ambient conditions (such as…
In this paper, we consider the problem of learning prediction models for spatiotemporal physical processes driven by unknown partial differential equations (PDEs). We propose a deep learning framework that learns the underlying dynamics and predicts its evolution using sparsely distributed data sites. Deep learning has shown promising results in modeling physical dynamics in recent years. However, most of the existing deep learning methods for modeling physical dynamics either focus on solving known PDEs or require data in a dense grid when the governing PDEs are unknown. In contrast, our meth…
Comparative Analysis of Predicting Subsequent Steps in Hénon Map
School of Digital Sciences, Digital University Kerala（喀拉拉邦数字大学数字科学学院）；SUPA, School of Physics and Astronomy, The University of Edinburgh（爱丁堡大学SUPA物理与天文学院）
Despite their massive success, training successful deep neural networks still largely relies on experimentally choosing an architecture, hyper-parameters, initialization, and training mechanism. In this work, we focus on determining the success of standard gradient descent method for training deep neural networks on a specified dataset, architecture, and initialization (DAI) combination. Through extensive systematic experiments, we show that the evolution of singular values of the matrix obtained from the hidden layers of a DNN can aid in determining the success of gradient descent technique t…
In this paper, we address the problem of predicting complex, nonlinear spatiotemporal dynamics when available data is recorded at irregularly-spaced sparse spatial locations. Most of the existing deep learning models for modeling spatiotemporal dynamics are either designed for data in a regular grid or struggle to uncover the spatial relations from sparse and irregularly-spaced data sites. We propose a deep learning model that learns to predict unknown spatiotemporal dynamics using data from sparsely-distributed data sites. We base our approach on Radial Basis Function (RBF) collocation method…
Space Telescope Science Institute（太空望远镜科学研究所）；Johns Hopkins University（约翰斯·霍普金斯大学）
Galaxies can be described by features of their optical spectra such as oxygen emission lines, or morphological features such as spiral arms. Although spectroscopy provides a rich description of the physical processes that govern galaxy evolution, spectroscopic data are observationally expensive to obtain. For the first time, we are able to robustly predict galaxy spectra directly from broad-band imaging. We present a powerful new approach using a hybrid convolutional neural network with deconvolution instead of batch normalization; this hybrid CNN outperforms other models in our tests. The lea…
