In this paper, we tackle the problem of relational behavior forecasting from sensor data. Towards this goal, we propose a novel spatially-aware graph neural network (SpAGNN) that models the interactions between agents in the scene. Specifically, we exploit a convolutional neural network to detect the actors and compute their initial states. A graph neural network then iteratively updates the actor states via a message passing process. Inspired by Gaussian belief propagation, we design the messages to be spatially-transformed parameters of the output distributions from neighboring agents. Our model is fully differentiable, thus enabling end-to-end training. Importantly, our probabilistic predictions can model uncertainty at the trajectory level. We demonstrate the effectiveness of our approach by achieving significant improvements over the state-of-the-art on two real-world self-driving datasets: ATG4D and nuScenes.
Traditional methods for demand forecasting only focus on modeling the temporal dependency. However, forecasting on spatio-temporal data requires modeling of complex nonlinear relational and spatial dependencies. In addition, dynamic contextual information can have a significant impact on the demand values, and therefore needs to be captured. For example, in a bike-sharing system, bike usage can be impacted by weather. Existing methods assume the contextual impact is fixed. However, we note that the contextual impact evolves over time. We propose a novel context integrated relational model, Con…
Leveraging Spatiotemporal Graph Neural Networks for Multi-Store Sales Forecasting
Department of Chemical Engineering, IIT Bombay（印度博亚理工大学化学工程系）；Rajiv Gandhi University（拉吉夫·甘地大学）；Tsinghua University（清华大学）；Palmer Research Laboratories（帕勒尔研究实验室）
Spatiotemporal forecasting plays an essential role in various applications in intelligent transportation systems (ITS), such as route planning, navigation, and traffic control and management. Deep Spatiotemporal graph neural networks (GNNs), which capture both spatial and temporal patterns, have achieved great success in traffic forecasting applications. Understanding how GNNs-based forecasting work and the vulnerability and robustness of these models becomes critical to real-world applications. For example, if spatiotemporal GNNs are vulnerable in real-world traffic prediction applications, a…
Ecole Polytechnique（巴黎综合理工学院）；Exoduspoint Capital Management France（法国Exoduspoint资本管理公司）；ENSAE Paris（巴黎国立统计与经济管理学院）
The existing publications demonstrate that the limit order book data is useful in predicting short-term volatility in stock markets. Since stocks are not independent, changes on one stock can also impact other related stocks. In this paper, we are interested in forecasting short-term realized volatility in a multivariate approach based on limit order book data and relational data. To achieve this goal, we introduce Graph Transformer Network for Volatility Forecasting. The model allows to combine limit order book features and an unlimited number of temporal and cross-sectional relations from di…
Institute of Informatics, LMU Munich（慕尼黑大学信息学研究所）；Corporate Technology, Siemens AG（西门子股份公司中央研究院）；Department of Informatics, Technical University of Munich（慕尼黑工业大学信息学系）
Modeling time-evolving knowledge graphs (KGs) has recently gained increasing interest. Here, graph representation learning has become the dominant paradigm for link prediction on temporal KGs. However, the embedding-based approaches largely operate in a black-box fashion, lacking the ability to interpret their predictions. This paper provides a link forecasting framework that reasons over query-relevant subgraphs of temporal KGs and jointly models the structural dependencies and the temporal dynamics. Especially, we propose a temporal relational attention mechanism and a novel reverse represen…
Chubu University（中部大学）；OMRON SINIC X（欧姆龙 SINIC X 研究院）
Forecasting human activities observed in videos is a long-standing challenge in computer vision, which leads to various real-world applications such as mobile robots, autonomous driving, and assistive systems. In this work, we present a new visual forecasting task called crowd density forecasting. Given a video of a crowd captured by a surveillance camera, our goal is to predict how that crowd will move in future frames. To address this task, we have developed the patch-based density forecasting network (PDFN), which enables forecasting over a sequence of crowd density maps describing how crow…
Google Research（谷歌研究院）；University of Maryland（马里兰大学）
This paper focuses on multi-person action forecasting in videos. More precisely, given a history of H previous frames, the goal is to detect actors and to predict their future actions for the next T frames. Our approach jointly models temporal and spatial interactions among different actors by constructing a recurrent graph, using actor proposals obtained with Faster R-CNN as nodes. Our method learns to select a subset of discriminative relations without requiring explicit supervision, thus enabling us to tackle challenging visual data. We refer to our model as Discriminative Relational Recurr…
University of Technology Sydney（悉尼科技大学）；Monash University（莫纳什大学）
Modeling multivariate time series has long been a subject that has attracted researchers from a diverse range of fields including economics, finance, and traffic. A basic assumption behind multivariate time series forecasting is that its variables depend on one another but, upon looking closely, it is fair to say that existing methods fail to fully exploit latent spatial dependencies between pairs of variables. In recent years, meanwhile, graph neural networks (GNNs) have shown high capability in handling relational dependencies. GNNs require well-defined graph structures for information propa…
Forecasting West Nile Virus with Graph Neural Networks: Harnessing Spatial Dependence in Irregularly Sampled Geospatial Data
Department of Statistics, University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校统计系）；Department of Statistics, Texas A&M University（德克萨斯A&M大学统计系）；Department of Pathobiology, University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校病理生物学系）
IBM Research（IBM研究院）；Sandia National Lab（桑迪亚国家实验室）；Texas A&M（德克萨斯A&M大学）；IBM Corporate Social Responsibility（IBM企业社会责任部）
Climate change, population growth, and water scarcity present unprecedented challenges for agriculture. This project aims to forecast soil moisture using domain knowledge and machine learning for crop management decisions that enable sustainable farming. Traditional methods for predicting hydrological response features require significant computational time and expertise. Recent work has implemented machine learning models as a tool for forecasting hydrological response features, but these models neglect a crucial component of traditional hydrological modeling that spatially close units can ha…
CPiRi: Channel Permutation-Invariant Relational Interaction for Multivariate Time Series Forecasting
School of Computing and Artificial Intelligence（计算与人工智能学院）；Shanghai University of Finance and Economics（上海财经大学）；School of Information Technology and Artificial Intelligence（信息科技与人工智能学院）；Zhejiang University of Finance and Economics（浙江财经大学）
Ensemble Graph Neural Networks for Probabilistic Sea Surface Temperature Forecasting via Input Perturbations
Centro de Tecnologías de la Imagen Instituto Universitario de Cibernética, Empresas y Sociedad (IUCES) University of Las Palmas de Gran Canaria, Spain（图像技术中心大学计算机科学、企业和社会研究院（IUCES）大学拉斯帕尔马斯德Gran Canaria分校）
Context-aware knowledge graph framework for traffic speed forecasting using graph neural network
Future Resilient Systems, Singapore-ETH Centre, ETH Zurich（未来弹性系统中心（新加坡-ETH中心，苏黎世联邦理工学院））；Institute of Cartography and Geoinformation, ETH Zurich（苏黎世联邦理工学院制图与地理信息研究所）；University of Canterbury（坎特伯雷大学）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）
Unifying Physics- and Data-Driven Modeling via Novel Causal Spatiotemporal Graph Neural Network for Interpretable Epidemic Forecasting
Frankfurt Institute for Advanced Studies（法兰克福高等研究院）；Goethe Universität Frankfurt（法兰克福大学）；Xidian-FIAS International Joint Research Center（西电-法兰克福高等研究院国际联合研究中心）；School of Science and Engineering, The Chinese University of Hong Kong（香港中文大学（深圳）理工学院）；GSI Helmholtzzentrum für Schwerionenforschung GmbH（GSI亥姆霍兹重离子研究中心）
Honda Research Institute, USA（本田美国研究所）；University of California, Berkeley（加州大学伯克利分校）
Multi-agent interacting systems are prevalent in the world, from pure physical systems to complicated social dynamic systems. In many applications, effective understanding of the situation and accurate trajectory prediction of interactive agents play a significant role in downstream tasks, such as decision making and planning. In this paper, we propose a generic trajectory forecasting framework (named EvolveGraph) with explicit relational structure recognition and prediction via latent interaction graphs among multiple heterogeneous, interactive agents. Considering the uncertainty of future be…
HyperVis: Continuous Latent Visual Relational Graphs on the Lorentz Hyperboloid for Compositional Reasoning
Data Science and AI, University of Doha for Science and Technology, Qatar（数据科学与人工智能，多哈科学技术大学，卡塔尔）；Pluralis Research, Australia（Pluralis研究，澳大利亚）；Department of Electrical and Computer Engineering, North South University, Bangladesh（电气与计算机工程系，北南大学，孟加拉国）
Convolutional Neural Networks (CNN) possess many positive qualities when it comes to spatial raster data. Translation invariance enables CNNs to detect features regardless of their position in the scene. However, in some domains, like geospatial, not all locations are exactly equal. In this work, we propose localized convolutional neural networks that enable convolutional architectures to learn local features in addition to the global ones. We investigate their instantiations in the form of learnable inputs, local weights, and a more general form. They can be added to any convolutional layers,…
