Forecasting complex dynamical phenomena in settings where only partial knowledge of their dynamics is available is a prevalent problem across various scientific fields. While purely data-driven approaches are arguably insufficient in this context, standard physical modeling based approaches tend to be over-simplistic, inducing non-negligible errors. In this work, we introduce the APHYNITY framework, a principled approach for augmenting incomplete physical dynamics described by differential equations with deep data-driven models. It consists in decomposing the dynamics into two components: a physical component accounting for the dynamics for which we have some prior knowledge, and a data-driven component accounting for errors of the physical model. The learning problem is carefully formulated such that the physical model explains as much of the data as possible, while the data-driven component only describes information that cannot be captured by the physical model, no more, no less. This not only provides the existence and uniqueness for this decomposition, but also ensures interpretability and benefits generalization. Experiments made on three important use cases, each representative of a different family of phenomena, i.e. reaction-diffusion equations, wave equations and the non-linear damped pendulum, show that APHYNITY can efficiently leverage approximate physical models to accurately forecast the evolution of the system and correctly identify relevant physical parameters. Code is available at https://github.com/yuan-yin/APHYNITY .
AWS AI Labs（AWS 人工智能实验室）；Technical University of Munich（慕尼黑工业大学）
Time series data are often corrupted by outliers or other kinds of anomalies. Identifying the anomalous points can be a goal on its own (anomaly detection), or a means to improving performance of other time series tasks (e.g. forecasting). Recent deep-learning-based approaches to anomaly detection and forecasting commonly assume that the proportion of anomalies in the training data is small enough to ignore, and treat the unlabeled data as coming from the nominal data distribution. We present a simple yet effective technique for augmenting existing time series models so that they explicitly ac…
Predicting patient volumes in a hospital setting is a well-studied application of time series forecasting. Existing tools usually make forecasts at the daily or weekly level to assist in planning for staffing requirements. Prompted by new COVID-related capacity constraints placed on our pediatric hospital's emergency department, we developed an hourly forecasting tool to make predictions over a 24 hour window. These forecasts would give our hospital sufficient time to be able to martial resources towards expanding capacity and augmenting staff (e.g. transforming wards or bringing in physicians…
HESAM Université（巴黎高科艺术与职业大学）；Conservatoire national des arts et métiers（法国国立工艺学院）；Simon Fraser University（西蒙弗雷泽大学）；Borealis AI（Borealis人工智能研究院）；Valeo AI（法雷奥人工智能研究院）；Sorbonne Université（索邦大学）；Criteo AI（科韬人工智能研究院）；Mines ParisTech（巴黎高等矿业学院）；EDF R&D（法国电力研发中心）；Université Rennes 1（雷恩第一大学）；INRIA（法国国家信息与自动化研究所）
This thesis tackles the subject of spatio-temporal forecasting with deep learning. The motivating application at Electricity de France (EDF) is short-term solar energy forecasting with fisheye images. We explore two main research directions for improving deep forecasting methods by injecting external physical knowledge. The first direction concerns the role of the training loss function. We show that differentiable shape and temporal criteria can be leveraged to improve the performances of existing models. We address both the deterministic context with the proposed DILATE loss function and the…
VQLTI: Long-Term Tropical Cyclone Intensity Forecasting with Physical Constraints
University of Science and Technology of China（中国科学技术大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Hong Kong University of Science and Technology（香港科技大学）
CortAIx Thales（泰雷兹CortAIx）；Inria（法国国家信息与自动化研究所）；University of Oxford（牛津大学）；The University of Aberdeen（阿伯丁大学）；Huawei（华为）；Zhejiang University（浙江大学）
Transfer learning aims at building robust prediction models by transferring knowledge gained from one problem to another. In the semantic Web, learning tasks are enhanced with semantic representations. We exploit their semantics to augment transfer learning by dealing with when to transfer with semantic measurements and what to transfer with semantic embeddings. We further present a general framework that integrates the above measurements and embeddings with existing transfer learning algorithms for higher performance. It has demonstrated to be robust in two real-world applications: bus delay …
Geomagnetic storms, disturbances of Earth's magnetosphere caused by masses of charged particles being emitted from the Sun, are an uncontrollable threat to modern technology. Notably, they have the potential to damage satellites and cause instability in power grids on Earth, among other disasters. They result from high sun activity, which are induced from cool areas on the Sun known as sunspots. Forecasting the storms to prevent disasters requires an understanding of how and when they will occur. However, current prediction methods at the National Oceanic and Atmospheric Administration (NOAA) …
PINN-Cast: Exploring the Role of Continuous-Depth NODE in Transformers and Physics Informed Loss as Soft Physical Constraints in Short-term Weather Forecasting
Multimodal Physical Activity Forecasting in Free-Living Clinical Settings: Hunting Opportunities for Just-in-Time Interventions
College of Health Solutions, Arizona State University（亚利桑那州立大学健康解决方案学院）；School of Computing and Augmented Intelligence, Arizona State University（亚利桑那州立大学计算与增强智能学院）；Edson College of Nursing and Health Innovation, Arizona State University（亚利桑那州立大学埃德森护理与健康创新学院）
Norwegian University of Science and Technology（挪威科技大学）
This paper presents an ensemble forecasting method that shows strong results on the M4 Competition dataset by decreasing feature and model selection assumptions, termed DONUT (DO Not UTilize human beliefs). Our assumption reductions, primarily consisting of auto-generated features and a more diverse model pool for the ensemble, significantly outperform the statistical, feature-based ensemble method FFORMA by Montero-Manso et al. (2020). We also investigate feature extraction with a Long Short-term Memory Network (LSTM) Autoencoder and find that such features contain crucial information not cap…
PCDCNet: A Surrogate Model for Air Quality Forecasting with Physical-Chemical Dynamics and Constraints
School of Systems Science Beijing Normal University（系统科学学院 北京师范大学）；D-ITET, ETH Zurich（ETH Zurich 信息科技系）；Swiss Data Science Center（瑞士数据科学中心）；ColorfulClouds Technology Co.,Ltd.（ColorfulClouds 技术有限公司）；Graz University of Technology（格拉茨技术大学）；Complexity Science Hub Vienna（维也纳复杂科学中心）；School of Systems Science / Institute of Nonequilibrium Systems Beijing Normal University（系统科学学院/非平衡系统研究所 北京师范大学）；Potsdam Institute for Climate Impact Research（波茨坦气候影响研究所）
GeoCert: Certified Geometric AI for Reliable Forecasting
Department of Biomedical Informatics & Data Science, Yale University（耶鲁大学生物医学信息学与数据科学系）；Department of Computer Science, The University of Hong Kong（香港大学计算机科学系）；School of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院）；Department of Computer Science and Technology, University of Cambridge（剑桥大学计算机科学与技术系）
DSPR: Dual-Stream Physics-Residual Networks for Trustworthy Industrial Time Series Forecasting
School of Computer Science and Engineering, University of Electronic Science and Technology of China（电子科技大学计算机科学与工程学院）；Research Center, East Hope Group Co., Ltd（东希望集团有限公司研究院）
Process-Informed Forecasting of Complex Thermal Dynamics in Pharmaceutical Manufacturing
Department of Agricultural and Environmental Sciences, University of Bari（巴里大学农业与环境科学系）；Department of Mechanical, Mathematics, and Management (DMMM), Polytechnic University of Bari（巴里理工学院机械、数学与管理系）；Division of Applied Mathematics, Brown University（布朗大学应用数学系）
Spatio-temporal dynamics of physical processes are generally modeled using partial differential equations (PDEs). Though the core dynamics follows some principles of physics, real-world physical processes are often driven by unknown external sources. In such cases, developing a purely analytical model becomes very difficult and data-driven modeling can be of assistance. In this paper, we present a hybrid framework combining physics-based numerical models with deep learning for source identification and forecasting of spatio-temporal dynamical systems with unobservable time-varying external sou…
CTP: A hybrid CNN-Transformer-PINN model for ocean front forecasting
School of Oceanography, Shanghai Jiao Tong University, China（上海交通大学海洋学院）；State Key Laboratory of Satellite Ocean Environment Dynamics, Second Institute of Oceanography, MNR, China（国家卫星海洋环境动力学重点实验室，第二海洋研究所）；College of Marine Science and Technology, China University of Geosciences, China（中国地质大学（北京）海洋科学与技术学院）
FuXi-RTM: A Physics-Guided Prediction Framework with Radiative Transfer Modeling
Artificial Intelligence Innovation and Incubation Institute, Fudan University（复旦大学人工智能创新与孵化研究院）；Shanghai Innovation Institute（上海创新研究院）；Shanghai Academy of Artificial Intelligence for Science（上海科学人工智能研究院）
Differential-Integral Neural Operator for Long-Term Turbulence Forecasting
Tsinghua University（清华大学）；University of Science and Technology of China（中国科学技术大学）；The Chinese University of Hong Kong（香港中文大学）；Nanyang Technological University（南洋理工大学）；Tencent（腾讯）
Physically consistent and uncertainty-aware learning of spatiotemporal dynamics
Technical University of Munich（慕尼黑技术大学）；Munich Center for Machine Learning（慕尼黑机器学习中心）；University of Bristol（布里斯托大学）；Universitat de València（瓦伦西亚大学）；Munich Climate Center（慕尼黑气候中心）
Aalborg University（奥尔堡大学）；University of Southern California（南加州大学）
Correlated time series (CTS) forecasting plays an essential role in many cyber-physical systems, where multiple sensors emit time series that capture interconnected processes. Solutions based on deep learning that deliver state-of-the-art CTS forecasting performance employ a variety of spatio-temporal (ST) blocks that are able to model temporal dependencies and spatial correlations among time series. However, two challenges remain. First, ST-blocks are designed manually, which is time consuming and costly. Second, existing forecasting models simply stack the same ST-blocks multiple times, whic…
Cyber-physical systems often consist of entities that interact with each other over time. Meanwhile, as part of the continued digitization of industrial processes, various sensor technologies are deployed that enable us to record time-varying attributes (a.k.a., time series) of such entities, thus producing correlated time series. To enable accurate forecasting on such correlated time series, this paper proposes two models that combine convolutional neural networks (CNNs) and recurrent neural networks (RNNs). The first model employs a CNN on each individual time series, combines the convoluted…
The increasing penetration level of energy generation from renewable sources is demanding for more accurate and reliable forecasting tools to support classic power grid operations (e.g., unit commitment, electricity market clearing or maintenance planning). For this purpose, many physical models have been employed, and more recently many statistical or machine learning algorithms, and data-driven methods in general, are becoming subject of intense research. While generally the power research community focuses on power forecasting at the level of single plants, in a short future horizon of time…
University of Science and Technology of China（中国科学技术大学）
In traffic forecasting, graph convolutional networks (GCNs), which model traffic flows as spatio-temporal graphs, have achieved remarkable performance. However, existing GCN-based methods heuristically define the graph structure as the physical topology of the road network, ignoring potential dependence of the graph structure over traffic data. And the defined graph structure is deterministic, which lacks investigation of uncertainty. In this paper, we propose a Bayesian Spatio-Temporal Graph Convolutional Network (BSTGCN) for traffic prediction. The graph structure in our network is learned f…
Learning more physically realistic dynamics in machine-learning based weather forecasting with latent-space constraints
Department of Earth and Environmental Engineering, Columbia University（哥伦比亚大学地球与环境工程系）；Learning the Earth with Artificial Intelligence and Physics (LEAP) Center, Columbia University（人工智能与物理联合地球学习中心（LEAP）, 哥伦比亚大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；The Chinese University of Hong Kong（香港中文大学）；Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系）
Graduate School of Engineering（工学研究生院）；University of Yamanashi（山梨大学）；Department of Civil and Environmental Engineering（土木与环境工程系）；National Research Institute for Earth Science and Disaster Resilience (NIED)（日本国立地球科学与防灾研究所（NIED））
With the accumulation of meteorological big data, data-driven models for short-term precipitation forecasting have shown increasing promise. We focus on Koopman operator analysis, which is a data-driven scheme to discover governing laws in observed data. We propose a method to apply this scheme to phenomena accompanying advection currents such as precipitation. The proposed method decomposes time evolutions of the phenomena between advection currents under a velocity field and changes in physical quantities under Lagrangian coordinates. The advection currents are estimated by kinematic analysi…
