Recurrent neural networks are widely used on time series data, yet such models often ignore the underlying physical structures in such sequences. A new class of physics-based methods related to Koopman theory has been introduced, offering an alternative for processing nonlinear dynamical systems. In this work, we propose a novel Consistent Koopman Autoencoder model which, unlike the majority of existing work, leverages the forward and backward dynamics. Key to our approach is a new analysis which explores the interplay between consistent dynamics and their associated Koopman operators. Our network is directly related to the derived analysis, and its computational requirements are comparable to other baselines. We evaluate our method on a wide range of high-dimensional and short-term dependent problems, and it achieves accurate estimates for significant prediction horizons, while also being robust to noise.
Graduate School of Engineering（工学研究生院）；University of Yamanashi（山梨大学）；Department of Civil and Environmental Engineering（土木与环境工程系）；National Research Institute for Earth Science and Disaster Resilience (NIED)（日本国立地球科学与防灾研究所（NIED））
With the accumulation of meteorological big data, data-driven models for short-term precipitation forecasting have shown increasing promise. We focus on Koopman operator analysis, which is a data-driven scheme to discover governing laws in observed data. We propose a method to apply this scheme to phenomena accompanying advection currents such as precipitation. The proposed method decomposes time evolutions of the phenomena between advection currents under a velocity field and changes in physical quantities under Lagrangian coordinates. The advection currents are estimated by kinematic analysi…
Probabilistic forecasting of complex phenomena is paramount to various scientific disciplines and applications. Despite the generality and importance of the problem, general mathematical techniques that allow for stable long-term forecasts with calibrated uncertainty measures are lacking. For most time series models, the difficulty of obtaining accurate probabilistic future time step predictions increases with the prediction horizon. In this paper, we introduce a surprisingly simple approach that characterizes time-varying distributions and enables reasonably accurate predictions thousands of …
Temporally Consistent Koopman Autoencoders for Forecasting Dynamical Systems
ElectroScience Laboratory, The Ohio State University（电科学实验室，俄亥俄州立大学）；Department of Electrical and Computer Engineering, The Ohio State University（电气与计算机工程系，俄亥俄州立大学）；SLAC National Accelerator Laboratory, Stanford University（斯坦福大学SLAC国家加速器实验室）
End-to-End Neural Decomposition with Koopman Operators for Time-Series Forecasting
National Taiwan University（台湾大学）；National Institute of Information and Communications Technology（信息通信研究机构）；Research Center for Information Technology Innovation（资讯科技创新研究中心）
Learnable Koopman-Enhanced Transformer-Based Time Series Forecasting with Spectral Control
Max Planck Institute of Geoanthropology（马克斯·普朗克地理人类学研究所）；Department of Electrical Engineering and Information Technology, University of Naples（那不勒斯大学电气工程与信息学院）
SKOLR: Structured Koopman Operator Linear RNN for Time-Series Forecasting
Department of Electrical and Computer Engineering, McGill University, Montreal, Canada（麦吉尔大学电气与计算机工程系）；Amazon Science. This work does not relate to the author’s position at Amazon（亚马逊科学）；Mila - Quebec Artificial Intelligence Institute, Montreal, Canada（魁北克人工智能研究所）；ILLS - International Laboratory on Learning Systems, Montreal, Canada（学习系统国际实验室）
DeepKoopFormer: A Koopman Enhanced Transformer Based Architecture for Time Series Forecasting
Helmholtz Center for Environmental Research-UFZ（海德堡环境研究中心-UFZ）；Delft Center for Systems and Control（代尔夫特系统与控制中心）；School of Engineering, University of Pittsburgh（匹兹堡大学工程学院）
Munich Data Science Institute（慕尼黑数据科学研究所）；School of Computation, Information and Technology, Technical University of Munich（技术大学慕尼黑计算、信息与技术学院）；Interdisciplinary Center for Scientific Computing (IWR)（跨学科科学计算中心）；Department of Mathematics and Computer Science, Heidelberg University（海德堡大学数学与计算机科学系）
Chiang Mai University（清迈大学）；Data Science Research Center（数据科学研究中心）
Short-term precipitation forecasting is essential for planning of human activities in multiple scales, ranging from individuals' planning, urban management to flood prevention. Yet the short-term atmospheric dynamics are highly nonlinear that it cannot be easily captured with classical time series models. On the other hand, deep learning models are good at learning nonlinear interactions, but they are not designed to deal with the seasonality in time series. In this study, we aim to develop a forecasting model that can both handle the nonlinearities and detect the seasonality hidden within the…
Nanjing Audit University（南京审计大学）；Southwestern University of Finance and Economics（西南财经大学）
Forecasting stock market direction is always an amazing but challenging problem in finance. Although many popular shallow computational methods (such as Backpropagation Network and Support Vector Machine) have extensively been proposed, most algorithms have not yet attained a desirable level of applicability. In this paper, we present a deep learning model with strong ability to generate high level feature representations for accurate financial prediction. Precisely, a stacked denoising autoencoder (SDAE) from deep learning is applied to predict the daily CSI 300 index, from Shanghai and Shenz…
The increasing complexity of mobility plus the growing population in cities, together with the importance of privacy when sharing data from vehicles or any device, makes traffic forecasting that uses data from infrastructure and citizens an open and challenging task. In this paper, we introduce a novel approach to deal with predictions of speed, volume, and main traffic direction, in a new aggregated way of traffic data presented as videos. The approach leverages the continuity in a sequence of frames and its dynamics, learning to predict changing areas in a low dimensional space and then, rec…
University of Science and Technology of China（中国科学技术大学）；Institute of Advanced Technology, University of Science and Technology of China（中国科学技术大学先进技术研究院）；Jiaxing University（嘉兴学院）
Large-scale self-supervised pre-training Transformer architecture have significantly boosted the performance for various tasks in natural language processing (NLP) and computer vision (CV). However, there is a lack of researches on processing multivariate time-series by pre-trained Transformer, and especially, current study on masking time-series for self-supervised learning is still a gap. Different from language and image processing, the information density of time-series increases the difficulty of research. The challenge goes further with the invalidity of the previous patch embedding and …
Distributional Drift Adaptation with Temporal Conditional Variational Autoencoder for Multivariate Time Series Forecasting
School of Medical Technology, Beijing Institute of Technology（北京理工大学医学技术学院）；School of Computer Science and Technology, Beijing Institute of Technology（北京理工大学计算机科学与技术学院）；Department of Computer Science, Tongji University（同济大学计算机科学系）；University of Technology Sydney（悉尼科技大学）；DataX Research Centre, School of Computing, Macquarie University（麦考瑞大学计算学院DataX研究中心）
VCformer: Variable Correlation Transformer with Inherent Lagged Correlation for Multivariate Time Series Forecasting
College of Computer Science and Software Engineering, Shenzhen University（深圳大学计算机与软件学院）；National Engineering Laboratory for Big Data System Computing Technology, Shenzhen University（深圳大学国家大数据系统计算技术工程实验室）
W-MAE: Pre-trained weather model with masked autoencoder for multi-variable weather forecasting
University of Electronic Science and Technology of China（电子科技大学）；Institute of Plateau Meteorology, China Meteorological Administration（中国气象局高原气象研究所）；Institute of Urban Meteorology, China Meteorological Administration（中国气象局城市气象研究所）
EPFL（洛桑联邦理工学院）；Cyber-Defence Campus, armasuisse Science and Technology（瑞士联邦国防采购局科技网络防御园区）；University of Geneva（日内瓦大学）；University of Applied Sciences of Western Switzerland（瑞士西部应用科学大学）
Because of the considerable heterogeneity and complexity of the technological landscape, building accurate models to forecast is a challenging endeavor. Due to their high prevalence in many complex systems, S-curves are a popular forecasting approach in previous work. However, their forecasting performance has not been directly compared to other technology forecasting approaches. Additionally, recent developments in time series forecasting that claim to improve forecasting accuracy are yet to be applied to technological development data. This work addresses both research gaps by comparing the …
A Time-Series Foundation Model by Universal Delay Embedding
Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences（中国科学院大学杭州高等研究院）；Shanghai Jiao Tong University（上海交通大学）；Fudan University（复旦大学）；Ocean University of China（中国海洋大学）；South China University of Technology（华南理工大学）
Generative Spatio-temporal GraphNet for Transonic Wing Pressure Distribution Forecasting
Faculty of Engineering and Physical Sciences, University of Southampton（南安普顿大学工程与物理科学学院）；School of Engineering, Zurich University of Applied Sciences ZHAW（苏黎世应用科技大学工程学院）；Federal Institute of Technology Zurich ETHZ（苏黎世联邦理工学院）
Dartmouth College（达特茅斯学院）；Courant Institute of Mathematical Sciences, New York University（纽约大学库朗数学科学研究所）；Oden Institute, University of Texas at Austin（得克萨斯大学奥斯汀分校奥登研究所）；California Institute of Technology（加州理工学院）；University of Texas at Austin（得克萨斯大学奥斯汀分校）
Kernel analog forecasting (KAF) is a powerful methodology for data-driven, non-parametric forecasting of dynamically generated time series data. This approach has a rigorous foundation in Koopman operator theory and it produces good forecasts in practice, but it suffers from the heavy computational costs common to kernel methods. This paper proposes a streaming algorithm for KAF that only requires a single pass over the training data. This algorithm dramatically reduces the costs of training and prediction without sacrificing forecasting skill. Computational experiments demonstrate that the st…
We introduce a data-driven method and shows its skills for spatiotemporal prediction of high-dimensional chaotic dynamics and turbulence. The method is based on a finite-dimensional approximation of the Koopman operator where the observables are vector-valued and delay-embedded, and the nonlinearities are treated as external forcings. The predictive capabilities of the method are demonstrated for well-known prototypes of chaos such as the Kuramoto-Sivashinsky equation and Lorenz-96 system, for which the data-driven predictions are accurate for several Lyapunov timescales. Similar performance i…
Not only can discovering patterns and insights from atmospheric data enable more accurate weather predictions, but it may also provide valuable information to help tackle climate change. Weather4cast is an open competition that aims to evaluate machine learning algorithms' capability to predict future atmospheric states. Here, we describe our third-place solution to Weather4cast. We present a novel Variational U-Net that combines a Variational Autoencoder's ability to consider the probabilistic nature of data with a U-Net's ability to recover fine-grained details. This solution is an evolution…
Qualcomm AI Research（高通AI研究院）；Qualcomm Technologies Netherlands B.V.（高通荷兰技术有限公司）；Qualcomm Technologies, Inc.（高通技术公司）；University of Amsterdam（阿姆斯特丹大学）
Autoregressive models (ARMs) currently hold state-of-the-art performance in likelihood-based modeling of image and audio data. Generally, neural network based ARMs are designed to allow fast inference, but sampling from these models is impractically slow. In this paper, we introduce the predictive sampling algorithm: a procedure that exploits the fast inference property of ARMs in order to speed up sampling, while keeping the model intact. We propose two variations of predictive sampling, namely sampling with ARM fixed-point iteration and learned forecasting modules. Their effectiveness is dem…
Weather forecast plays an essential role in multiple aspects of the daily life of human beings. Currently, physics based numerical weather prediction is used to predict the weather and requires enormous amount of computational resources. In recent years, deep learning based models have seen wide success in many weather-prediction related tasks. In this paper we describe our experiments for the Weather4cast 2021 Challenge, where 8 hours of spatio-temporal weather data is predicted based on an initial one hour of spatio-temporal data. We focus on SmaAt-UNet, an efficient U-Net based autoencoder.…
This paper propose DualVDT, a generative model for Time-series forecasting. Introduced dual reparametrized variational mechanisms on variational autoencoder (VAE) to tighter the evidence lower bound (ELBO) of the model, prove the advance performance analytically. This mechanism leverage the latent score based generative model (SGM), explicitly denoising the perturbation accumulated on latent vector through reverse time stochastic differential equation and variational ancestral sampling. The posterior of denoised latent distribution fused with dual reparametrized variational density. The KL div…
Data-driven quantum Koopman method for simulating nonlinear dynamics
State Key Laboratory for Turbulent and Complex Systems, School of Mechanics and Engineering Science, Peking University, Beijing 100871, China（湍流与复杂系统国家重点实验室，力学与工程科学学院，北京大学，北京100871，中国）；HEDPS-CAPT, Peking University, Beijing 100871, China（HEDPS-CAPT，北京大学，北京100871，中国）
