Empirical Mode Modeling: A data-driven approach to recover and forecast nonlinear dynamics from noisy data
Salk Institute for Biological Studies（索尔克生物研究所）；U.S. Department of the Interior（美国内政部）；Scripps Institution of Oceanography（斯克里普斯海洋研究所）；University of California San Diego（加州大学圣地亚哥分校）
Joseph Park, Gerald M Pao, Erik Stabenau, George Sugihara, Thomas Lorimer
Data-driven, model-free analytics are natural choices for discovery and forecasting of complex, nonlinear systems. Methods that operate in the system state-space require either an explicit multidimensional state-space, or, one approximated from available observations. Since observational data are frequently sampled with noise, it is possible that noise can corrupt the state-space representation degrading analytical performance. Here, we evaluate the synthesis of empirical mode decomposition with empirical dynamic modeling, which we term empirical mode modeling, to increase the information content of state-space representations in the presence of noise. Evaluation of a mathematical, and, an ecologically important geophysical application across three different state-space representations suggests that empirical mode modeling may be a useful technique for data-driven, model-free, state-space analysis in the presence of noise.
University of Sydney（悉尼大学）；University of Potsdam（波茨坦大学）
We present a supervised learning method to learn the propagator map of a dynamical system from partial and noisy observations. In our computationally cheap and easy-to-implement framework a neural network consisting of random feature maps is trained sequentially by incoming observations within a data assimilation procedure. By employing Takens' embedding theorem, the network is trained on delay coordinates. We show that the combination of random feature maps and data assimilation, called RAFDA, outperforms standard random feature maps for which the dynamics is learned using batch data.
BBVA Data and Analytics（BBVA数据与分析公司）；Universitat de Barcelona（巴塞罗那大学）
Deep Learning is a consolidated, state-of-the-art Machine Learning tool to fit a function when provided with large data sets of examples. However, in regression tasks, the straightforward application of Deep Learning models provides a point estimate of the target. In addition, the model does not take into account the uncertainty of a prediction. This represents a great limitation for tasks where communicating an erroneous prediction carries a risk. In this paper we tackle a real-world problem of forecasting impending financial expenses and incomings of customers, while displaying predictable m…
Stanford University（斯坦福大学）；University of California, Berkeley（加州大学伯克利分校）；Toyota Research Institute（丰田研究院）
We tackle the problem of Human Locomotion Forecasting, a task for jointly predicting the spatial positions of several keypoints on the human body in the near future under an egocentric setting. In contrast to the previous work that aims to solve either the task of pose prediction or trajectory forecasting in isolation, we propose a framework to unify the two problems and address the practically useful task of pedestrian locomotion prediction in the wild. Among the major challenges in solving this task is the scarcity of annotated egocentric video datasets with dense annotations for pose, depth…
NeuTSFlow: Modeling Continuous Functions Behind Time Series Forecasting
University of Science and Technology of China（中国科学技术大学）；Tianjin University（天津大学）；Xi’an Jiaotong University（西安交通大学）；University of Cambridge（剑桥大学）；YMSC, Tsinghua University（清华大学数学科学中心）
Is Precise Recovery Necessary? A Task-Oriented Imputation Approach for Time Series Forecasting on Variable Subset
University of Macau（澳门大学）；Northeast Normal University（东北师范大学）；Computer Network Information Center, Chinese Academy of Sciences（中国科学院计算机网络信息中心）；University of Oxford（牛津大学）；Dalian Maritime University（大连海事大学）
Exploring the Limitations of kNN Noisy Feature Detection and Recovery for Self-Driving Labs
University of Toronto（多伦多大学）；Artificial, Inc.（Artificial公司）；National Research Council Canada（加拿大国家研究委员会）；Natural Resources Canada（加拿大自然资源部）；Vector Institute for Artificial Intelligence（Vector人工智能研究所）；Schwartz Reisman Institute for Technology and Society（Schwartz Reisman技术与社会研究所）
Probabilistic Forecasting for Dynamical Systems with Missing or Imperfect Data
Institute of Applied Mathematics（应用数学研究所）；University of British Columbia（不列颠哥伦比亚大学）；Recherche en prévision numérique atmosphérique（大气数值预报研究中心）；Environnement et Changement climatique Canada（加拿大环境与气候变化部）
Optimal training of finitely-sampled quantum reservoir computers for forecasting of chaotic dynamics
Imperial College London（伦敦帝国理工学院）；City St George’s, University of London（伦敦大学城市圣乔治学院）；The Alan Turing Institute（艾伦·图灵研究所）；Politecnico di Torino（都灵理工大学）
Université de Lorraine（洛林大学）；AL.I.E.N. (ALgèbre pour Identification & Estimation Numériques)（AL.I.E.N.（数字识别与估计代数公司））；INRIA Lille – Nord-Europe（法国里尔-东北欧洲数字研究院）；Hôpital de Castelluccio（卡斯泰卢乔医院）；Università di Corsica Pasquale Paoli（科西嘉大学帕斯夸莱·保利）；École polytechnique（巴黎综合理工学院）；Université Galatasaray（加拉塔萨雷大学）
This communication is devoted to solar irradiance and irradiation short-term forecasts, which are useful for electricity production. Several different time series approaches are employed. Our results and the corresponding numerical simulations show that techniques which do not need a large amount of historical data behave better than those which need them, especially when those data are quite noisy.
Univariate to Multivariate: LLMs as Zero-Shot Predictors for Time-Series Forecasting
School of Computing Technologies RMIT University, Australia（计算机技术学院皇家墨尔本理工大学，澳大利亚）；School of Information Technologies Deakin University, Australia（信息科技学院德肯大学，澳大利亚）
Out-of-Sight Embodied Agents: Multimodal Tracking, Sensor Fusion, and Trajectory Forecasting
Department of Electrical and Computer Engineering, Northeastern University（东北大学电气与计算机工程系）；Khoury College of Computer Sciences, Northeastern University（东北大学库里计算机科学学院）
Nanyang Technological University（南洋理工大学）；School of Materials Science and Engineering（材料科学与工程学院）
We introduce a self-consistent deep-learning framework which, for a noisy deterministic time series, provides unsupervised filtering, state-space reconstruction, identification of the underlying differential equations and forecasting. Without a priori information on the signal, we embed the time series in a state space, where deterministic structures, i.e. attractors, are revealed. Under the assumption that the evolution of solution trajectories is described by an unknown dynamical system, we filter out stochastic outliers. The embedding function, the solution trajectories and the dynamical sy…
School of Mathematics and Statistics University of Sydney（悉尼大学数学与统计学院）；Institute of Mathematics University of Potsdam（波茨坦大学数学研究所）
Data-driven prediction and physics-agnostic machine-learning methods have attracted increased interest in recent years achieving forecast horizons going well beyond those to be expected for chaotic dynamical systems. In a separate strand of research data-assimilation has been successfully used to optimally combine forecast models and their inherent uncertainty with incoming noisy observations. The key idea in our work here is to achieve increased forecast capabilities by judiciously combining machine-learning algorithms and data assimilation. We combine the physics-agnostic data-driven approac…
Independent Component Analysis (ICA) is a popular model for blind signal separation. The ICA model assumes that a number of independent source signals are linearly mixed to form the observed signals. We propose a new algorithm, PEGI (for pseudo-Euclidean Gradient Iteration), for provable model recovery for ICA with Gaussian noise. The main technical innovation of the algorithm is to use a fixed point iteration in a pseudo-Euclidean (indefinite "inner product") space. The use of this indefinite "inner product" resolves technical issues common to several existing algorithms for noisy ICA. This l…
Rensselaer Polytechnic Institute（伦斯勒理工学院）；IBM Thomas J. Watson Research Center（IBM托马斯·J·沃森研究中心）
Higher-order tensors can represent scores in a rating system, frames in a video, and images of the same subject. In practice, the measurements are often highly quantized due to the sampling strategies or the quality of devices. Existing works on tensor recovery have focused on data losses and random noises. Only a few works consider tensor recovery from quantized measurements but are restricted to binary measurements. This paper, for the first time, addresses the problem of tensor recovery from multi-level quantized measurements. Leveraging the low-rank property of the tensor, this paper propo…
We study a practical algorithm for sparse principal component analysis (PCA) of incomplete and noisy data. Our algorithm is based on the semidefinite program (SDP) relaxation of the non-convex $l_1$-regularized PCA problem. We provide theoretical and experimental evidence that SDP enables us to exactly recover the true support of the sparse leading eigenvector of the unknown true matrix, despite only observing an incomplete (missing uniformly at random) and noisy version of it. We derive sufficient conditions for exact recovery, which involve matrix incoherence, the spectral gap between the la…
We present a deep learning model, DE-LSTM, for the simulation of a stochastic process with an underlying nonlinear dynamics. The deep learning model aims to approximate the probability density function of a stochastic process via numerical discretization and the underlying nonlinear dynamics is modeled by the Long Short-Term Memory (LSTM) network. It is shown that, when the numerical discretization is used, the function estimation problem can be solved by a multi-label classification problem. A penalized maximum log likelihood method is proposed to impose a smoothness condition in the predicti…
Many models for sparse regression typically assume that the covariates are known completely, and without noise. Particularly in high-dimensional applications, this is often not the case. This paper develops efficient OMP-like algorithms to deal with precisely this setting. Our algorithms are as efficient as OMP, and improve on the best-known results for missing and noisy data in regression, both in the high-dimensional setting where we seek to recover a sparse vector from only a few measurements, and in the classical low-dimensional setting where we recover an unstructured regressor. In the hi…
Carnegie Mellon University（卡内基梅隆大学）；University of Alberta（阿尔伯塔大学）
Boolean matrix factorization and Boolean matrix completion from noisy observations are desirable unsupervised data-analysis methods due to their interpretability, but hard to perform due to their NP-hardness. We treat these problems as maximum a posteriori inference problems in a graphical model and present a message passing approach that scales linearly with the number of observations and factors. Our empirical study demonstrates that message passing is able to recover low-rank Boolean matrices, in the boundaries of theoretically possible recovery and compares favorably with state-of-the-art …
We consider the Orthogonal Least-Squares (OLS) algorithm for the recovery of a $m$-dimensional $k$-sparse signal from a low number of noisy linear measurements. The Exact Recovery Condition (ERC) in bounded noisy scenario is established for OLS under certain condition on nonzero elements of the signal. The new result also improves the existing guarantees for Orthogonal Matching Pursuit (OMP) algorithm. In addition, This framework is employed to provide probabilistic guarantees for the case that the coefficient matrix is drawn at random according to Gaussian or Bernoulli distribution where we e…
