Probabilistic forecasting of irregularly sampled multivariate time series with missing values is an important problem in many fields, including health care, astronomy, and climate. State-of-the-art methods for the task estimate only marginal distributions of observations in single channels and at single timepoints, assuming a fixed-shape parametric distribution. In this work, we propose a novel model, ProFITi, for probabilistic forecasting of irregularly sampled time series with missing values using conditional normalizing flows. The model learns joint distributions over the future values of the time series conditioned on past observations and queried channels and times, without assuming any fixed shape of the underlying distribution. As model components, we introduce a novel invertible triangular attention layer and an invertible non-linear activation function on and onto the whole real line. We conduct extensive experiments on four datasets and demonstrate that the proposed model provides $4$ times higher likelihood over the previously best model.
University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）；IBM Almaden Research Center（IBM阿尔马登研究中心）
Sparse and irregularly sampled multivariate time series are common in clinical, climate, financial and many other domains. Most recent approaches focus on classification, regression or forecasting tasks on such data. In forecasting, it is necessary to not only forecast the right value but also to forecast when that value will occur in the irregular time series. In this work, we present an approach to forecast not only the values but also the time at which they are expected to occur.
In this paper we propose a data augmentation method for time series with irregular sampling, Time-Conditional Generative Adversarial Network (T-CGAN). Our approach is based on Conditional Generative Adversarial Networks (CGAN), where the generative step is implemented by a deconvolutional NN and the discriminative step by a convolutional NN. Both the generator and the discriminator are conditioned on the sampling timestamps, to learn the hidden relationship between data and timestamps, and consequently to generate new time series. We evaluate our model with synthetic and real-world datasets. F…
Conditionally Whitened Generative Models for Probabilistic Time Series Forecasting
Graduate University of Advanced Studies & The Institute of Statistical Mathematics（高级研究大学及统计数学研究所）；East China Normal University（华东师范大学）；The Institute of Statistical Mathematics（统计数学研究所）
KTH Royal Institute of Technology（瑞典皇家理工学院）；RISE Research Institutes of Sweden（瑞典RISE研究院）
We present a representation learning framework for financial time series forecasting. One challenge of using deep learning models for finance forecasting is the shortage of available training data when using small datasets. Direct trend classification using deep neural networks trained on small datasets is susceptible to the overfitting problem. In this paper, we propose to first learn compact representations from time series data, then use the learned representations to train a simpler model for predicting time series movements. We consider a class-conditioned latent variable model. We train …
Heterogeneity and irregularity of multi-source data sets present a significant challenge to time-series analysis. In the literature, the fusion of multi-source time-series has been achieved either by using ensemble learning models which ignore temporal patterns and correlation within features or by defining a fixed-size window to select specific parts of the data sets. On the other hand, many studies have shown major improvement to handle the irregularity of time-series, yet none of these studies has been applied to multi-source data. In this work, we design a novel architecture, PIETS, to mod…
Spatiotemporal System Forecasting with Irregular Time Steps via Masked Autoencoder
University College London（伦敦大学学院）；Imperial College London（帝国理工学院）；University of Liverpool（利物浦大学）；ENPC, EDF R&D, Institut Polytechnique de Paris（巴黎高科国立路桥学校，法国电力集团研发部，巴黎综合理工学院）
Neural networks inspired by differential equations have proliferated for the past several years. Neural ordinary differential equations (NODEs) and neural controlled differential equations (NCDEs) are two representative examples of them. In theory, NCDEs provide better representation learning capability for time-series data than NODEs. In particular, it is known that NCDEs are suitable for processing irregular time-series data. Whereas NODEs have been successfully extended after adopting attention, however, it had not been studied yet how to integrate attention into NCDEs. To this end, we pres…
Transformer with Koopman-Enhanced Graph Convolutional Network for Spatiotemporal Dynamics Forecasting
Charles F. Dolan School of Business, Fairfield University（Fairfield大学商学院）；Department of Industrial & Systems Engineering, The University of Tennessee（田纳西大学工业与系统工程系）
We present a method for conditional time series forecasting based on an adaptation of the recent deep convolutional WaveNet architecture. The proposed network contains stacks of dilated convolutions that allow it to access a broad range of history when forecasting, a ReLU activation function and conditioning is performed by applying multiple convolutional filters in parallel to separate time series which allows for the fast processing of data and the exploitation of the correlation structure between the multivariate time series. We test and analyze the performance of the convolutional network …
This paper proposes a temporal graph neural network model for forecasting of graph-structured irregularly observed time series. Our TGNN4I model is designed to handle both irregular time steps and partial observations of the graph. This is achieved by introducing a time-continuous latent state in each node, following a linear Ordinary Differential Equation (ODE) defined by the output of a Gated Recurrent Unit (GRU). The ODE has an explicit solution as a combination of exponential decay and periodic dynamics. Observations in the graph neighborhood are taken into account by integrating graph neu…
We introduce a new category of multivariate conditional generative models and demonstrate its performance and versatility in probabilistic time series forecasting and simulation. Specifically, the output of quantile regression networks is expanded from a set of fixed quantiles to the whole Quantile Function by a univariate mapping from a latent uniform distribution to the target distribution. Then the multivariate case is solved by learning such quantile functions for each dimension's marginal distribution, followed by estimating a conditional Copula to associate these latent uniform random va…
Monash University（蒙纳士大学）；National Technical University of Athens（雅典国家技术大学）
Model selection has been proven an effective strategy for improving accuracy in time series forecasting applications. However, when dealing with hierarchical time series, apart from selecting the most appropriate forecasting model, forecasters have also to select a suitable method for reconciling the base forecasts produced for each series to make sure they are coherent. Although some hierarchical forecasting methods like minimum trace are strongly supported both theoretically and empirically for reconciling the base forecasts, there are still circumstances under which they might not produce t…
We propose a new framework of CatBoost that predicts the entire conditional distribution of a univariate response variable. In particular, CatBoostLSS models all moments of a parametric distribution (i.e., mean, location, scale and shape [LSS]) instead of the conditional mean only. Choosing from a wide range of continuous, discrete and mixed discrete-continuous distributions, modelling and predicting the entire conditional distribution greatly enhances the flexibility of CatBoost, as it allows to gain insight into the data generating process, as well as to create probabilistic forecasts from w…
OceanMoE: Structured Conditional Sparse Computation for Long-Horizon Multivariate Ocean Forecasting
Hangzhou Institute for Advanced Study, University of the Chinese Academy of Sciences（中国科学院大学杭州高等研究院）；Computer Network Information Center, Chinese Academy of Sciences（中国科学院计算机网络信息中心）
Forecasting is challenging since uncertainty resulted from exogenous factors exists. This work investigates the rank position forecasting problem in car racing, which predicts the rank positions at the future laps for cars. Among the many factors that bring changes to the rank positions, pit stops are critical but irregular and rare. We found existing methods, including statistical models, machine learning regression models, and state-of-the-art deep forecasting model based on encoder-decoder architecture, all have limitations in the forecasting. By elaborative analysis of pit stops events, we…
In several disciplines it is common to find time series measured at irregular observational times. In particular, in astronomy there are a large number of surveys that gather information over irregular time gaps and in more than one passband. Some examples are Pan-STARRS, ZTF and also the LSST. However, current commonly used time series models that estimate the time dependency in astronomical light curves consider the information of each band separately (e.g, CIAR, IAR and CARMA models) disregarding the dependency that might exist between different passbands. In this paper we propose a novel b…
Conditional Neural ODE for Longitudinal Parkinson's Disease Progression Forecasting
Emory University（埃默里大学）；University of California, Los Angeles（加州大学洛杉矶分校）；Illinois Institute of Technology（伊利诺伊理工学院）；Lehigh University（莱斯大学）；University of Pittsburgh（匹兹堡大学）；University of Wisconsin–Madison（威斯康星大学麦迪逊分校）
We propose a new framework of XGBoost that predicts the entire conditional distribution of a univariate response variable. In particular, XGBoostLSS models all moments of a parametric distribution (i.e., mean, location, scale and shape [LSS]) instead of the conditional mean only. Choosing from a wide range of continuous, discrete and mixed discrete-continuous distribution, modelling and predicting the entire conditional distribution greatly enhances the flexibility of XGBoost, as it allows to gain additional insight into the data generating process, as well as to create probabilistic forecasts…
This study was conducted to find an appropriate statistical model to forecast the volatilities of PSEi using the model Generalized Autoregressive Conditional Heteroskedasticity (GARCH). Using the R software, the log returns of PSEi is modeled using various ARIMA models and with the presence of heteroskedasticity, the log returns was modeled using GARCH. Based on the analysis, GARCH models are the most appropriate to use for the log returns of PSEi. Among the selected GARCH models, GARCH (1,2) has the lowest AIC value and also has the highest LL value implying that GARCH (1,2) is the best model…
The decomposition of a time series is an essential task that helps to understand its very nature. It facilitates the analysis and forecasting of complex time series expressing various hidden components such as the trend, seasonal components, cyclic components and irregular fluctuations. Therefore, it is crucial in many fields for forecasting and decision processes. In recent years, many methods of time series decomposition have been developed, which extract and reveal different time series properties. Unfortunately, they neglect a very important property, i.e. time series variance. To deal wit…
The increasing penetration of embedded renewables makes forecasting net-load, consumption less embedded generation, a significant and growing challenge. Here a framework for producing probabilistic forecasts of net-load is proposed with particular attention given to the tails of predictive distributions, which are required for managing risk associated with low-probability events. Only small volumes of data are available in the tails, by definition, so estimation of predictive models and forecast evaluation requires special attention. We propose a solution based on a best-in-class load forecast…
Kernel analog forecasting (KAF), alternatively known as kernel principal component regression, is a kernel method used for nonparametric statistical forecasting of dynamically generated time series data. This paper synthesizes descriptions of kernel methods and Koopman operator theory in order to provide a single consistent account of KAF. The framework presented here illuminates the property of the KAF method that, under measure-preserving and ergodic dynamics, it consistently approximates the conditional expectation of observables that are acted upon by the Koopman operator of the dynamical …
The conditional distribution of the next outcome given the infinite past of a stationary process can be inferred from finite but growing segments of the past. Several schemes are known for constructing pointwise consistent estimates, but they all demand prohibitive amounts of input data. In this paper we consider real-valued time series and construct conditional distribution estimates that make much more efficient use of the input data. The estimates are consistent in a weak sense, and the question whether they are pointwise consistent is still open. For finite-alphabet processes one may rely …
Many wind speed forecasting approaches have been proposed in literature. In this paper a new statistical approach for jointly predicting wind speed, wind direction and air pressure is introduced. The wind direction and the air pressure are important to extend the forecasting accuracy of wind speed forecasts. A good forecast for the wind direction helps to bring the turbine into the predominant wind direction. We combine a multivariate seasonal time varying threshold autoregressive model with interactions (TVARX) with a threshold seasonal autoregressive conditional heteroscedastic (TARCHX) mode…
Combining forecast from different models has shown to perform better than single forecast in most time series. To improve the quality of forecast we can go for combining forecast. We study the effect of decomposing a series into multiple components and performing forecasts on each component separately... The original series is decomposed into trend, seasonality and an irregular component for each series. The statistical methods such as ARIMA, Holt-Winter have been used to forecast these components. In this paper we focus on how the best models of one series can be applied to similar frequency …
By significant improvements in modern electrical systems, planning for unit commitment and power dispatching of them are two big concerns between the researchers. Short-term load forecasting plays a significant role in planning and dispatching them. In recent years, numerous works have been done on Short-term load forecasting. Having an accurate model for predicting the load can be beneficial for optimizing the electrical sources and protecting energy. Several models such as Artificial Intelligence and Statistics model have been used to improve the accuracy of load forecasting. Among the stati…
ASTGI: Adaptive Spatio-Temporal Graph Interactions for Irregular Multivariate Time Series Forecasting
School of Data Science and Engineering, East China Normal University, Shanghai, China（东华大学数据科学与工程学院，上海，中国）；Engineering Research Center of Blockchain Data Management, Ministry of Education, China（教育部区块链数据管理工程研究中心，中国）
