This paper studies some temporal dependence properties and addresses the issue of parametric estimation for a class of state-dependent autoregressive models for nonlinear time series in which we assume a stochastic autoregressive coefficient depending on the first lagged value of the process itself. We call such a model state-dependent first-order autoregressive process, (SDAR). We introduce some assumptions under which this class of models is strictly stationary and uniformly ergodic and we establish consistency and asymptotic normality of the quasi-maximum likelihood estimator of the parameters. In order to capture the potentiality of the model, we present an empirical application to nonlinear time series provided by the weekly realized volatility extracted from returns of some European financial indices. The comparison of forecasting accuracy is made considering an alternative approach provided by a two-regime SETAR model
This paper introduces a new parsimonious structure for mixture of autoregressive models. the weighting coefficients are determined through latent random variables, following a hidden Markov model. We propose a dynamic programming algorithm for the application of forecasting. We also derive the limiting behavior of unconditional first moment of the process and an appropriate upper bound for the limiting value of the variance. This can be considered as long run behavior of the process. Finally we show convergence and stability of the second moment. Further, we illustrate the efficacy of the prop…
The predictive capability of a modification of Rissanen's accumulated prediction error (APE) criterion, APE$_{δ_n}$, is investigated in infinite-order autoregressive (AR($\infty$)) models. Instead of accumulating squares of sequential prediction errors from the beginning, APE$_{δ_n}$ is obtained by summing these squared errors from stage $nδ_n$, where $n$ is the sample size and $1/n\leq δ_n\leq 1-(1/n)$ may depend on $n$. Under certain regularity conditions, an asymptotic expression is derived for the mean-squared prediction error (MSPE) of an AR predictor with order determined by APE$_{δ_n}$.…
In this paper we derive the consistency of the penalized likelihood method for the number state of the hidden Markov chain in autoregressive models with Markov regimen. Using a SAEM type algorithm to estimate the models parameters. We test the null hypothesis of hidden Markov Model against an autoregressive process with Markov regime.
This report introduces a parsimonious structure for mixture of autoregressive models, where the weighting coefficients are determined through latent random variables as functions of all past observations. These variables follow a hidden Markov model. We modify EM and Baum-Welch algorithms to estimate the parameters of the model.
We propose three methods for forecasting a time series modeled using a functional coefficient autoregressive model (FCAR) fit via spline-backfitted local linear (SBLL) smoothing. The three methods are a "naive" plug-in method, a bootstrap method, and a multistage method. We present asymptotic results of the SBLL estimation method for FCAR models and show the estimators are oracally efficient. The three forecasting methods are compared through simulation. We find that the naive method performs just as well as the multistage method and even outperforms it in some situations. We apply the naive a…
Probabilistic forecasting, i.e. estimating the probability distribution of a time series' future given its past, is a key enabler for optimizing business processes. In retail businesses, for example, forecasting demand is crucial for having the right inventory available at the right time at the right place. In this paper we propose DeepAR, a methodology for producing accurate probabilistic forecasts, based on training an auto regressive recurrent network model on a large number of related time series. We demonstrate how by applying deep learning techniques to forecasting, one can overcome many…
Although Transformer has made breakthrough success in widespread domains especially in Natural Language Processing (NLP), applying it to time series forecasting is still a great challenge. In time series forecasting, the autoregressive decoding of canonical Transformer models could introduce huge accumulative errors inevitably. Besides, utilizing Transformer to deal with spatial-temporal dependencies in the problem still faces tough difficulties.~To tackle these limitations, this work is the first attempt to propose a Non-Autoregressive Transformer architecture for time series forecasting, aim…
We construct efficient robust truncated sequential estimators for the pointwise estimation problem in nonparametric autoregression models with smooth coefficients. For Gaussian models we propose an adaptive procedure based on the constructed sequential estimators. The minimax nonadaptive and adaptive convergence rates are established. It turns out that in this case these rates are the same as for regression models.
Bifurcating autoregressive processes, which can be seen as an adaptation of au-toregressive processes for a binary tree structure, have been extensively studied during the last decade in a parametric context. In this work we do not specify any a priori form for the two autoregressive functions and we use nonparametric techniques. We investigate both nonasymp-totic and asymptotic behavior of the Nadaraya-Watson type estimators of the autoregressive functions. We build our estimators observing the process on a finite subtree denoted by Tn, up to the depth n. Estimators achieve the classical rate…
The purpose of this paper is to propose a time-varying vector autoregressive model (TV-VAR) for forecasting multivariate time series. The model is casted into a state-space form that allows flexible description and analysis. The volatility covariance matrix of the time series is modelled via inverted Wishart and singular multivariate beta distributions allowing a fully conjugate Bayesian inference. Model performance and model comparison is done via the likelihood function, sequential Bayes factors, the mean of squared standardized forecast errors, the mean of absolute forecast errors (known al…
Department of Computer Science, ETH Zürich（苏黎世联邦理工学院计算机科学系）；Department of Computer Science, ETH Zürich, Switzerland（苏黎世联邦理工学院计算机科学系）
We develop an effective generation of adversarial attacks on neural models that output a sequence of probability distributions rather than a sequence of single values. This setting includes the recently proposed deep probabilistic autoregressive forecasting models that estimate the probability distribution of a time series given its past and achieve state-of-the-art results in a diverse set of application domains. The key technical challenge we address is effectively differentiating through the Monte-Carlo estimation of statistics of the joint distribution of the output sequence. Additionally,…
This paper studies the threshold estimation of a TAR model when the underlying threshold parameter is a random variable. It is shown that the Bayesian estimator is consistent and its limit distribution is expressed in terms of a limit likelihood ratio. Furthermore, convergence of moments of the estimators is also established. The limit distribution can be computed via explicit simulations from which testing and inference for the threshold parameter can be conducted. The obtained results are illustrated with numerical simulations.
Lung tumor tracking for radiotherapy requires real-time, multiple-step ahead forecasting of a quasi-periodic time series recording instantaneous tumor locations. We introduce a location-mixture autoregressive (LMAR) process that admits multimodal conditional distributions, fast approximate inference using the EM algorithm and accurate multiple-step ahead predictive distributions. LMAR outperforms several commonly used methods in terms of out-of-sample prediction accuracy using clinical data from lung tumor patients. With its superior predictive performance and real-time computation, the LMAR m…
This study was conducted to find an appropriate statistical model to forecast the volatilities of PSEi using the model Generalized Autoregressive Conditional Heteroskedasticity (GARCH). Using the R software, the log returns of PSEi is modeled using various ARIMA models and with the presence of heteroskedasticity, the log returns was modeled using GARCH. Based on the analysis, GARCH models are the most appropriate to use for the log returns of PSEi. Among the selected GARCH models, GARCH (1,2) has the lowest AIC value and also has the highest LL value implying that GARCH (1,2) is the best model…
Toronto Metropolitan University（多伦多都会大学）；University of Toronto（多伦多大学）
Despite the high performance of neural network-based time series forecasting methods, the inherent challenge in explaining their predictions has limited their applicability in certain application areas. Due to the difficulty in identifying causal relationships between the input and output of such black-box methods, they rarely have been adopted in domains such as legal and medical fields in which the reliability and interpretability of the results can be essential. In this paper, we propose \model, a novel deep learning-based probabilistic time series forecasting architecture that is intrinsic…
This work concerns estimation of linear autoregressive models with Markov-switching using expectation maximisation (E.M.) algorithm. Our method generalise the method introduced by Elliot for general hidden Markov models and avoid to use backward recursion.
In this article we develop a tractable procedure for testing strict stationarity in a double autoregressive model and formulate the problem as testing if the top Lyapunov exponent is negative. Without strict stationarity assumption, we construct a consistent estimator of the associated top Lyapunov exponent and employ a random weighting approach for its variance estimation, which in turn are used in a t-type test. We also propose a GLAD estimation for parameters of interest, relaxing key assumptions on the commonly used QMLE. All estimators, except for the intercept, are shown to be consistent…
Qualcomm AI Research（高通AI研究院）；Qualcomm Technologies Netherlands B.V.（高通荷兰技术有限公司）；Qualcomm Technologies, Inc.（高通技术公司）；University of Amsterdam（阿姆斯特丹大学）
Autoregressive models (ARMs) currently hold state-of-the-art performance in likelihood-based modeling of image and audio data. Generally, neural network based ARMs are designed to allow fast inference, but sampling from these models is impractically slow. In this paper, we introduce the predictive sampling algorithm: a procedure that exploits the fast inference property of ARMs in order to speed up sampling, while keeping the model intact. We propose two variations of predictive sampling, namely sampling with ARM fixed-point iteration and learned forecasting modules. Their effectiveness is dem…
We investigate the estimation of parameters in the random coefficient autoregressive model. We consider a nonstationary RCA process and show that the innovation variance parameter cannot be estimated by the quasi-maximum likelihood method. The asymptotic normality of the quasi-maximum likelihood estimator for the remaining model parameters is proven so the unit root problem does not exist in the random coefficient autoregressive model.
University of Tübingen（图宾根大学）；Amazon Research（亚马逊研究院）；Technical University of Munich（慕尼黑工业大学）
Despite the increasing relevance of forecasting methods, causal implications of these algorithms remain largely unexplored. This is concerning considering that, even under simplifying assumptions such as causal sufficiency, the statistical risk of a model can differ significantly from its \textit{causal risk}. Here, we study the problem of \textit{causal generalization} -- generalizing from the observational to interventional distributions -- in forecasting. Our goal is to find answers to the question: How does the efficacy of an autoregressive (VAR) model in predicting statistical association…
Consider an autoregressive model with measurement error: we observe $Z_i=X_i+ε_i$, where $X_i$ is a stationary solution of the equation $X_i=f_{θ^0}(X_{i-1})+ξ_i$. The regression function $f_{θ^0}$ is known up to a finite dimensional parameter $θ^0$. The distributions of $X_0$ and $ξ_1$ are unknown whereas the distribution of $ε_1$ is completely known. We want to estimate the parameter $θ^0$ by using the observations $Z_0,..,Z_n$. We propose an estimation procedure based on a modified least square criterion involving a weight function $w$, to be suitably chosen. We give upper bounds for the ri…
Spatial unilateral autoregressive model $X_{k,\ell}=αX_{k-1,\ell}+βX_{k,\ell-1}+γX_{k-1,\ell-1}+ε_{k,\ell}$ is investigated in the unit root case, that is when the parameters are on the boundary of the domain of stability that forms a tetrahedron with vertices $(1,1,-1), \ (1,-1,1),\ (-1,1,1)$ and $(-1,-1,-1)$. It is shown that the limiting distribution of the least squares estimator of the parameters is normal and the rate of convergence is $n$ when the parameters are in the faces or on the edges of the tetrahedron, while on the vertices the rate is $n^{3/2}$.
We develop a Bayesian median autoregressive (BayesMAR) model for time series forecasting. The proposed method utilizes time-varying quantile regression at the median, favorably inheriting the robustness of median regression in contrast to the widely used mean-based methods. Motivated by a working Laplace likelihood approach in Bayesian quantile regression, BayesMAR adopts a parametric model bearing the same structure as autoregressive models by altering the Gaussian error to Laplace, leading to a simple, robust, and interpretable modeling strategy for time series forecasting. We estimate model…
Knowledge of the current state of economies, how they respond to COVID-19 mitigations and indicators, and what the future might hold for them is important. We use recently-developed generalised network autoregressive (GNAR) models, using trade-determined networks, to model and forecast the Purchasing Managers' Indices for a number of countries. We use networks that link countries where the links themselves, or their weights, are determined by the degree of export trade between the countries. We extend these models to include node-specific time series exogenous variables (GNARX models), using t…
Large sample statistical analysis of threshold autoregressive (TAR) models is usually based on the assumption that the underlying driving noise is uncorrelated. In this paper, we consider a model, driven by Gaussian noise with geometric correlation tail and derive a complete characterization of the asymptotic distribution for the Bayes estimator of the threshold parameter.
Autoregressive with Slack Time Series Model for Forecasting a Partially-Observed Dynamical Time Series
The Institute of Statistical Mathematics（统计数理研究所）；RIKEN Center for Advanced Intelligence Project（理化学研究所先进智能项目中心）；Kyoto University（京都大学）；Hitotsubashi University（一桥大学）
Time series prediction has been a long-standing research topic and an essential application in many domains. Modern time series collected from sensor networks (e.g., energy consumption and traffic flow) are often large-scale and incomplete with considerable corruption and missing values, making it difficult to perform accurate predictions. In this paper, we propose a low-rank autoregressive tensor completion (LATC) framework to model multivariate time series data. The key of LATC is to transform the original multivariate time series matrix (e.g., sensor$\times$time point) to a third-order tens…
Universidad Carlos III de Madrid（马德里卡洛斯三世大学）；Health Research Institute Gregorio Marañón（格雷戈里奥·马拉尼翁健康研究所）；CIBERSAM, ISCIII（西班牙生物医学研究网络中心（卡洛斯三世健康研究所））
Time series forecasting is an important problem across many domains, playing a crucial role in multiple real-world applications. In this paper, we propose a forecasting architecture that combines deep autoregressive models with a Spectral Attention (SA) module, which merges global and local frequency domain information in the model's embedded space. By characterizing in the spectral domain the embedding of the time series as occurrences of a random process, our method can identify global trends and seasonality patterns. Two spectral attention models, global and local to the time series, integr…
Autoregressive (AR) models are useful tools in time series analysis. Inferences under such models are distorted in the presence of measurement error, which is very common in practice. In this article, we establish analytical results for quantifying the biases of the parameter estimation in AR models if the measurement error effects are neglected. We propose two measurement error models to describe different processes of data contamination. An estimating equation approach is proposed for the estimation of the model parameters with measurement error effects accounted for. We further discuss fore…
Motivated by a variety of applications, high-dimensional time series have become an active topic of research. In particular, several methods and finite-sample theories for individual stable autoregressive processes with known lag have become available very recently. We, instead, consider multiple stable autoregressive processes that share an unknown lag. We use information across the different processes to simultaneously select the lag and estimate the parameters. We prove that the estimated process is stable, and we establish rates for the forecasting error that can outmatch the known rate in…
Many wind speed forecasting approaches have been proposed in literature. In this paper a new statistical approach for jointly predicting wind speed, wind direction and air pressure is introduced. The wind direction and the air pressure are important to extend the forecasting accuracy of wind speed forecasts. A good forecast for the wind direction helps to bring the turbine into the predominant wind direction. We combine a multivariate seasonal time varying threshold autoregressive model with interactions (TVARX) with a threshold seasonal autoregressive conditional heteroscedastic (TARCHX) mode…
In several disciplines it is common to find time series measured at irregular observational times. In particular, in astronomy there are a large number of surveys that gather information over irregular time gaps and in more than one passband. Some examples are Pan-STARRS, ZTF and also the LSST. However, current commonly used time series models that estimate the time dependency in astronomical light curves consider the information of each band separately (e.g, CIAR, IAR and CARMA models) disregarding the dependency that might exist between different passbands. In this paper we propose a novel b…
This study evaluates the performance of cointegrated vector autoregressive (VAR) models for very short- and short-term wind power forecasting. Preliminary results for a German data set comprising six wind power production time series indicate that taking into account potential cointegrating relations between the individual series can improve forecasts at short-term time horizons.
As a result of the greater availability of big data, as well as the decreasing costs and increasing power of modern computing, the use of artificial neural networks for financial time series forecasting is once again a major topic of discussion and research in the financial world. Despite this academic focus, there are still contrasting opinions and bodies of literature on which artificial neural networks perform the best and whether or not they outperform the forecasting capabilities of conventional time series models. This paper uses a long-short term memory network to perform financial time…
When considering the problem of forecasting a continuous-time stochastic process over an entire time-interval in terms of its recent past, the notion of Autoregressive Hilbert space processes (ARH) arises. This model can be seen as a generalization of the classical autoregressive processes to Hilbert space valued random variables. Its estimation presents several challenges that were addressed by many authors in recent years. In this paper, we propose an extension based on this model by introducing a conditioning process on the arh. In this way, we are aiming a double objective. First, the intr…
Hainan University（海南大学）；University of Chinese Academy of Science（中国科学院大学）
Traffic flow forecasting is hot spot research of intelligent traffic system construction. The existing traffic flow prediction methods have problems such as poor stability, high data requirements, or poor adaptability. In this paper, we define the traffic data time singularity ratio in the dropout module and propose a combination prediction method based on the improved long short-term memory neural network and time series autoregressive integrated moving average model (SDLSTM-ARIMA), which is derived from the Recurrent Neural Networks (RNN) model. It compares the traffic data time singularity …
Univariate time series often take the form of a collection of curves observed sequentially over time. Examples of these include hourly ground-level ozone concentration curves. These curves can be viewed as a time series of functions observed at equally spaced intervals over a dense grid. Since functional time series may contain various types of outliers, we introduce a robust functional time series forecasting method to down-weigh the influence of outliers in forecasting. Through a robust principal component analysis based on projection pursuit, a time series of functions can be decomposed int…
By significant improvements in modern electrical systems, planning for unit commitment and power dispatching of them are two big concerns between the researchers. Short-term load forecasting plays a significant role in planning and dispatching them. In recent years, numerous works have been done on Short-term load forecasting. Having an accurate model for predicting the load can be beneficial for optimizing the electrical sources and protecting energy. Several models such as Artificial Intelligence and Statistics model have been used to improve the accuracy of load forecasting. Among the stati…
Data consisting of time-indexed distributions of cross-sectional or intraday returns have been extensively studied in finance, and provide one example in which the data atoms consist of serially dependent probability distributions. Motivated by such data, we propose an autoregressive model for density time series by exploiting the tangent space structure on the space of distributions that is induced by the Wasserstein metric. The densities themselves are not assumed to have any specific parametric form, leading to flexible forecasting of future unobserved densities. The main estimation targets…
In this paper, we propose multi-variable LSTM capable of accurate forecasting and variable importance interpretation for time series with exogenous variables. Current attention mechanism in recurrent neural networks mostly focuses on the temporal aspect of data and falls short of characterizing variable importance. To this end, the multi-variable LSTM equipped with tensorized hidden states is developed to learn hidden states for individual variables, which give rise to our mixture temporal and variable attention. Based on such attention mechanism, we infer and quantify variable importance. Ext…
In this article, we propose the fractional lower order covariance method (FLOC) for estimating the parameters of vector autoregressive process (VAR) of order $p$, $p\geq 1$ with symmetric stable noise. Further, we show the efficiency, accuracy, and simplicity of our methods through Monte-Carlo simulation.
A new forecasting method based on the concept of the profile predictive the likelihood function is proposed for discrete-valued processes. In particular, generalized autoregressive and moving average (GARMA) models for Poisson distributed data are explored in details. Highest density regions are used to construct forecasting regions. The proposed forecast estimates and regions are coherent. Large sample results are derived for the forecasting distribution. Numerical studies using simulations and a real data set are used to establish the performance of the proposed forecasting method. Robustnes…
Time series forecasting is often fundamental to scientific and engineering problems and enables decision making. With ever increasing data set sizes, a trivial solution to scale up predictions is to assume independence between interacting time series. However, modeling statistical dependencies can improve accuracy and enable analysis of interaction effects. Deep learning methods are well suited for this problem, but multivariate models often assume a simple parametric distribution and do not scale to high dimensions. In this work we model the multivariate temporal dynamics of time series via a…
In river flow analysis and forecasting there are some key elements to consider in order to obtain reliable results. For example, seasonality is often accounted for in statistical models because climatic oscillations occurring every year have an obvious impact on river flow. Further sources of alteration could be caused by changes in reservoir management, instrumentation or even unexpected shifts in climatic conditions. When these changes are ignored the statistical results can be strongly misleading. This paper develops an automatic procedure to estimate number and locations of changepoints in…
Linear autoregressive models serve as basic representations of discrete time stochastic processes. Different attempts have been made to provide non-linear versions of the basic autoregressive process, including different versions based on kernel methods. Motivated by the powerful framework of Hilbert space embeddings of distributions, in this paper we apply this methodology for the kernel embedding of an autoregressive process of order $p$. By doing so, we provide a non-linear version of an autoregressive process, that shows increased performance over the linear model in highly complex time se…
A year following the initial COVID-19 outbreak in China, many countries have approved emergency vaccines. Public-health practitioners and policymakers must understand the predicted populational willingness for vaccines and implement relevant stimulation measures. This study developed a framework for predicting vaccination uptake rate based on traditional clinical data-involving an autoregressive model with autoregressive integrated moving average (ARIMA)- and innovative web search queries-involving a linear regression with ordinary least squares/least absolute shrinkage and selection operator,…
When performing a time series analysis of continuous data, for example from climate or environmental problems, the assumption that the process is Gaussian is often violated. Therefore, we introduce two non-Gaussian autoregressive time series models that are able to fit skewed and heavy-tailed time series data. Our two models are based on the Tukey g-and-h transformation. We discuss parameter estimation, order selection, and forecasting procedures for our models and examine their performances in a simulation study. We demonstrate the usefulness of our models by applying them to two sets of wind…
