Many real-world problems encountered in several disciplines deal with the modeling of time-series containing different underlying dynamical regimes, for which probabilistic approaches are very often employed. In this paper we describe several such approaches in the common framework of graphical models. We give a unified overview of models previously introduced in the literature, which is simpler and more comprehensive than previous descriptions and enables us to highlight commonalities and differences among models that were not observed in the past. In addition, we present several new models and inference routines, which are naturally derived within this unified viewpoint.
Bilkent University（比尔肯特大学）；DataBoss A. S.（DataBoss股份公司）
We investigate nonlinear regression for nonstationary sequential data. In most real-life applications such as business domains including finance, retail, energy and economy, timeseries data exhibits nonstationarity due to the temporally varying dynamics of the underlying system. We introduce a novel recurrent neural network (RNN) architecture, which adaptively switches between internal regimes in a Markovian way to model the nonstationary nature of the given data. Our model, Markovian RNN employs a hidden Markov model (HMM) for regime transitions, where each regime controls hidden state transi…
Time series and sequential data have gained significant attention recently since many real-world processes in various domains such as finance, education, biology, and engineering can be modeled as time series. Although many algorithms and methods such as the Kalman filter, hidden Markov model, and long short term memory (LSTM) are proposed to make inferences and predictions for the data, their usage significantly depends on the application, type of the problem, available data, and sufficient accuracy or loss. In this paper, we compare the supervised and unsupervised hidden Markov model to LSTM…
This paper investigates the role of high-dimensional information sets in the context of Markov switching models with time varying transition probabilities. Markov switching models are commonly employed in empirical macroeconomic research and policy work. However, the information used to model the switching process is usually limited drastically to ensure stability of the model. Increasing the number of included variables to enlarge the information set might even result in decreasing precision of the model. Moreover, it is often not clear a priori which variables are actually relevant when it c…
This paper develops tests of the null hypothesis of linearity in the context of autoregressive models with Markov-switching means and variances. These tests are robust to the identification failures that plague conventional likelihood-based inference methods. The approach exploits the moments of normal mixtures implied by the regime-switching process and uses Monte Carlo test techniques to deal with the presence of an autoregressive component in the model specification. The proposed tests have very respectable power in comparison to the optimal tests for Markov-switching parameters of Carrasco…
We perform a detailed comparison between a Markov Switching Jump Diffusion Model and a Markov Switching α-Stable Distribution Model with respect to the analysis of non-stationary data. We show that the jump diffusion model is extremely robust, flexible and accurate in fitting of financial time series. A thorough computational study involving the two models being applied to real data, namely, the S&P500 index, is provided. The study shows that the jump-diffusion model solves the over-smoothing issue stated in (Di Persio and Frigo, 2016), while the α-stable distribution approach is a good compro…
State-space models (SSM) with Markov switching offer a powerful framework for detecting multiple regimes in time series, analyzing mutual dependence and dynamics within regimes, and asserting transitions between regimes. These models however present considerable computational challenges due to the exponential number of possible regime sequences to account for. In addition, high dimensionality of time series can hinder likelihood-based inference. This paper proposes novel statistical methods for Markov-switching SSMs using maximum likelihood estimation, Expectation-Maximization (EM), and parame…
Markovian-regime-switching (MRS) models are commonly used for modelling economic time series, including electricity prices where independent regime models are used, since they can more accurately and succinctly capture electricity price dynamics than dependent regime MRS models can. We can think of these independent regime MRS models for electricity prices as a collection of independent AR(1) processes, of which only one process is observed at each time; which is observed is determined by a (hidden) Markov chain. Here we develop novel, computationally feasible methods for MRS models with indep…
Spatio-temporal counts of infectious disease cases often contain an excess of zeros. With existing zero inflated count models applied to such data it is difficult to quantify space-time heterogeneity in the effects of disease spread between areas. Also, existing methods do not allow for separate dynamics to affect the reemergence and persistence of the disease. As an alternative, we develop a new zero-state coupled Markov switching negative binomial model, under which the disease switches between periods of presence and absence in each area through a series of partially hidden nonhomogeneous M…
Dipartimento di Informatica - Università di Pisa（比萨大学计算机系）
Hidden tree Markov models allow learning distributions for tree structured data while being interpretable as nondeterministic automata. We provide a concise summary of the main approaches in literature, focusing in particular on the causality assumptions introduced by the choice of a specific tree visit direction. We will then sketch a novel non-parametric generalization of the bottom-up hidden tree Markov model with its interpretation as a nondeterministic tree automaton with infinite states.
Tsinghua University（清华大学）；University of Washington（华盛顿大学）；University of California, Los Angeles（加利福尼亚大学洛杉矶分校）
Many real-world applications, such as those in medical domains, recommendation systems, etc, can be formulated as large state space reinforcement learning problems with only a small budget of the number of policy changes, i.e., low switching cost. This paper focuses on the linear Markov Decision Process (MDP) recently studied in [Yang et al 2019, Jin et al 2020] where the linear function approximation is used for generalization on the large state space. We present the first algorithm for linear MDP with a low switching cost. Our algorithm achieves an $\widetilde{O}\left(\sqrt{d^3H^4K}\right)$ …
