We provide a new strategy built on the divide-and-conquer approach by Lindsten et al. (2017) to investigate the smoothing problem in a hidden Markov model. We employ this approach to decompose a hidden Markov model into sub-models with intermediate target distributions based on an auxiliary tree structure and produce independent samples from the sub-models at the leaf nodes towards the original model of interest at the root. We review the target distribution in the sub-models suggested by Lindsten et al. and propose two new classes of target distributions, which are the estimates of the (joint) filtering distributions and the (joint) smoothing distributions. The first proposed type is straightforwardly constructible by running a filtering algorithm in advance. The algorithm using the second type of target distributions has an advantage of roughly retaining the marginals of all random variables invariant at all levels of the tree at the cost of approximating the marginal smoothing distributions in advance. We further propose the constructions of these target distributions using pre-generated Monte Carlo samples. We show empirically the algorithms with the proposed intermediate target distributions give stable and comparable results as the conventional smoothing methods in a linear Gaussian model and a non-linear model.
In this article we consider the smoothing problem for hidden Markov models (HMM). Given a hidden Markov chain $\{X_n\}_{n\geq 0}$ and observations $\{Y_n\}_{n\geq 0}$, our objective is to compute $\mathbb{E}[φ(X_0,\dots,X_k)|y_{0},\dots,y_n]$ for some real-valued, integrable functional $φ$ and $k$ fixed, $k \ll n$ and for some realisation $(y_0,\dots,y_n)$ of $(Y_0,\dots,Y_n)$. We introduce a novel application of the multilevel Monte Carlo (MLMC) method with a coupling based on the Knothe-Rosenblatt rearrangement. We prove that this method can approximate the afore-mentioned quantity with a me…
In this note we introduce an estimate for the marginal likelihood associated to hidden Markov models (HMMs) using sequential Monte Carlo (SMC) approximations of the generalized two-filter smoothing decomposition (Briers, 2010). This estimate is shown to be unbiased and a central limit theorem (CLT) is established. This latter CLT also allows one to prove a CLT associated to estimates of expectations w.r.t. a marginal of the joint smoothing distribution; these form some of the first theoretical results associated to the SMC approximation of the generalized two-filter smoothing decomposition. Th…
Hidden Markov models provide a natural statistical framework for the detection of the copy number variations (CNV) in genomics. In this paper, we consider a Hidden Markov Model involving several correlated hidden processes at the same time. When dealing with a large number of series, maximum likelihood inference (performed classically using the EM algorithm) becomes intractable. We thus propose an approximate inference algorithm based on a variational approach (VEM). A simulation study is performed to assess the performance of the proposed method and an application to the detection of structur…
We propose a unified framework that extends the inference methods for classical hidden Markov models to continuous settings, where both the hidden states and observations occur in continuous time. Two different settings are analyzed: hidden jump process with a finite state space, and hidden diffusion process with a continuous state space. For each setting, we first estimate the hidden states given the observations and model parameters, showing that the posterior distribution of the hidden states can be described by differential equations in continuous time. We then consider the estimation of u…
Particle smoothing methods are used for inference of stochastic processes based on noisy observations. Typically, the estimation of the marginal posterior distribution given all observations is cumbersome and computational intensive. In this paper, we propose a simple algorithm based on path integral control theory to estimate the smoothing distribution of continuous-time diffusion processes with partial observations. In particular, we use an adaptive importance sampling method to improve the effective sampling size of the posterior over processes given the observations and the reliability of …
As an important Markov Chain Monte Carlo (MCMC) method, stochastic gradient Langevin dynamics (SGLD) algorithm has achieved great success in Bayesian learning and posterior sampling. However, SGLD typically suffers from slow convergence rate due to its large variance caused by the stochastic gradient. In order to alleviate these drawbacks, we leverage the recently developed Laplacian Smoothing (LS) technique and propose a Laplacian smoothing stochastic gradient Langevin dynamics (LS-SGLD) algorithm. We prove that for sampling from both log-concave and non-log-concave densities, LS-SGLD achieve…
We propose sequential Monte Carlo based algorithms for maximum likelihood estimation of the static parameters in hidden Markov models with an intractable likelihood using ideas from approximate Bayesian computation. The static parameter estimation algorithms are gradient based and cover both offline and online estimation. We demonstrate their performance by estimating the parameters of three intractable models, namely the alpha-stable distribution, g-and-k distribution, and the stochastic volatility model with alpha-stable returns, using both real and synthetic data.
We propose to model time-varying periodic and oscillatory processes by means of a hidden Markov model where the states are defined through the spectral properties of a periodic regime. The number of states is unknown along with the relevant periodicities, the role and number of which may vary across states. We address this inference problem by a Bayesian nonparametric hidden Markov model assuming a sticky hierarchical Dirichlet process for the switching dynamics between different states while the periodicities characterizing each state are explored by means of a trans-dimensional Markov chain …
We consider the approximation of expectations with respect to the distribution of a latent Markov process given noisy measurements. This is known as the smoothing problem and is often approached with particle and Markov chain Monte Carlo (MCMC) methods. These methods provide consistent but biased estimators when run for a finite time. We propose a simple way of coupling two MCMC chains built using Particle Independent Metropolis-Hastings (PIMH) to produce unbiased smoothing estimators. Unbiased estimators are appealing in the context of parallel computing, and facilitate the construction of co…
This is a supplementary material to the paper "Online Expectation Maximization based algorithms for inference in hidden Markov models". It contains further technical derivations and additional simulation results.
We give simple conditions that ensure exponential forgetting of the initial conditions of the filter for general state-space hidden Markov chain. The proofs are based on the coupling argument applied to the posterior Markov kernels. These results are useful both for filtering hidden Markov models using approximation methods (e.g., particle filters) and for proving asymptotic properties of estimators. The results are general enough to cover models like the Gaussian state space model, without using the special structure that permits the application of the Kalman filter.
Stochastic variational inference for collapsed models has recently been successfully applied to large scale topic modelling. In this paper, we propose a stochastic collapsed variational inference algorithm for hidden Markov models, in a sequential data setting. Given a collapsed hidden Markov Model, we break its long Markov chain into a set of short subchains. We propose a novel sum-product algorithm to update the posteriors of the subchains, taking into account their boundary transitions due to the sequential dependencies. Our experiments on two discrete datasets show that our collapsed algor…
The study of animal behavioural states inferred through hidden Markov models and similar state switching models has seen a significant increase in popularity in recent years. The ability to account for varying levels of behavioural scale has become possible through hierarchical hidden Markov models, but additional levels lead to higher complexity and increased correlation between model components. Maximum likelihood approaches to inference using the EM algorithm and direct optimisation of likelihoods are more frequently used, with Bayesian approaches being less favoured due to computational de…
To estimate the emission parameters in hidden Markov models one commonly uses the EM algorithm or its variation. Our primary motivation, however, is the Philips speech recognition system wherein the EM algorithm is replaced by the Viterbi training algorithm. Viterbi training is faster and computationally less involved than EM, but it is also biased and need not even be consistent. We propose an alternative to the Viterbi training -- adjusted Viterbi training -- that has the same order of computational complexity as Viterbi training but gives more accurate estimators. Elsewhere, we studied the …
By making use of martingale representations, we derive the asymptotic normality of particle filters in hidden Markov models and a relatively simple formula for their asymptotic variances. Although repeated resamplings result in complicated dependence among the sample paths, the asymptotic variance formula and martingale representations lead to consistent estimates of the standard errors of the particle filter estimates of the hidden states.
University of California, San Diego（加州大学圣地亚哥分校）；City University of Hong Kong（香港城市大学）
The hidden Markov model (HMM) is a generative model that treats sequential data under the assumption that each observation is conditioned on the state of a discrete hidden variable that evolves in time as a Markov chain. In this paper, we derive a novel algorithm to cluster HMMs through their probability distributions. We propose a hierarchical EM algorithm that i) clusters a given collection of HMMs into groups of HMMs that are similar, in terms of the distributions they represent, and ii) characterizes each group by a "cluster center", i.e., a novel HMM that is representative for the group. …
We present a locally adaptive nonparametric curve fitting method that operates within a fully Bayesian framework. This method uses shrinkage priors to induce sparsity in order-k differences in the latent trend function, providing a combination of local adaptation and global control. Using a scale mixture of normals representation of shrinkage priors, we make explicit connections between our method and kth order Gaussian Markov random field smoothing. We call the resulting processes shrinkage prior Markov random fields (SPMRFs). We use Hamiltonian Monte Carlo to approximate the posterior distri…
In this paper, we prove that finite state space non parametric hidden Markov models are identifiable as soon as the transition matrix of the latent Markov chain has full rank and the emission probability distributions are linearly independent. We then propose several non parametric likelihood based estimation methods, which we apply to models used in applications. We finally show on examples that the use of non parametric modeling and estimation may improve the classification performances.
We introduce multiple hidden Markov models (MHMMs) where an observed multivariate categorical time series depends on an unobservable multivariate Mar- kov chain. MHMMs provide an elegant framework for specifying various independence relationships between multiple discrete time processes. These independencies are interpreted as Markov properties of a mixed graph and a chain graph associated to the latent and observable components of the MHMM, respectively. These Markov properties are also translated into zero restrictions on the parameters of marginal models for the transition probabilities and…
In this article a flexible Bayesian non-parametric model is proposed for non-homogeneous hidden Markov models. The model is developed through the amalgamation of the ideas of hidden Markov models and predictor dependent stick-breaking processes. Computation is carried out using auxiliary variable representation of the model which enable us to perform exact MCMC sampling from the posterior. Furthermore, the model is extended to the situation when the predictors can simultaneously in influence the transition dynamics of the hidden states as well as the emission distribution. Estimates of few ste…
This paper presents a method for calculating the smoothed state distribution for Jump Markov Linear Systems. More specifically, the paper details a novel two-filter smoother that provides closed-form expressions for the smoothed hybrid state distribution. This distribution can be expressed as a Gaussian mixture with a known, but exponentially increasing, number of Gaussian components as the time index increases. This is accompanied by exponential growth in memory and computational requirements, which rapidly becomes intractable. To ameliorate this, we limit the number of allowed mixture terms …
This paper reviews recent advances in Bayesian nonparametric techniques for constructing and performing inference in infinite hidden Markov models. We focus on variants of Bayesian nonparametric hidden Markov models that enhance a posteriori state-persistence in particular. This paper also introduces a new Bayesian nonparametric framework for generating left-to-right and other structured, explicit-duration infinite hidden Markov models that we call the infinite structured hidden semi-Markov model.
We consider missing data in the context of hidden Markov models with a focus on situations where data is missing not at random (MNAR) and missingness depends on the identity of the hidden states. In simulations, we show that including a submodel for state-dependent missingness reduces bias when data is MNAR and state-dependent, whilst not reducing accuracy when data is missing at random (MAR). When missingness depends on time but not the hidden states, a model which only allows for state-dependent missingness is biased, whilst a model that allows for both state- and time-dependent missingness …
A simple linear algebraic explanation of the algorithm in "A Spectral Algorithm for Learning Hidden Markov Models" (COLT 2009). Most of the content is in Figure 2; the text just makes everything precise in four nearly-trivial claims.
Hidden Markov models can describe time series arising in various fields of science, by treating the data as noisy measurements of an arbitrarily complex Markov process. Sequential Monte Carlo (SMC) methods have become standard tools to estimate the hidden Markov process given the observations and a fixed parameter value. We review some of the recent developments allowing the inclusion of parameter uncertainty as well as model uncertainty. The shortcomings of the currently available methodology are emphasised from an algorithmic complexity perspective. The statistical objects of interest for ti…
We consider the problem of filtering an unseen Markov chain from noisy observations, in the presence of uncertainty regarding the parameters of the processes involved. Using the theory of nonlinear expectations, we describe the uncertainty in terms of a penalty function, which can be propagated forward in time in the place of the filter.
There is much interest in the Hierarchical Dirichlet Process Hidden Markov Model (HDP-HMM) as a natural Bayesian nonparametric extension of the ubiquitous Hidden Markov Model for learning from sequential and time-series data. However, in many settings the HDP-HMM's strict Markovian constraints are undesirable, particularly if we wish to learn or encode non-geometric state durations. We can extend the HDP-HMM to capture such structure by drawing upon explicit-duration semi-Markovianity, which has been developed mainly in the parametric frequentist setting, to allow construction of highly interp…
We consider parameter estimation in finite hidden state space Markov models with time-dependent inhomogeneous noise, where the inhomogeneity vanishes sufficiently fast. Based on the concept of asymptotic mean stationary processes we prove that the maximum likelihood and a quasi-maximum likelihood estimator (QMLE) are strongly consistent. The computation of the QMLE ignores the inhomogeneity, hence, is much simpler and robust. The theory is motivated by an example from biophysics and applied to a Poisson- and linear Gaussian model.
Financial markets exhibit alternating periods of rising and falling prices. Stock traders seeking to make profitable investment decisions have to account for those trends, where the goal is to accurately predict switches from bullish towards bearish markets and vice versa. Popular tools for modeling financial time series are hidden Markov models, where a latent state process is used to explicitly model switches among different market regimes. In their basic form, however, hidden Markov models are not capable of capturing both short- and long-term trends, which can lead to a misinterpretation o…
