On the reversibility of the observed process of three-state hidden Markov model

可逆不精确马尔可夫链

Reversible Imprecise Markov Chains

本文为可逆不精确马尔可夫链建立理论框架，引入基于两步联合分布矩阵信任集的对称表示，将可逆性简化为矩阵对称性，并用于计算路径依赖泛函的期望。

Analysis of a non-reversible Markov chain speedup by a single edge

We present a Markov chain example where non-reversibility and an added edge jointly improve mixing time: when a random edge is added to a cycle of $n$ vertices and a Markov chain with a drift is introduced, we get mixing time of $O(n^{3/2})$ with probability bounded away from 0. If only one of the two modifications were performed, the mixing time would stay $Ω(n^2)$.

Convergence in distribution for filtering processes associated to Hidden Markov Models with densities

Consider a filtering process associated to a hidden Markov model with densities for which both the state space and the observation space are complete, separable, metric spaces. If the underlying, hidden Markov chain is strongly ergodic and the filtering process fulfills a certain coupling condition we prove that, in the limit, the distribution of the filtering process is independent of the initial distribution of the hidden Markov chain. If furthermore the hidden Markov chain is uniformly ergodic, then we prove that the filtering process converges in distribution.

On hidden Markov chains and finite stochastic systems

In this paper we study various properties of finite stochastic systems or hidden Markov chains as they are alternatively called. We discuss their construction following different approaches and we also derive recursive filtering formulas for the different systems that we consider. The key tool is a simple lemma on conditional expectations.

Robust Filtering and Propagation of Uncertainty in Hidden Markov Models

We consider the filtering of continuous-time finite-state hidden Markov models, where the rate and observation matrices depend on unknown time-dependent parameters, for which no prior or stochastic model is available. We quantify and analyze how the induced uncertainty may be propagated through time as we collect new observations, and used to simultaneously provide robust estimates of the hidden signal and to learn the unknown parameters, via techniques based on pathwise filtering and new results on the optimal control of rough differential equations.

Markov 与隐 Markov 过程研究中的散度度量

Divergence metrics in the study of Markov and hidden Markov processes

本文综述并统一连续时间 Markov 与隐 Markov 过程中的 f-散度，展示其在随机稳定性、热力学、滤波器稳定性及 Maxwell 妖模型中的应用。

Area anomaly in the rough path Brownian scaling limit of hidden Markov walks

We study the convergence in rough path topology of a certain class of discrete processes, the hidden Markov walks, to a Brownian motion with an area anomaly. This area anomaly, which is a new object, keeps track of the time-correlation of the discrete models and brings into light the question of embeddings of discrete processes into continuous time. We also identify an underlying combinatorial structure in the hidden Markov walks, which turns out to be a generalization of the occupation time from the classical ergodic theorem in the spirit of rough paths.

Normal approximation for functions of hidden Markov models

The generalized perturbative approach is an all purpose variant of Stein's method used to obtain rates of normal approximation. Originally developed for functions of independent random variables this method is here extended to functions of the realization of a hidden Markov model. In this dependent setting, rates of convergence are provided in some applications to stochastic geometry, leading, in each instance, to an extra log-factor vis a vis the rate in the independent case.

Markov Chain Sampling for Non-linear State Space Models Using Embedded Hidden Markov Models

I describe a new Markov chain method for sampling from the distribution of the state sequences in a non-linear state space model, given the observation sequence. This method updates all states in the sequence simultaneously using an embedded Hidden Markov model (HMM). An update begins with the creation of a ``pool'' of K states at each time, by applying some Markov chain update to the current state. These pools define an embedded HMM whose states are indexes within this pool. Using the forward-backward dynamic programming algorithm, we can then efficiently choose a state sequence at random wit…

Forgetting of the initial distribution for non-ergodic Hidden Markov Chains

In this paper, the forgetting of the initial distribution for a non-ergodic Hidden Markov Models (HMM) is studied. A new set of conditions is proposed to establish the forgetting property of the filter, which significantly extends all the existing results. Both a pathwise-type convergence of the total variation distance of the filter started from two different initial distributions, and a convergence in expectation are considered. The results are illustrated using generic models of non-ergodic HMM and extend all the results known so far.

Nonlinear Filters for Hidden Markov Models of Regime Change with Fast Mean-Reverting States

We consider filtering for a hidden Markov model that evolves with multiple time scales in the hidden states. In particular, we consider the case where one of the states is a scaled Ornstein-Uhlenbeck process with fast reversion to a shifting-mean that is controlled by a continuous time Markov chain modeling regime change. We show that the nonlinear filter for such a process can be approximated by an averaged filter that asymptotically coincides with the true nonlinear filter of the regime-changing Markov chain as the rate of mean reversion approaches infinity. The asymptotics exploit weak conv…

Almost Sure Stabilization for Adaptive Controls of Regime-switching LQ Systems with A Hidden Markov Chain

This work is devoted to the almost sure stabilization of adaptive control systems that involve an unknown Markov chain. The control system displays continuous dynamics represented by differential equations and discrete events given by a hidden Markov chain. Different from previous work on stabilization of adaptive controlled systems with a hidden Markov chain, where average criteria were considered, this work focuses on the almost sure stabilization or sample path stabilization of the underlying processes. Under simple conditions, it is shown that as long as the feedback controls have linear g…

Measure Concentration of Hidden Markov Processes

We prove what appears to be the first concentration of measure result for hidden Markov processes. Our bound is stated in terms of the contraction coefficients of the underlying Markov process, and strictly generalizes the Markov process concentration results of Marton (1996) and Samson (2000). Somewhat surprisingly, the bound turns out to be the same as for ordinary Markov processes; this property, however, fails for general hidden/observed process pairs.

Stability of the optimal filter in a hidden Markov model with multiplicative noise

We consider a hidden Markov model with multiplicative noise emerging from studies of software reliability. We show the stability of the optimal filter with respect to general initial conditions in the total variation- and $L^p$-norm and deduce explicit rates. Remarkably, stability turns out to be independent of the ergodic behavior of the signal.

Entropy Rate for Hidden Markov Chains with rare transitions

We consider Hidden Markov Chains obtained by passing a Markov Chain with rare transitions through a noisy memoryless channel. We obtain asymptotic estimates for the entropy of the resulting Hidden Markov Chain as the transition rate is reduced to zero.

Run-and-tumble motion: the role of reversibility

We study a model of active particles that perform a simple random walk and on top of that have a preferred direction determined by an internal state which is modelled by a stationary Markov process. First we calculate the limiting diffusion coefficient. Then we show that the `active part' of the diffusion coefficient is in some sense maximal for reversible state processes. Further, we obtain a large deviations principle for the active particle in terms of the large deviations rate function of the empirical process corresponding to the state process. Again we show that the rate function and fre…

Hidden Markov processes in the context of symbolic dynamics

In an effort to aid communication among different fields and perhaps facilitate progress on problems common to all of them, this article discusses hidden Markov processes from several viewpoints, especially that of symbolic dynamics, where they are known as sofic measures, or continuous shift-commuting images of Markov measures. It provides background, describes known tools and methods, surveys some of the literature, and proposes several open problems.

Analyticity of Entropy Rate of Continuous-State Hidden Markov Chains

We prove that under certain mild assumptions, the entropy rate of a hidden Markov chain, observed when passing a finite-state stationary Markov chain through a discrete-time continuous-output channel, is jointly analytic as a function of the input Markov chain parameters and the channel parameters. In particular, as consequences of the main theorems, we obtain analyticity for the entropy rate associated with representative channels: Cauchy and Gaussian.

Uncertainty and filtering of hidden Markov models in discrete time

We consider the problem of filtering an unseen Markov chain from noisy observations, in the presence of uncertainty regarding the parameters of the processes involved. Using the theory of nonlinear expectations, we describe the uncertainty in terms of a penalty function, which can be propagated forward in time in the place of the filter.

On the Sequential Probability Ratio Test in Hidden Markov Models

We consider the Sequential Probability Ratio Test applied to Hidden Markov Models. Given two Hidden Markov Models and a sequence of observations generated by one of them, the Sequential Probability Ratio Test attempts to decide which model produced the sequence. We show relationships between the execution time of such an algorithm and Lyapunov exponents of random matrix systems. Further, we give complexity results about the execution time taken by the Sequential Probability Ratio Test.

On estimation states of hidden markov models in condition of unknown transition matrix

In this paper, we develop methods of nonlinear filtering and prediction of an unobservable Markov chain with a finite set of states. This Markov chain controls coefficients of AR(p) model. Using observations generated by AR(p) model we have to estimate the state of Markov chain in the case of an unknown probability transition matrix. Comparison of proposed non-parametric algorithms with the optimal methods in the case of the known transition matrix is carried out by simulating.

以二叉树为索引的隐马尔可夫模型最大似然估计量的渐近性质

Asymptotic properties of the maximum likelihood estimator for Hidden Markov Models indexed by binary trees

本文研究以二叉树为索引、隐状态为一般度量空间的隐马尔可夫模型，在平稳与非平稳情形下证明最大似然估计量的强相合性与渐近正态性，并借助树索引马尔可夫链的遍历定理完成证明。

Analyticity of Entropy Rate of Hidden Markov Chains

We prove that under mild positivity assumptions the entropy rate of a hidden Markov chain varies analytically as a function of the underlying Markov chain parameters. A general principle to determine the domain of analyticity is stated. An example is given to estimate the radius of convergence for the entropy rate. We then show that the positivity assumptions can be relaxed, and examples are given for the relaxed conditions. We study a special class of hidden Markov chains in more detail: binary hidden Markov chains with an unambiguous symbol, and we give necessary and sufficient conditions fo…

隐马尔可夫过程的Ziv-Merhav估计

Ziv-Merhav estimation for hidden-Markov processes

本文证明了隐马尔可夫过程对交叉熵率的Ziv-Merhav型估计量具有强一致性，其证明创新性地利用了定律解耦性质和热力学形式化工具。

Uniformization stable Markov models and their Jordan algebraic structure

We provide a characterisation of the continuous-time Markov models where the Markov matrices from the model can be parameterised directly in terms of the associated rate matrices (generators). That is, each Markov matrix can be expressed as the sum of the identity matrix and a rate matrix from the model. We show that the existence of an underlying Jordan algebra provides a sufficient condition, which becomes necessary for (so-called) linear models. We connect this property to the well-known uniformization procedure for continuous-time Markov chains by demonstrating that the property is equival…

Peskun-Tierney ordering for Markov chain and process Monte Carlo: beyond the reversible scenario

Historically time-reversibility of the transitions or processes underpinning Markov chain Monte Carlo methods (MCMC) has played a key rôle in their development, while the self-adjointness of associated operators together with the use of classical functional analysis techniques on Hilbert spaces have led to powerful and practically successful tools to characterize and compare their performance. Similar results for algorithms relying on nonreversible Markov processes are scarce. We show that for a type of nonreversible Monte Carlo Markov chains and processes, of current or renewed interest in th…

Exponential Bounds for Convergence of Entropy Rate Approximations in Hidden Markov Models Satisfying a Path-Mergeability Condition

A hidden Markov model (HMM) is said to have path-mergeable states if for any two states i,j there exists a word w and state k such that it is possible to transition from both i and j to k while emitting w. We show that for a finite HMM with path-mergeable states the block estimates of the entropy rate converge exponentially fast. We also show that the path-mergeability property is asymptotically typical in the space of HMM topolgies and easily testable.

Learning nonsingular phylogenies and hidden Markov models

In this paper we study the problem of learning phylogenies and hidden Markov models. We call a Markov model nonsingular if all transition matrices have determinants bounded away from 0 (and 1). We highlight the role of the nonsingularity condition for the learning problem. Learning hidden Markov models without the nonsingularity condition is at least as hard as learning parity with noise, a well-known learning problem conjectured to be computationally hard. On the other hand, we give a polynomial-time algorithm for learning nonsingular phylogenies and hidden Markov models.

Adjusted Viterbi training for hidden Markov models

To estimate the emission parameters in hidden Markov models one commonly uses the EM algorithm or its variation. Our primary motivation, however, is the Philips speech recognition system wherein the EM algorithm is replaced by the Viterbi training algorithm. Viterbi training is faster and computationally less involved than EM, but it is also biased and need not even be consistent. We propose an alternative to the Viterbi training -- adjusted Viterbi training -- that has the same order of computational complexity as Viterbi training but gives more accurate estimators. Elsewhere, we studied the …