Stability of the optimal filter in a hidden Markov model with multiplicative noise

连续流驱动的 Markov 过程与乘性 $L^p$-半群

Continuous flows driving Markov processes and multiplicative $L^p$-semigroups

本文提出通过连续流驱动 Markov 过程的方法，研究生成元可交换时一阶算子的添加，证明流的连续性并展示 Lp 空间上的 Markov 乘性半群由连续流生成，同时将弱生成元与鞅问题扩展至无界函数。

Convergence in distribution for filtering processes associated to Hidden Markov Models with densities

Consider a filtering process associated to a hidden Markov model with densities for which both the state space and the observation space are complete, separable, metric spaces. If the underlying, hidden Markov chain is strongly ergodic and the filtering process fulfills a certain coupling condition we prove that, in the limit, the distribution of the filtering process is independent of the initial distribution of the hidden Markov chain. If furthermore the hidden Markov chain is uniformly ergodic, then we prove that the filtering process converges in distribution.

On the reversibility of the observed process of three-state hidden Markov model

For the continuous-time and the discrete-time three-state hidden Markov model, the flux of the likelihood function up to 3-dimension of the observed process is shown explicitly. As an application, the sufficient and necessary condition of the reversibility of the observed process is shown.

On hidden Markov chains and finite stochastic systems

In this paper we study various properties of finite stochastic systems or hidden Markov chains as they are alternatively called. We discuss their construction following different approaches and we also derive recursive filtering formulas for the different systems that we consider. The key tool is a simple lemma on conditional expectations.

加性增加与乘性减少的马尔可夫调制过程的退出问题

Exit problems for additive-increase and multiplicative-decrease Markov-modulated processes

研究有限状态马尔可夫调制加性增加乘性减少过程的一侧、两侧及反射退出问题，通过首次跳跃分解、拉普拉斯变换和递归方法推导退出矩阵，并构造相关杜布鞅。

Measure Concentration of Hidden Markov Processes

We prove what appears to be the first concentration of measure result for hidden Markov processes. Our bound is stated in terms of the contraction coefficients of the underlying Markov process, and strictly generalizes the Markov process concentration results of Marton (1996) and Samson (2000). Somewhat surprisingly, the bound turns out to be the same as for ordinary Markov processes; this property, however, fails for general hidden/observed process pairs.

Entropy Rate for Hidden Markov Chains with rare transitions

We consider Hidden Markov Chains obtained by passing a Markov Chain with rare transitions through a noisy memoryless channel. We obtain asymptotic estimates for the entropy of the resulting Hidden Markov Chain as the transition rate is reduced to zero.

Robust Filtering and Propagation of Uncertainty in Hidden Markov Models

We consider the filtering of continuous-time finite-state hidden Markov models, where the rate and observation matrices depend on unknown time-dependent parameters, for which no prior or stochastic model is available. We quantify and analyze how the induced uncertainty may be propagated through time as we collect new observations, and used to simultaneously provide robust estimates of the hidden signal and to learn the unknown parameters, via techniques based on pathwise filtering and new results on the optimal control of rough differential equations.

Large deviations asymptotics and the spectral theory of multiplicatively regular Markov processes

We continue the investigation of the spectral theory and exponential asymptotics of Markov processes, following Kontoyiannis and Meyn (2003). We introduce a new family of nonlinear Lyapunov drift criteria, characterizing distinct subclasses of geometrically ergodic Markov processes in terms of inequalities for the nonlinear generator. We concentrate on the class of "multiplicatively regular" Markov processes, characterized via conditions similar to (but weaker than) those of Donsker-Varadhan. For any such process {Phi(t)} with transition kernel P on a general state space, the following are obt…

Markov 与隐 Markov 过程研究中的散度度量

Divergence metrics in the study of Markov and hidden Markov processes

本文综述并统一连续时间 Markov 与隐 Markov 过程中的 f-散度，展示其在随机稳定性、热力学、滤波器稳定性及 Maxwell 妖模型中的应用。

Area anomaly in the rough path Brownian scaling limit of hidden Markov walks

We study the convergence in rough path topology of a certain class of discrete processes, the hidden Markov walks, to a Brownian motion with an area anomaly. This area anomaly, which is a new object, keeps track of the time-correlation of the discrete models and brings into light the question of embeddings of discrete processes into continuous time. We also identify an underlying combinatorial structure in the hidden Markov walks, which turns out to be a generalization of the occupation time from the classical ergodic theorem in the spirit of rough paths.

Normal approximation for functions of hidden Markov models

The generalized perturbative approach is an all purpose variant of Stein's method used to obtain rates of normal approximation. Originally developed for functions of independent random variables this method is here extended to functions of the realization of a hidden Markov model. In this dependent setting, rates of convergence are provided in some applications to stochastic geometry, leading, in each instance, to an extra log-factor vis a vis the rate in the independent case.

Markov Chain Sampling for Non-linear State Space Models Using Embedded Hidden Markov Models

I describe a new Markov chain method for sampling from the distribution of the state sequences in a non-linear state space model, given the observation sequence. This method updates all states in the sequence simultaneously using an embedded Hidden Markov model (HMM). An update begins with the creation of a ``pool'' of K states at each time, by applying some Markov chain update to the current state. These pools define an embedded HMM whose states are indexes within this pool. Using the forward-backward dynamic programming algorithm, we can then efficiently choose a state sequence at random wit…

Forgetting of the initial distribution for non-ergodic Hidden Markov Chains

In this paper, the forgetting of the initial distribution for a non-ergodic Hidden Markov Models (HMM) is studied. A new set of conditions is proposed to establish the forgetting property of the filter, which significantly extends all the existing results. Both a pathwise-type convergence of the total variation distance of the filter started from two different initial distributions, and a convergence in expectation are considered. The results are illustrated using generic models of non-ergodic HMM and extend all the results known so far.

Nonlinear Filters for Hidden Markov Models of Regime Change with Fast Mean-Reverting States

We consider filtering for a hidden Markov model that evolves with multiple time scales in the hidden states. In particular, we consider the case where one of the states is a scaled Ornstein-Uhlenbeck process with fast reversion to a shifting-mean that is controlled by a continuous time Markov chain modeling regime change. We show that the nonlinear filter for such a process can be approximated by an averaged filter that asymptotically coincides with the true nonlinear filter of the regime-changing Markov chain as the rate of mean reversion approaches infinity. The asymptotics exploit weak conv…

Almost Sure Stabilization for Adaptive Controls of Regime-switching LQ Systems with A Hidden Markov Chain

This work is devoted to the almost sure stabilization of adaptive control systems that involve an unknown Markov chain. The control system displays continuous dynamics represented by differential equations and discrete events given by a hidden Markov chain. Different from previous work on stabilization of adaptive controlled systems with a hidden Markov chain, where average criteria were considered, this work focuses on the almost sure stabilization or sample path stabilization of the underlying processes. Under simple conditions, it is shown that as long as the feedback controls have linear g…

Approximating a Diffusion by a Hidden Markov Model

For a wide class of continuous-time Markov processes, including all irreducible hypoelliptic diffusions evolving on an open, connected subset of $\RL^d$, the following are shown to be equivalent: (i) The process satisfies (a slightly weaker version of) the classical Donsker-Varadhan conditions; (ii) The transition semigroup of the process can be approximated by a finite-state hidden Markov model, in a strong sense in terms of an associated operator norm; (iii) The resolvent kernel of the process is `$v$-separable', that is, it can be approximated arbitrarily well in operator norm by finite-ran…

On the rate of convergence for the length of the longest common subsequences in hidden Markov models

Let $(X, Y) = (X_n, Y_n)_{n \geq 1}$ be the output process generated by a hidden chain $Z = (Z_n)_{n \geq 1}$, where $Z$ is a finite state, aperiodic, time homogeneous, and irreducible Markov chain. Let $LC_n$ be the length of the longest common subsequences of $X_1, \ldots, X_n$ and $Y_1, \ldots, Y_n$. Under a mixing hypothesis, a rate of convergence result is obtained for $\mathbb{E}[LC_n]/n$.

Analyticity of Entropy Rate of Continuous-State Hidden Markov Chains

We prove that under certain mild assumptions, the entropy rate of a hidden Markov chain, observed when passing a finite-state stationary Markov chain through a discrete-time continuous-output channel, is jointly analytic as a function of the input Markov chain parameters and the channel parameters. In particular, as consequences of the main theorems, we obtain analyticity for the entropy rate associated with representative channels: Cauchy and Gaussian.

On the Sequential Probability Ratio Test in Hidden Markov Models

We consider the Sequential Probability Ratio Test applied to Hidden Markov Models. Given two Hidden Markov Models and a sequence of observations generated by one of them, the Sequential Probability Ratio Test attempts to decide which model produced the sequence. We show relationships between the execution time of such an algorithm and Lyapunov exponents of random matrix systems. Further, we give complexity results about the execution time taken by the Sequential Probability Ratio Test.

Uncertainty and filtering of hidden Markov models in discrete time

We consider the problem of filtering an unseen Markov chain from noisy observations, in the presence of uncertainty regarding the parameters of the processes involved. Using the theory of nonlinear expectations, we describe the uncertainty in terms of a penalty function, which can be propagated forward in time in the place of the filter.

On estimation states of hidden markov models in condition of unknown transition matrix

In this paper, we develop methods of nonlinear filtering and prediction of an unobservable Markov chain with a finite set of states. This Markov chain controls coefficients of AR(p) model. Using observations generated by AR(p) model we have to estimate the state of Markov chain in the case of an unknown probability transition matrix. Comparison of proposed non-parametric algorithms with the optimal methods in the case of the known transition matrix is carried out by simulating.

以二叉树为索引的隐马尔可夫模型最大似然估计量的渐近性质

Asymptotic properties of the maximum likelihood estimator for Hidden Markov Models indexed by binary trees

本文研究以二叉树为索引、隐状态为一般度量空间的隐马尔可夫模型，在平稳与非平稳情形下证明最大似然估计量的强相合性与渐近正态性，并借助树索引马尔可夫链的遍历定理完成证明。

Learning nonsingular phylogenies and hidden Markov models

In this paper we study the problem of learning phylogenies and hidden Markov models. We call a Markov model nonsingular if all transition matrices have determinants bounded away from 0 (and 1). We highlight the role of the nonsingularity condition for the learning problem. Learning hidden Markov models without the nonsingularity condition is at least as hard as learning parity with noise, a well-known learning problem conjectured to be computationally hard. On the other hand, we give a polynomial-time algorithm for learning nonsingular phylogenies and hidden Markov models.

Analyticity of Entropy Rate of Hidden Markov Chains

We prove that under mild positivity assumptions the entropy rate of a hidden Markov chain varies analytically as a function of the underlying Markov chain parameters. A general principle to determine the domain of analyticity is stated. An example is given to estimate the radius of convergence for the entropy rate. We then show that the positivity assumptions can be relaxed, and examples are given for the relaxed conditions. We study a special class of hidden Markov chains in more detail: binary hidden Markov chains with an unambiguous symbol, and we give necessary and sufficient conditions fo…

隐马尔可夫过程的Ziv-Merhav估计

Ziv-Merhav estimation for hidden-Markov processes

本文证明了隐马尔可夫过程对交叉熵率的Ziv-Merhav型估计量具有强一致性，其证明创新性地利用了定律解耦性质和热力学形式化工具。

Exponential Bounds for Convergence of Entropy Rate Approximations in Hidden Markov Models Satisfying a Path-Mergeability Condition

A hidden Markov model (HMM) is said to have path-mergeable states if for any two states i,j there exists a word w and state k such that it is possible to transition from both i and j to k while emitting w. We show that for a finite HMM with path-mergeable states the block estimates of the entropy rate converge exponentially fast. We also show that the path-mergeability property is asymptotically typical in the space of HMM topolgies and easily testable.

Adjusted Viterbi training for hidden Markov models

To estimate the emission parameters in hidden Markov models one commonly uses the EM algorithm or its variation. Our primary motivation, however, is the Philips speech recognition system wherein the EM algorithm is replaced by the Viterbi training algorithm. Viterbi training is faster and computationally less involved than EM, but it is also biased and need not even be consistent. We propose an alternative to the Viterbi training -- adjusted Viterbi training -- that has the same order of computational complexity as Viterbi training but gives more accurate estimators. Elsewhere, we studied the …