Ad Ridder
The correspondence between the cross-entropy method and the zero-variance approximation to simulate a rare event problem in Markov chains is shown. This leads to a sufficient condition that the cross-entropy estimator is asymptotically optimal.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
继续探索
This paper is devoted to studying the average optimality in continuous-time Markov decision processes with fairly general state and action spaces. The criterion to be maximized is expected average rewards. The transition rates of underlying continuous-time jump Markov processes are allowed to be unbounded, and the reward rates may have neither upper nor lower bounds. We first provide two optimality inequalities with opposed directions, and also give suitable conditions under which the existence of solutions to the two optimality inequalities is ensured. Then, from the two optimality inequaliti…
This paper deals with the long run average continuous control problem of piecewise deterministic Markov processes (PDMP's) taking values in a general Borel space and with compact action space depending on the state variable. The control variable acts on the jump rate and transition measure of the PDMP, and the running and boundary costs are assumed to be positive but not necessarily bounded. Our first main result is to obtain an optimality equation for the long run average cost in terms of a discrete-time optimality equation related to the embedded Markov chain given by the post-jump location …
In this paper we investigate the existence of $L^{2}(π)$-spectral gaps for $π$-irreducible, positive recurrent Markov chains on general state space. We obtain necessary and sufficient conditions for the existence of $L^{2}(π)$-spectral gaps in terms of a sequence of isoperimetric constants and establish their asymptotic behavior. It turns out that in some cases the spectral gap can be understood in terms of convergence of an induced probability flow to the uniform flow. The obtained theorems can be interpreted as mixing results and yield sharp estimates for the spectral gap of some Markov chai…
Let $(ξ_n)_{n=0}^\infty$ be a nonhomogeneous Markov chain taking values from finite state-space of $\mathbf{X}=\{1,2,\ldots,b\}$. In this paper, we will study the generalized entropy ergodic theorem with almost-everywhere and $\mathcal{L}_1$ convergence for nonhomogeneous Markov chains, which generalizes the corresponding classical results for the Markov chains.
This paper studies the bailout optimal dividend problem with regime switching under the constraint that dividend payments can be made only at the arrival times of an independent Poisson process while capital can be injected continuously in time. We show the optimality of the regime-modulated Parisian-classical reflection strategy when the underlying risk model follows a general spectrally negative Markov additive process. In order to verify the optimality, first we study an auxiliary problem driven by a single spectrally negative \lev process with a final payoff at an exponential terminal time…
We consider Hidden Markov Chains obtained by passing a Markov Chain with rare transitions through a noisy memoryless channel. We obtain asymptotic estimates for the entropy of the resulting Hidden Markov Chain as the transition rate is reduced to zero.
We prove that under certain mild assumptions, the entropy rate of a hidden Markov chain, observed when passing a finite-state stationary Markov chain through a discrete-time continuous-output channel, is jointly analytic as a function of the input Markov chain parameters and the channel parameters. In particular, as consequences of the main theorems, we obtain analyticity for the entropy rate associated with representative channels: Cauchy and Gaussian.
We study ergodic properties of nonlinear Markov chains and stochastic McKean-Vlasov equations. For nonlinear Markov chains we obtain sufficient conditions for existence and uniqueness of an invariant measure and uniform ergodicity. We also prove optimality of these conditions. For stochastic McKean-Vlasov equations we establish exponential convergence of their solutions to stationarity in the total variation metric under Veretennikov-Khasminskii-type conditions.
In this paper we calculate Kolmogorov-Sinai entropy $h_M(S)$ of the generalized Markov shift associated with a contractive Markov system (CMS) \cite{Wer1} using the coding map constructed in \cite{Wer3}. We show that \[h_M(S)=-\sum\limits_{e\in E}\int\limits_{K_{i(e)}} p_e\log p_edμ\] where $μ$ is a unique invariant Borel probability measure of the CMS. I. Werner, Contractive Markov systems, J. London Math. Soc. (2005) 236-258. I. Werner, Coding map for a contractive Markov system, Math. Proc. Camb. Phil. Soc. to appear 140 (2), March 2006.
We study the existence, optimality, and construction of non-randomised stopping times that solve the Skorokhod embedding problem (SEP) for Markov processes which satisfy a duality assumption. These stopping times are hitting times of space-time subsets, so-called Root barriers. Our main result is, besides the existence and optimality, a potential-theoretic characterisation of this Root barrier as a free boundary. If the generator of the Markov process is sufficiently regular, this reduces to an obstacle PDE that has the Root barrier as free boundary and thereby generalises previous results fro…
Motivated by entropic optimal transport, time reversal of Markov jump processes in $\mathbb{R}^n$ is investigated. Relying on an abstract integration by parts formula for the carré du champ of a Markov process recently obtained by Cattiaux, Gentil and the authors, and using an entropic improvement strategy discovered by Föllmer in the eighties, we compute the semimartingale characteristics of the time reversed process for a wide class of jump processes with possibly unbounded variation sample paths and singular intensities of jump.
We consider a strictly substochastic matrix or an stochastic matrix with absorbing states. By using quasi-stationary distributions one shows there is a canonical associated stationary Markov chain. Based upon $2-$stringing representation of the resurrected chain we supply a stationary representation of the killed and the absorbed chains. The entropies of these representations has a clear meaning when one identifies the probability measure of natural factors. The balance between the entropies of these representations and the canonical chain, serves to check the suitability of the whole construc…
This paper is devoted to studying constrained continuous-time Markov decision processes (MDPs) in the class of randomized policies depending on state histories. The transition rates may be unbounded, the reward and costs are admitted to be unbounded from above and from below, and the state and action spaces are Polish spaces. The optimality criterion to be maximized is the expected discounted rewards, and the constraints can be imposed on the expected discounted costs. First, we give conditions for the nonexplosion of underlying processes and the finiteness of the expected discounted rewards/c…
We present some new results on sample path optimality for the ergodic control problem of a class of non-degenerate diffusions controlled through the drift. The hypothesis most often used in the literature to ensure the existence of an a.s. sample path optimal stationary Markov control requires finite second moments of the first hitting times $τ$ of bounded domains over all admissible controls. We show that this can be considerably weakened: ${\mathrm E}[τ^2]$ may be replaced with ${\mathrm E}[τ\ln^+(τ)]$, thus reducing the required rate of convergence of averages from polynomial to logarithmic…
We show that if the random walk on a graph has positive coarse Ricci curvature in the sense of Ollivier, then the stationary measure satisfies a W^1 transport-entropy inequality. Peres and Tetali have conjectured a stronger consequence, that a modified log-Sobolev inequality (MLSI) should hold, in analogy with the setting of Markov diffusions. We discuss how our entropy interpolation approach suggests a natural attack on the MLSI conjecture.
In the absence of acceleration, the velocity formula gives "distance travelled equals speed multiplied by time". For a broad class of Markov chains such as circulant Markov chains or random walk on complete graphs, we prove a probabilistic analogue of the velocity formula between entropy and hitting time, where distance is the entropy of the Markov trajectories from state $i$ to state $j$ in the sense of [L. Ekroot and T. M. Cover. The entropy of Markov trajectories. IEEE Trans. Inform. Theory 39(4): 1418-1421.], speed is the classical entropy rate of the chain, and the time variable is the ex…
An invariant state of a quantum Markov semigroup is an equilibrium state if it satisfies a quantum detailed balance condition. In this paper, we introduce a notion of entropy production for faithful normal invariant states of a quantum Markov semigroup on B(h) as a numerical index measuring "how much far" they are from equilibrium. The entropy production is defined as the derivative of the relative entropy of the one-step forward and backward evolution in analogy with the classical probabilistic concept. We prove an explicit trace formula expressing the entropy production in terms of the compl…
In this work we study the Hausdorff dimension of measures whose weight distribution satisfies a markov non-homogeneous property. We prove, in particular, that the Hausdorff dimensions of this kind of measures coincide with their lower Rényi dimensions (entropy). Moreover, we show that the Tricot dimensions (packing dimension) equal the upper Rényi dimensions. As an application we get a continuity property of the Hausdorff dimension of the measures, when it is seen as a function of the distributed weights under the $\ell^{\infty}$ norm.
A lumping of a Markov chain is a coordinate-wise projection of the chain. We characterise the entropy rate preservation of a lumping of an aperiodic and irreducible Markov chain on a finite state space by the random growth rate of the cardinality of the realisable preimage of a finite-length trajectory of the lumped chain and by the information needed to reconstruct original trajectories from their lumped images. Both are purely combinatorial criteria, depending only on the transition graph of the Markov chain and the lumping function. A lumping is strongly k-lumpable, iff the lumped process i…
For discrete-time stochastic processes, there is a close connection between return/waiting times and entropy. Such a connection cannot be straightforwardly extended to the continuous-time setting. Contrarily to the discrete-time case one does need a reference measure and so the natural object is relative entropy rather than entropy. In this paper we elaborate on this in the case of continuous-time Markov processes with finite state space. A reference measure of special interest is the one associated to the time-reversed process. In that case relative entropy is interpreted as the entropy produ…
Stochastic control with dividend payments and capital injections for Markov additive processes
针对马尔可夫可加过程，研究允许任意时刻资本注入和离散或连续分红的最优控制问题，提出马尔可夫调制的周期-经典障碍策略并证明其最优性。
Blackwell optimality in risk-sensitive stochastic control
本文研究了风险敏感随机控制中Blackwell最优性的概念，探讨了其与终极平稳性的关系，并通过示例说明了两者之间的结构性差异。
In the paper, we introduce the maximum entropy estimator based on 2-dimensional empirical distribution of the observation sequence of hidden Markov model , when the sample size is big: in that case computing the maximum likelihood estimator is too consuming in time by the classical Baum-Welch EM algorithm. We prove the consistency and the asymptotic normality of the maximum entropy estimator in a quite general framework, where the asymptotic covariance matrix is explicitly estimated in terms of the 2-dimensional Fisher information. To complement it, the 2-dimensional relative entropy is skillf…
A language L over a finite alphabet is growth-sensitive (or entropy sensitive) if forbidding any set of subwords F yields a sub-language L^F whose exponential growth rate (entropy) is smaller than that of L. Let (X, E, l) be an infinite, oriented, labelled graph. Considering the graph as an (infinite) automaton, we associate with any pair of vertices x,y in X the language consisting of all words that can be read as the labels along some path from x to y. Under suitable, general assumptions we prove that these languages are growth-sensitive. This is based on using Markov chains with forbidden t…
This paper deals with discrete-time Markov control processes on a general state space. A long-run risk-sensitive average cost criterion is used as a performance measure. The one-step cost function is nonnegative and possibly unbounded. Using the vanishing discount factor approach, the optimality inequality and an optimal stationary strategy for the decision maker are established.
A hidden Markov model (HMM) is said to have path-mergeable states if for any two states i,j there exists a word w and state k such that it is possible to transition from both i and j to k while emitting w. We show that for a finite HMM with path-mergeable states the block estimates of the entropy rate converge exponentially fast. We also show that the path-mergeability property is asymptotically typical in the space of HMM topolgies and easily testable.
We prove that under mild positivity assumptions the entropy rate of a hidden Markov chain varies analytically as a function of the underlying Markov chain parameters. A general principle to determine the domain of analyticity is stated. An example is given to estimate the radius of convergence for the entropy rate. We then show that the positivity assumptions can be relaxed, and examples are given for the relaxed conditions. We study a special class of hidden Markov chains in more detail: binary hidden Markov chains with an unambiguous symbol, and we give necessary and sufficient conditions fo…
For any stationary $\mZ^d$-Gibbs measure that satisfies strong spatial mixing, we obtain sequences of upper and lower approximations that converge to its entropy. In the case, $d=2$, these approximations are efficient in the sense that the approximations are accurate to within $ε$ and can be computed in time polynomial in $1/ε$.
Consider a hidden Markov chain obtained as the observation process of an ordinary Markov chain corrupted by noise. Zuk, et. al. [13], [14] showed how, in principle, one can explicitly compute the derivatives of the entropy rate of at extreme values of the noise. Namely, they showed that the derivatives of standard upper approximations to the entropy rate actually stabilize at an explicit finite time. We generalize this result to a natural class of hidden Markov chains called ``Black Holes.'' We also discuss in depth special cases of binary Markov chains observed in binary symmetric noise, and …
Large deviations for quadratic functionals of stable Gauss-Markov chains and entropy production
该研究通过发展拟 Toeplitz 二次泛函的大偏差理论，证明了非平稳中心化稳定 Gauss-Markov 链熵产生率的大偏差原理及 Gallavotti-Cohen 对称性。
In this paper we consider the convergence of the conditional entropy to the entropy rate for Markov chains. Convergence of certain statistics of long range dependent processes, such as the sample mean, is slow. It has been shown in Carpio and Daley \cite{carpio2007long} that the convergence of the $n$-step transition probabilities to the stationary distribution is slow, without quantifying the convergence rate. We prove that the slow convergence also applies to convergence to an information-theoretic measure, the entropy rate, by showing that the convergence rate is equivalent to the convergen…
We study a new notion of Ricci curvature that applies to Markov chains on discrete spaces. This notion relies on geodesic convexity of the entropy and is analogous to the one introduced by Lott, Sturm, and Villani for geodesic measure spaces. In order to apply to the discrete setting, the role of the Wasserstein metric is taken over by a different metric, having the property that continuous time Markov chains are gradient flows of the entropy. Using this notion of Ricci curvature we prove discrete analogues of fundamental results by Bakry--Emery and Otto--Villani. Furthermore we show that Ricc…
Let K be an irreducible and reversible Markov kernel on a finite set X. We construct a metric W on the set of probability measures on X and show that with respect to this metric, the law of the continuous time Markov chain evolves as the gradient flow of the entropy. This result is a discrete counterpart of the Wasserstein gradient flow interpretation of the heat flow in R^n by Jordan, Kinderlehrer, and Otto (1998). The metric W is similar to, but different from, the L^2-Wasserstein metric, and is defined via a discrete variant of the Benamou-Brenier formula.
In the setting of reversible continuous-time Markov chains, the $CD_Υ$ condition has been shown recently to be a consistent analogue to the Bakry-Émery condition in the diffusive setting in terms of proving Li-Yau inequalities under a finite dimension term and proving the modified logarithmic Sobolev inequality under a positive curvature bound. In this article we examine the case where both is given, a finite dimension term and a positive curvature bound. For this purpose we introduce the $CD_Υ(κ,F)$ condition, where the dimension term is expressed by a so called $CD$-function $F$. We derive f…
Markov Categories and Entropy
本研究将马尔可夫范畴框架与信息论定量概念结合，通过增广马尔可夫范畴刻画信息论定量特性，基于范畴的确定性概念定义熵，可复原多种经典熵并给出广义熵的概念性定义。
扫码添加微信好友，提出您的宝贵建议 👇
💡 备注请填写：网站反馈
发表机构由 AI 从论文文本抽取，如发现错误，欢迎反馈修正建议。