Lower tails via relative entropy

A trajectorial approach to relative entropy dissipation of McKean$\boldsymbol{-}$Vlasov diffusions: gradient flows and HWBI inequalities

We formulate a trajectorial version of the relative entropy dissipation identity for McKean$-$Vlasov diffusions, extending the results of the papers [FJ16,KST20a], which apply to non-interacting diffusions. Our stochastic analysis approach is based on time-reversal of diffusions and Lions' differential calculus over Wasserstein space. It allows us to compute explicitly the rate of relative entropy dissipation along every trajectory of the underlying diffusion via the semimartingale decomposition of the corresponding relative entropy process. As a first application, we obtain a new interpretati…

On the form of the relative entropy between measures on the space of continuous functions

In this paper we derive an integral (with respect to time) representation of the relative entropy (or Kullback-Leibler Divergence) between measures mu and P on the space of continuous functions from time 0 to T. The underlying measure P is a weak solution to a Martingale Problem with continuous coefficients. Since the relative entropy governs the exponential rate of convergence of the empirical measure (according to Sanov's Theorem), this representation is of use in the numerical and analytical investigation of finite-size effects in systems of interacting diffusions.

Relative and Discrete Utility Maximising Entropy

The notion of utility maximising entropy (u-entropy) of a probability density, which was introduced and studied by Slomczynski and Zastawniak (Ann. Prob 32 (2004) 2261-2285, arXiv:math.PR/0410115 v1), is extended in two directions. First, the relative u-entropy of two probability measures in arbitrary probability spaces is defined. Then, specialising to discrete probability spaces, we also introduce the absolute u-entropy of a probability measure. Both notions are based on the idea, borrowed from mathematical finance, of maximising the expected utility of the terminal wealth of an investor. Mo…

Relative entropy and waiting times for continuous-time Markov processes

For discrete-time stochastic processes, there is a close connection between return/waiting times and entropy. Such a connection cannot be straightforwardly extended to the continuous-time setting. Contrarily to the discrete-time case one does need a reference measure and so the natural object is relative entropy rather than entropy. In this paper we elaborate on this in the case of continuous-time Markov processes with finite state space. A reference measure of special interest is the one associated to the time-reversed process. In that case relative entropy is interpreted as the entropy produ…

Relative entropy, Gaussian concentration and uniqueness of equilibrium states

For a general class of lattice-spin systems, we prove that an abstract Gaussian concentration bound implies positivity of lower relative entropy density. As a consequence we obtain uniqueness of translation-invariant Gibbs measures from the Gaussian concentration bound in this general setting. This extends earlier results from [Chazottes-Moles-REdig-Ugalde] with a different and very short proof.

Limiting behavior of relative Rényi entropy in a non-regular location shift family

We calculate the limiting behavior of relative Renyi entropy when the first probability distribution is close to the second one in a non-regular location-shift family which is generated by a probability distribution whose support is an interval or a half-line. This limit can be regarded as a generalization of Fisher information, and plays an important role in large deviation theory.

An Ergodic Theorem for the Quantum Relative Entropy

We prove the ergodic version of the quantum Stein's lemma which was conjectured by Hiai and Petz. The result provides an operational and statistical interpretation of the quantum relative entropy as a statistical measure of distinguishability, and contains as a special case the quantum version of the Shannon-McMillan theorem for ergodic states. A version of the quantum relative Asymptotic Equipartition Property (AEP) is given.

有放回与无放回采样中的相对熵界

Relative entropy bounds for sampling with and without replacement

本文提出了一种基于相对熵的有放回与无放回采样界，结合斯塔姆的结果和相对熵的凸性，得到新的有限德芬尼界，实现最优渐近收敛速率。

The Quantum Relative Entropy as a Rate Function and Information Criteria

We prove that the quantum relative entropy is a rate function in large deviation principle. Next, we define information criteria for quantum states and estimate the accuracy of the use of them. Most of the results in this paper are essentially based on Hiai-Ohya-Tsukada theorem.

Minimum relative entropy distributions with a large mean are Gaussian

We consider the following frustrated optimization problem: given a prior probability distribution $q$, find the distribution $p$ minimizing the relative entropy with respect to $q$ such that $\textrm{mean}(p)$ is fixed and large. We show that solutions to this problem are asymptotically Gaussian. As an application we derive an $H$-type theorem for evolutionary dynamics: the entropy of the (standardized) distribution of fitness of a population evolving under natural selection is eventually increasing.