In this document we are interested in entropy. Entropy is multiple, the idea is to describe the definition proposed by the physicist Clausius. Indeed, Clausius exposes in 1865 the second principle of thermodynamics and also proposes the concept of entropy. Instead of simply defining a functional, central point for the development of the second principle, it will in fact define a concept sufficiently general to be used in many fields of mathematics.In these few pages, I wish to show the role played by entropy in the field of gradient flows and functional inequalities. Starting from the definition of Clausius in 1865, I will try to explain how fundamental functional inequalities like Sobolev inequality, key point in analysis, are natural inked with a particular entropy. This path allows me to give an overview of the use of gradient flows in finite or infinite dimension, of Bakry-Emery theory and more recently of Otto's calculus.
We provide a stochastic mathematical representation for Clausius' and Kelvin-Planck's statements of the Second Law of Thermodynamics in terms of the entropy productions of a finite, compact driven Markov system and its lift. A surjective map is rigorously established through the lift when the state space is either a discrete graph or a continuous n-dimensional torus T^n. The corresponding lifted processes have detailed balance thus a natural potential function but no stationary probability. We show that in the long-time limit the entropy production of the finite driven system precisely equals …
In this work, we study the validity of Entropy inequalities under decorrelation assumptions at large-scales, and also we study the problem of Weighted Meyers estimate in perforated domains. As a consequence, we obtain the hypercontractivity of correctors with Neumann boundary conditions.
We present new $Φ$-entropy inequalities for diffusion semigroups under the curvature-dimension criterion. They include the isoperimetric function of the Gaussian measure. Applications to the long time behaviour of solutions to Fokker-Planck equations are given.
We obtain and study new $Φ$-entropy inequalities for diffusion semigroups, with Poincaré or logarithmic Sobolev inequalities as particular cases. From this study we derive the asymptotic behaviour of a large class of linear Fokker-Plank type equations under simple conditions, widely extending previous results. Nonlinear diffusion equations are also studied by means of these inequalities. The $Γ_2$ criterion of D. Bakry and M. Emery appears as a main tool in the analysis, in local or integral forms.
We give a characterization of transport-entropy inequalities in metric spaces. As an application we deduce that such inequalities are stable under bounded perturbation (Holley-Stroock perturbation Lemma).
We give a necessary and sufficient condition for transport-entropy inequalities in dimension one. As an application, we construct a new example of a probability distribution verifying Talagrand's T2 inequality and not the logarithmic Sobolev inequality.
We prove a stochastic formula for the Gaussian relative entropy in the spirit of Borell's formula for the Laplace transform. As an application, we give unified and short proofs of a number of functional inequalities.
In this paper, we use the semi-group method and an adaptation of the $L^2-$method of Hörmander to establish some $Φ-$entropy inequalities and asymmetric covariance estimates for the strictly convex measures in $\mathbb R^n$. These inequalities extends the ones for the strictly log-concave measures to more general setting of convex measures. The $Φ-$entropy inequalities are turned out to be sharp in the special case of Cauchy measures. Finally, we show that the similar inequalities for log-concave measures can be obtained from our results in the limiting case.
In this paper the author analyses the weighted Renyi entropy in order to derive several inequalities in weighted case. Furthermore, using the proposed notions $α$-th generalized derivation and ($α$; p)-th weighted Fisher information, extended versions of the moment-entropy, Fisher information and Cramer-Rao inequalities in terms of generalized Gaussian densities are given.
This paper represents an extended version of an earlier note [10]. The concept of weighted entropy takes into account values of different outcomes, i.e., makes entropy context-dependent, through the weight function. We analyse analogs of the Fisher information inequality and entropy power inequality for the weighted entropy and discuss connections with weighted Lieb's splitting inequality. The concepts of rates of the weighted entropy and information are also discussed.
We investigate the role of convexity in Rényi entropy power inequalities. After proving that a general Rényi entropy power inequality in the style of Bobkov-Chistyakov (2015) fails when the Rényi parameter $r\in(0,1)$, we show that random vectors with $s$-concave densities do satisfy such a Rényi entropy power inequality. Along the way, we establish the convergence in the Central Limit Theorem for Rényi entropies of order $r\in(0,1)$ for log-concave densities and for compactly supported, spherically symmetric and unimodal densities, complementing a celebrated result of Barron (1986). Additiona…
The goal of this note is to show that some convolution type inequalities from Harmonic Analysis and Information Theory, such as Young's convolution inequality (with sharp constant), Nelson's hypercontractivity of the Hermite semi-group or Shannon's inequality, can be reduced to a simple geometric study of frames of $\R^2$. We shall derive directly entropic inequalities, which were recently proved to be dual to the Brascamp-Lieb convolution type inequalities.
We prove an Hopf-Lax-Oleinik formula for the solutions of some Hamilton- Jacobi equations on a general metric space. As a first consequence, we show in full gener- ality that the log-Sobolev inequality is equivalent to an hypercontractivity property of the Hamilton-Jacobi semi-group. As a second consequence, we prove that Talagrand's transport- entropy inequalities in metric space are characterized in terms of log-Sobolev inequalities restricted to the class of c-convex functions.
We show that Talagrand's transport inequality is equivalent to a restricted logarithmic Sobolev inequality. This result clarifies the links between these two important functional inequalities. As an application, we give the first proof of the fact that Talagrand's inequality is stable under bounded perturbations.
We introduce the notion of an interpolating path on the set of probability measures on finite graphs. Using this notion, we first prove a displacement convexity property of entropy along such a path and derive Prekopa-Leindler type inequalities, a Talagrand transport-entropy inequality, certain HWI type as well as log-Sobolev type inequalities in discrete settings. To illustrate through examples, we apply our results to the complete graph and to the hypercube for which our results are optimal -- by passing to the limit, we recover the classical log-Sobolev inequality for the standard Gaussian …
We study an optimal weak transport cost related to the notion of convex order between probability measures. On the real line, we show that this weak transport cost is reached for a coupling that does not depend on the underlying cost function. As an application, we give a necessary and sufficient condition for weak transport-entropy inequalities in dimension one. In particular, we obtain a weak transport-entropy form of the convex Poincar{é} inequality in dimension one.
This paper is concerned with certain invariant random processes (called factors of IID) on infinite trees. Given such a process, one can assign entropies to different finite subgraphs of the tree. There are linear inequalities between these entropies that hold for any factor of IID process (e.g. "edge versus vertex" or "star versus edge"). These inequalities turned out to be very useful: they have several applications already, the most recent one is the Backhausz-Szegedy result on the eigenvectors of random regular graphs. We present new entropy inequalities in this paper. In fact, our approac…
We derive weighted log-Sobolev inequalities from a class of super Poincaré inequalities. As an application, the Talagrand inequality with larger distances are obtained. In particular, on a complete connected Riemannian manifold, we prove that the $\log^\dd$-Sobolev inequality with $\dd\in (1,2)$ implies the $L^{2/(2-\dd)}$-transportation cost inequality $$W^\rr_{2/(2-\dd)}(fμ,μ)^{2/(2-\dd)}\le Cμ(f\log f), μ(f)=1, f\ge 0$$ for some constant $C>0$, and they are equivalent if the curvature of the corresponding generator is bounded below. Weighted log-Sobolev and entropy-cost inequalities are als…
Talagrand's inequalities make a link between two fundamentals concepts of probability: transport of measures and entropy. The study of the counterpart of these inequalities in the context of free probability has been initiated by Biane and Voiculescu and later extended by Hiai, Petz and Ueda for convex potentials. In this work, we prove a free analogue of a result of Bobkov and Götze in the classical setting, thus providing free transport-entropy inequalities for a very natural class of measures appearing in random matrix theory. These inequalities are weaker than the ones of Hiai, Petz and Ue…
Following Talagrand's concentration results for permutations picked uniformly at random from a symmetric group [Tal95], Luczak and McDiarmid have generalized it to more general groups G of permutations which act suitably 'locally'. Here we extend their results by setting transport-entropy inequalities on these permutations groups. Talagrand and Luczak-Mc-Diarmid concentra- tion properties are consequences of these inequalities. The results are also gen- eralised to a larger class of measures including Ewens distributions of arbitrary parameter $θ$ on the symmetric group. By projection, we deri…
We prove a sharp analog of Young's inequality on $S^N$, and deduce from it certain sharp entropy inequalities. The proof turns on constructing a nonlinear heat flow that drives trial functions to optimizers in a monotonic manner. This strategy also works for the generalization of Young's inequality on $R^N$ to more than three functions, and leads to significant new information about the optimizers and the constants.
Various properties of isoperimetric, functional, Transport-Entropy and concentration inequalities are studied on a Riemannian manifold equipped with a measure, whose generalized Ricci curvature is bounded from below. First, stability of these inequalities with respect to perturbation of the measure is obtained. The extent of the perturbation is measured using several different distances between perturbed and original measure, such as a one-sided $L^\infty$ bound on the ratio between their densities, Wasserstein distances, and Kullback--Leibler divergence. In particular, an extension of the Hol…
In the setting of reversible continuous-time Markov chains, the $CD_Υ$ condition has been shown recently to be a consistent analogue to the Bakry-Émery condition in the diffusive setting in terms of proving Li-Yau inequalities under a finite dimension term and proving the modified logarithmic Sobolev inequality under a positive curvature bound. In this article we examine the case where both is given, a finite dimension term and a positive curvature bound. For this purpose we introduce the $CD_Υ(κ,F)$ condition, where the dimension term is expressed by a so called $CD$-function $F$. We derive f…
We establish a discrete analog of the Rényi entropy comparison due to Bobkov and Madiman. For log-concave variables on the integers, the min entropy is within log e of the usual Shannon entropy. Additionally we investigate the entropic Rogers-Shephard inequality studied by Madiman and Kontoyannis, and establish a sharp Rényi version for certain parameters in both the continuous and discrete cases
This note contributes to the understanding of generalized entropy power inequalities. Our main goal is to construct a counter-example regarding monotonicity and entropy comparison of weighted sums of independent identically distributed log-concave random variables. We also present a complex analogue of a recent dependent entropy power inequality of Hao and Jog, and give a very simple proof.
We show that if the random walk on a graph has positive coarse Ricci curvature in the sense of Ollivier, then the stationary measure satisfies a W^1 transport-entropy inequality. Peres and Tetali have conjectured a stronger consequence, that a modified log-Sobolev inequality (MLSI) should hold, in analogy with the setting of Markov diffusions. We discuss how our entropy interpolation approach suggests a natural attack on the MLSI conjecture.
We develop a reverse entropy power inequality for convex measures, which may be seen as an affine-geometric inverse of the entropy power inequality of Shannon and Stam. The specialization of this inequality to log-concave measures may be seen as a version of Milman's reverse Brunn-Minkowski inequality. The proof relies on a demonstration of new relationships between the entropy of high dimensional random vectors and the volume of convex bodies, and on a study of effective supports of convex measures, both of which are of independent interest, as well as on Milman's deep technology of $M$-ellip…
The notion of tree entropy was introduced by the author as a normalized limit of the number of spanning trees in finite graphs, but is defined on random infinite rooted graphs. We give some new expressions for tree entropy; one uses Fuglede-Kadison determinants, while another uses effective resistance. We use the latter to prove that tree entropy respects stochastic domination. We also prove that tree entropy is non-negative in the unweighted case, a special case of which establishes Lueck's Determinant Conjecture for Cayley-graph Laplacians. We use techniques from the theory of operators affi…
The entropy power inequality, which plays a fundamental role in information theory and probability, may be seen as an analogue of the Brunn-Minkowski inequality. Motivated by this connection to Convex Geometry, we survey various recent developments on forward and reverse entropy power inequalities not just for the Shannon-Boltzmann entropy but also more generally for Rényi entropy. In the process, we discuss connections between the so-called functional (or integral) and probabilistic (or entropic) analogues of some classical inequalities in geometric functional analysis
Lower bounds for the Rényi entropies of sums of independent random variables taking values in cyclic groups of prime order under permutations are established. The main ingredients of our approach are extended rearrangement inequalities in prime cyclic groups building on Lev (2001), and notions of stochastic ordering. Several applications are developed, including to discrete entropy power inequalities, the Littlewood-Offord problem, and counting solutions of certain linear systems.
A natural link between the notions of majorization and strongly Sperner posets is elucidated. It is then used to obtain a variety of consequences, including new Rényi entropy inequalities for sums of independent, integer-valued random variables.
In this paper we have shall generalize Shearer's entropy inequality and its recent extensions by Madiman and Tetali, and shall apply projection inequalities to deduce extensions of some of the inequalities concerning sums of sets of integers proved recently by Gyarmati, Matolcsi and Ruzsa. We shall also discuss projection and entropy inequalities and their connections.
In a recent paper by A. Backhausz, B. Gerencsér and V. Harangi, it was shown that factors of independent identically distributed random processes (IID) on trees obey certain geometry-driven inequalities. In particular, the mutual information shared between two vertices decays exponentially, and there is an explicit bound for this decay. In this note we show that all of these inequalities could be verbatim translated to the setting of factors of processes driven by unique Gibbs measures. As a consequence, we show that correlations decay exponentially for unique Gibbs measures on trees.
A new notion of partition-determined functions is introduced, and several basic inequalities are developed for the entropy of such functions of independent random variables, as well as for cardinalities of compound sets obtained using these functions. Here a compound set means a set obtained by varying each argument of a function of several variables over a set associated with that argument, where all the sets are subsets of an appropriate algebraic structure so that the function is well defined. On the one hand, the entropy inequalities developed for partition-determined functions imply entro…
