Luca Tamanini
Aim of this short note is to study Shannon's entropy power along entropic interpolations, thus generalizing Costa's concavity theorem. We shall provide two proofs of independent interest: the former by $Γ$-calculus, hence applicable to more abstract frameworks; the latter with an explicit remainder term, reminiscent of [20], allowing us to characterize the case of equality.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
继续探索
Costa's entropy power inequality is an important generalization of Shannon's entropy power inequality. Related with Costa's entropy power inequality and a conjecture proposed by McKean in 1966, Cheng-Geng recently conjectured that $D(m,n): (-1)^{m+1}(\partial^m/\partial^m t)H(X_t)\ge0$, where $X_t$ is the $n$-dimensional random variable in Costa's entropy power inequality and $H(X_t)$ the differential entropy of $X_t$. $D(1,n)$ and $D(2,n)$ were proved by Costa as consequences of Costa's entropy power inequality. Cheng-Geng proved $D(3,1)$ and $D(4,1)$. In this paper, we propose a systematical…
This paper considers an entropy-power inequality (EPI) of Costa and presents a natural vector generalization with a real positive semidefinite matrix parameter. This new inequality is proved using a perturbation approach via a fundamental relationship between the derivative of mutual information and the minimum mean-square error (MMSE) estimate in linear vector Gaussian channels. As an application, a new extremal entropy inequality is derived from the generalized Costa EPI and then used to establish the secrecy capacity regions of the degraded vector Gaussian broadcast channel with layered con…
We give a counterexample to the vector generalization of Costa's entropy power inequality (EPI) due to Liu, Liu, Poor and Shamai. In particular, the claimed inequality can fail if the matix-valued parameter in the convex combination does not commute with the covariance of the additive Gaussian noise. Conversely, the inequality holds if these two matrices commute.
A simple multivariate version of Costa's entropy power inequality is proved. In particular, it is shown that if independent white Gaussian noise is added to an arbitrary multivariate signal, the entropy power of the resulting random variable is a multidimensional concave function of the individual variances of the components of the signal. As a side result, we also give an expression for the Hessian matrix of the entropy and entropy power functions with respect to the variances of the signal components, which is an interesting result in its own right.
Let $X$ be an arbitrary continuous random variable and $Z$ be an independent Gaussian random variable with zero mean and unit variance. For $t~>~0$, Costa proved that $e^{2h(X+\sqrt{t}Z)}$ is concave in $t$, where the proof hinged on the first and second order derivatives of $h(X+\sqrt{t}Z)$. Specifically, these two derivatives are signed, i.e., $\frac{\partial}{\partial t}h(X+\sqrt{t}Z) \geq 0$ and $\frac{\partial^2}{\partial t^2}h(X+\sqrt{t}Z) \leq 0$. In this paper, we show that the third order derivative of $h(X+\sqrt{t}Z)$ is nonnegative, which implies that the Fisher information $J(X+\sq…
The Marton-Gelfand-Pinsker inner bound on the capacity region of broadcast channels was extended by Han-Costa to include arbitrarily correlated sources where the capacity region is replaced by an admissible source region. The main arguments of Han-Costa are correct but unfortunately the authors overlooked an inequality in their derivation. The corrected region is presented and the absence of the omitted inequality is shown to sometimes admit sources that are not admissible.
Several conjectures concern the lower bound for the differential entropy $H(X_t)$ of an $n$-dimensional random vector $X_t$ introduced by Costa. Cheng and Geng conjectured that $H(X_t)$ is completely monotone, that is, $C_1(m,n): (-1)^{m+1}(d^m/d^m t)H(X_t)\ge0$. McKean conjectured that Gaussian $X_{Gt}$ achieves the minimum of $(-1)^{m+1}(d^m/d^m t)H(X_t)$ under certain conditions, that is, $C_2(m,n): (-1)^{m+1}(d^m/d^m t)H(X_t)\ge(-1)^{m+1}(d^m/d^m t)H(X_{Gt})$. McKean's conjecture was only considered in the univariate case before: $C_2(1,1)$ and $C_2(2,1)$ were proved by McKean and $C_2(i,1…
Elaborating on the similarity between the entropy power inequality and the Brunn-Minkowski inequality, Costa and Cover conjectured in {\it On the similarity of the entropy power inequality and the Brunn-Minkowski inequality} (IEEE Trans. Inform. Theory 30 (1984), no. 6, 837-839) the $\frac{1}{n}$-concavity of the outer parallel volume of measurable sets as an analogue of the concavity of entropy power. We investigate this conjecture and study its relationship with geometric inequalities.
We associate to the p-th Rényi entropy a definition of entropy power, which is the natural extension of Shannon's entropy power and exhibits a nice behaviour along solutions to the p-nonlinear heat equation in $R^n$. We show that the Rényi entropy power of general probability densities solving such equations is always a concave function of time, whereas it has a linear behaviour in correspondence to the Barenblatt source-type solutions. We then shown that the p-th Rényi entropy power of a probability density which solves the nonlinear diffusion of order p, is a concave function of time. This r…
We show that Shannon's entropy--power inequality admits a strengthened version in the case in which the densities are log-concave. In such a case, in fact, one can extend the Blachman--Stam argument to obtain a sharp inequality for the second derivative of Shannon's entropy functional with respect to the heat semigroup.
We tighten the Entropy Power Inequality (EPI) when one of the random summands is Gaussian. Our strengthening is closely connected to the concept of strong data processing for Gaussian channels and generalizes the (vector extension of) Costa's EPI. This leads to a new reverse entropy power inequality and, as a corollary, sharpens Stam's inequality relating entropy power and Fisher information. Applications to network information theory are given, including a short self-contained proof of the rate region for the two-encoder quadratic Gaussian source coding problem. Our argument is based on weak …
Recently, Savaré-Toscani proved that the Rényi entropy power of general probability densities solving the $p$-nonlinear heat equation in $\mathbb{R}^n$ is always a concave function of time, which extends Costa's concavity inequality for Shannon's entropy power to Rényi entropies. In this paper, we give a generalization of Savaré-Toscani's result by giving a class of sufficient conditions of the parameters under which the concavity of the Rényi entropy power is still valid. These conditions are quite general and include the parameter range given by Savaré-Toscani as special cases. Also, the con…
A lower bound on the Rényi differential entropy of a sum of independent random vectors is demonstrated in terms of rearrangements. For the special case of Boltzmann-Shannon entropy, this lower bound is better than that given by the entropy power inequality. Several applications are discussed, including a new proof of the classical entropy power inequality and an entropy inequality involving symmetrization of Lévy processes.
Within the framework of linear vector Gaussian channels with arbitrary signaling, closed-form expressions for the Jacobian of the minimum mean square error and Fisher information matrices with respect to arbitrary parameters of the system are calculated in this paper. Capitalizing on prior research where the minimum mean square error and Fisher information matrices were linked to information-theoretic quantities through differentiation, closed-form expressions for the Hessian of the mutual information and the differential entropy are derived. These expressions are then used to assess the conca…
We prove a quantitative form of the celebrated Ball's theorem on cube slicing in $\mathbb{R}^n$ and obtain, as a consequence, equality cases in the min-entropy power inequality. Independently, we also give a quantitative form of Khintchine's inequality in the special case $p=1$.
We analyse an analog of the entropy-power inequality for the weighted entropy.
Yet another simple proof of the entropy power inequality is given, which avoids both the integration over a path of Gaussian perturbation and the use of Young's inequality with sharp constant or Rényi entropies. The proof is based on a simple change of variables, is formally identical in one and several dimensions, and easily settles the equality case.
In this paper, we proved an exact asymptotically sharp upper bound of the $L^p$ Lebesgue Constant (i.e. the $L^p$ norm of Dirichlet kernel) for $p\ge 2$. As an application, we also verified the implication of a new $\infty $-Rényi entropy power inequality for integer valued random variables.
We present a simple proof of the entropy-power inequality using an optimal transportation argument which takes the form of a simple change of variables. The same argument yields a reverse inequality involving a conditional differential entropy which has its own interest. It can also be generalized in various ways. The equality case is easily captured by this method and the proof is formally identical in one and several dimensions.
The entropy power inequality (EPI) provides lower bounds on the differential entropy of the sum of two independent real-valued random variables in terms of the individual entropies. Versions of the EPI for discrete random variables have been obtained for special families of distributions with the differential entropy replaced by the discrete entropy, but no universal inequality is known (beyond trivial ones). More recently, the sumset theory for the entropy function provides a sharp inequality $H(X+X')-H(X)\geq 1/2 -o(1)$ when $X,X'$ are i.i.d. with high entropy. This paper provides the inequa…
We consider the entropy of sums of independent discrete random variables, in analogy with Shannon's Entropy Power Inequality, where equality holds for normals. In our case, infinite divisibility suggests that equality should hold for Poisson variables. We show that some natural analogues of the Entropy Power Inequality do not in fact hold, but propose an alternative formulation which does always hold. The key to many proofs of Shannon's Entropy Power Inequality is the behaviour of entropy on scaling of continuous random variables. We believe that Rényi's operation of thinning discrete random v…
This paper is twofold. In the first part, we present a refinement of the Rényi Entropy Power Inequality (EPI) recently obtained in \cite{BM16}. The proof largely follows the approach in \cite{DCT91} of employing Young's convolution inequalities with sharp constants. In the second part, we study the reversibility of the Rényi EPI, and confirm a conjecture in \cite{BNT15, MMX16} in two cases. Connections with various $p$-th mean bodies in convex geometry are also explored.
The matrix version of the entropy-power inequality for real or complex coefficients and variables is proved using a transportation argument that easily settles the equality case. An application to blind source extraction is given.
The paper establishes the equality condition in the I-MMSE proof of the entropy power inequality (EPI). This is done by establishing an exact expression for the deficit between the two sides of the EPI. Interestingly, a necessary condition for the equality is established by making a connection to the famous Cauchy functional equation.
We establish a general class of entropy inequalities that take the concise form of Gaussian comparisons. The main result unifies many classical and recent results, including the Shannon-Stam inequality, the Brunn-Minkowski inequality, the Zamir-Feder inequality, the Brascamp-Lieb and Barthe inequalities, the Anantharam-Jog-Nair inequality, and others.
Using a sharp version of the reverse Young inequality, and a Rényi entropy comparison result due to Fradelizi, Madiman, and Wang, the authors are able to derive Rényi entropy power inequalities for log-concave random vectors when Rényi parameters belong to $(0,1)$. Furthermore, the estimates are shown to be sharp up to absolute constants.
A Quantitative Entropy Power Inequality for Dependent Random Vectors
本文提出了一种针对依赖随机向量的熵功率不等式定量版本，适用于联合密度为对数超模ular的随机向量。
Let $\mathsf{N}_{\rm d}\left[X\right]=\frac{1}{2π{\rm e}}{\rm e}^{2\mathsf{H}\left[X\right]}$ denote the entropy power of the discrete random variable $X$ where $\mathsf{H}\left[X\right]$ denotes the discrete entropy of $X$. In this paper, we show that for two independent discrete random variables $X$ and $Y$, the entropy power inequality $\mathsf{N}_{\rm d}\left[X\right]+\mathsf{N}_{\rm d}\left[Y\right]\leq 2 \mathsf{N}_{\rm d}\left[X+Y\right]$ holds and it can be tight. The basic idea behind the proof is to perturb the discrete random variables using suitably designed continuous random varia…
The Discounted Least Information Theory of Entropy (DLITE) is a new information measure that quantifies the amount of entropic difference between two probability distributions. It manifests multiple critical properties both as an information-theoretic quantity and as metric distance. In the report, we provide a proof of the triangular inequality of DLITE's cube root ($\sqrt[3]{DL}$), an important property of a metric, along with alternative proofs for two additional properties.
On the problem of reversibility of the entropy power inequality
本文研究熵功率不等式的可逆性问题，证明凸概率分布类整体不满足可逆性，同时探讨了同分布加项的相关现象，完善了熵功率不等式可逆性的适用范围结论。
We provide a simple proof of a curious inequality for the binary entropy function, an inequality that has been used in two different contexts. In the 1980's, Boppana used this entropy inequality to prove lower bounds on Boolean formulas. More recently, the inequality was used to achieve major progress on Frankl's union-closed sets conjecture. Our proof of the entropy inequality uses basic differential calculus.
An extension of the entropy power inequality to the form $N_r^α(X+Y) \geq N_r^α(X) + N_r^α(Y)$ with arbitrary independent summands $X$ and $Y$ in $\mathbb{R}^n$ is obtained for the Rényi entropy and powers $α\geq (r+1)/2$.
We show that an information-theoretic property of Shannon's entropy power, known as concavity of entropy power, can be fruitfully employed to prove inequalities in sharp form. In particular, the concavity of entropy power implies the logarithmic Sobolev inequality, and Nash's inequality with the sharp constant.
A Massey-like inequality is any useful lower bound on guessing entropy in terms of the computationally scalable Shannon entropy. The asymptotically optimal Massey-like inequality is determined and further refined for finite-support distributions. The impact of these results are highlighted for side-channel attack evaluation where guessing entropy is a key metric. In this context, the obtained bounds are compared to the state of the art.
The entropy power inequality (EPI) and the Brascamp-Lieb inequality (BLI) are fundamental inequalities concerning the differential entropies of linear transformations of random vectors. The EPI provides lower bounds for the differential entropy of linear transformations of random vectors with independent components. The BLI, on the other hand, provides upper bounds on the differential entropy of a random vector in terms of the differential entropies of some of its linear transformations. In this paper, we define a family of entropy functionals, which we show are subadditive. We then establish …
It is known that the Entropy Power Inequality (EPI) always holds if the random variables have density. Not much work has been done to identify discrete distributions for which the inequality holds with the differential entropy replaced by the discrete entropy. Harremoës and Vignat showed that it holds for the pair (B(m,p), B(n,p)), m,n \in \mathbb{N}, (where B(n,p) is a Binomial distribution with n trials each with success probability p) for p = 0.5. In this paper, we considerably expand the set of Binomial distributions for which the inequality holds and, in particular, identify n_0(p) such t…
A noisy entropy inequality for boolean functions by Samorodnitsky is applied to binary codes. It is shown that a binary code that achieves capacity on the binary erasure channel admits optimal list size for list decoding on some binary symmetric channels (in a regime where this optimal list size is exponentially large).
Relative to the Gaussian measure on $\mathbb{R}^d$, entropy and Fisher information are famously related via Gross' logarithmic Sobolev inequality (LSI). These same functionals also separately satisfy convolution inequalities, as proved by Stam. We establish a dimension-free inequality that interpolates among these relations. Several interesting corollaries follow: (i) the deficit in the LSI satisfies a convolution inequality itself; (ii) the deficit in the LSI controls convergence in the entropic and Fisher information central limit theorems; and (iii) the LSI is stable with respect to HWI jum…
In this paper, we investigate the partition inequality, joint convexity, and Pinsker's inequality, for a divergence that generalizes the Tsallis Relative Entropy and Kullback-Leibler divergence. The generalized divergence is defined in terms of a deformed exponential function, which replaces the Tsallis $q$-exponential. We also constructed a family of probability distributions related to the generalized divergence. We found necessary and sufficient conditions for the partition inequality to be satisfied. A sufficient condition for the joint convexity was established. We proved that the general…
While most useful information theoretic inequalities can be deduced from the basic properties of entropy or mutual information, Shannon's entropy power inequality (EPI) seems to be an exception: available information theoretic proofs of the EPI hinge on integral representations of differential entropy using either Fisher's information (FI) or minimum mean-square error (MMSE). In this paper, we first present a unified view of proofs via FI and MMSE, showing that they are essentially dual versions of the same proof, and then fill the gap by providing a new, simple proof of the EPI, which is sole…
We establish quantitative stability results for the entropy power inequality (EPI). Specifically, we show that if uniformly log-concave densities nearly saturate the EPI, then they must be close to Gaussian densities in the quadratic Wasserstein distance. Further, if one of the densities is log-concave and the other is Gaussian, then the deficit in the EPI can be controlled in terms of the $L^1$-Wasserstein distance. As a counterpoint, an example shows that the EPI can be unstable with respect to the quadratic Wasserstein distance when densities are uniformly log-concave on sets of measure arb…
This paper concerns the folklore statement that ``entropy is a lower bound for compression''. More precisely we derive from the entropy theorem a simple proof of a pointwise inequality firstly stated by Ornstein and Shields and which is the almost-sure version of an average inequality firstly stated by Khinchin in 1953. We further give an elementary proof of original Khinchin inequality that can be used as an exercise for Information Theory students and we conclude by giving historical and technical notes of such inequality.
Determining the ultimate classical information carrying capacity of electromagnetic waves requires quantum-mechanical analysis to properly account for the bosonic nature of these waves. Recent work has established capacity theorems for bosonic single-user, broadcast, and wiretap channels, under the presumption of two minimum output entropy conjectures. Despite considerable accumulated evidence that supports the validity of these conjectures, they have yet to be proven. Here we show that the preceding minimum output entropy conjectures are simple consequences of an Entropy Photon-Number Inequal…
It is shown that given two copies of a q-ary input channel $W$, where q is prime, it is possible to create two channels $W^-$ and $W^+$ whose symmetric capacities satisfy $I(W^-)\le I(W)\le I(W^+)$, where the inequalities are strict except in trivial cases. This leads to a simple proof of channel polarization in the q-ary case.
We prove a new extremal inequality, motivated by the vector Gaussian broadcast channel and the distributed source coding with a single quadratic distortion constraint problems. As a corollary, this inequality yields a generalization of the classical entropy-power inequality (EPI). As another corollary, this inequality sheds insight into maximizing the differential entropy of the sum of two dependent random variables.
While most useful information theoretic inequalities can be deduced from the basic properties of entropy or mutual information, up to now Shannon's entropy power inequality (EPI) is an exception: Existing information theoretic proofs of the EPI hinge on representations of differential entropy using either Fisher information or minimum mean-square error (MMSE), which are derived from de Bruijn's identity. In this paper, we first present an unified view of these proofs, showing that they share two essential ingredients: 1) a data processing argument applied to a covariance-preserving linear tran…
Motivated by the entropy computations relevant to the evaluation of decrease in entropy in bit reset operations, the authors investigate the deficit in an entropic inequality involving two independent random variables, one continuous and the other discrete. In the case where the continuous random variable is Gaussian, we derive strong quantitative bounds on the deficit in the inequality. More explicitly it is shown that the decay of the deficit is sub-Gaussian with respect to the reciprocal of the standard deviation of the Gaussian variable. What is more, up to rational terms these results are…
Strengthening Han's Fourier Entropy-Influence Inequality via an Information-Theoretic Proof
通过信息论证明，该研究加强了韩的傅里叶熵-影响不等式，证明其对于所有实值布尔函数具有精确常数1。
We prove stability estimates for the Shannon-Stam inequality (also known as the entropy-power inequality) for log-concave random vectors in terms of entropy and transportation distance. In particular, we give the first stability estimate for general log-concave random vectors in the following form: for log-concave random vectors $X,Y \in \mathbb{R}^d$, the deficit in the Shannon-Stam inequality is bounded from below by the expression $$ C \left(\mathrm{D}\left(X||G\right) + \mathrm{D}\left(Y||G\right)\right), $$ where $\mathrm{D}\left( \cdot ~ ||G\right)$ denotes the relative entropy with resp…
The conditional entropy power inequality is a fundamental inequality in information theory, stating that the conditional entropy of the sum of two conditionally independent vector-valued random variables each with an assigned conditional entropy is minimum when the random variables are Gaussian. We prove the conditional entropy power inequality in the scenario where the conditioning system is quantum. The proof is based on the heat semigroup and on a generalization of the Stam inequality in the presence of quantum conditioning. The entropy power inequality with quantum conditioning will be a k…
Determining the ultimate classical information carrying capacity of electromagnetic waves requires quantum-mechanical analysis to properly account for the bosonic nature of these waves. Recent work has established capacity theorems for bosonic single-user and broadcast channels, under the presumption of two minimum output entropy conjectures. Despite considerable accumulated evidence that supports the validity of these conjectures, they have yet to be proven. In this paper, it is shown that the second conjecture suffices to prove the classical capacity of the bosonic wiretap channel, which in …
Generalized Talagrand Inequality for Sinkhorn Distance using Entropy Power Inequality
本文利用最优传输的无穷小位移凸性和熵功率不等式，证明了HWI型不等式并导出两个Talagrand型不等式，将Sinkhorn距离下的高斯Talagrand不等式推广至强对数凹情形。
In this paper we study the capacity/entropy region of finite, directed, acyclic, multiple-sources, multiple-sinks network by means of group theory and entropy vectors coming from groups. There is a one-to-one correspondence between the entropy vector of a collection of n random variables and a certain group-characterizable vector obtained from a finite group and n of its subgroups. We are looking at nilpotent group characterizable entropy vectors and show that they are all also Abelian group characterizable, and hence they satisfy the Ingleton inequality. It is known that not all entropic vect…
We show that the Entropy Photon-Number Inequality (EPnI) holds where one of the input states is the vacuum state and for several candidates of the other input state that includes the cases when the state has the eigenvectors as the number states and either has only two non-zero eigenvalues or has arbitrary number of non-zero eigenvalues but is a high entropy state. We also discuss the conditions, which if satisfied, would lead to an extension of these results.
We prove the quantum conditional Entropy Power Inequality for quantum additive noise channels. This inequality lower bounds the quantum conditional entropy of the output of an additive noise channel in terms of the quantum conditional entropies of the input state and the noise when they are conditionally independent given the memory. We also show that this conditional Entropy Power Inequality is optimal in the sense that we can achieve equality asymptotically by choosing a suitable sequence of Gaussian input states. We apply the conditional Entropy Power Inequality to find an array of informat…
This paper first focuses on deriving an alternative approach for proving an extremal entropy inequality (EEI), originally presented in [11]. The proposed approach does not rely on the channel enhancement technique, and has the advantage that it yields an explicit description of the optimal solution as opposed to the implicit approach of [11]. Compared with the proofs in [11], the proposed alternative proof is also simpler, more direct, more information-theoretic, and has the additional advantage that it offers a new perspective for establishing novel as well as known challenging results such t…
New sharp inequalities involving non-relative, relative and cross informational functionals with some remarkable minimizers of generalized Gaussian and Beta types
该研究通过特定不等式推导出几个新的尖锐信息不等式，包括连接雷尼熵幂等的类似斯坦姆不等式等，这些不等式均尖锐，类似斯坦姆不等式极小值有时是高斯等概率密度对，类矩不等式极小值是广义贝塔分布概率密度。
We prove the conditional Entropy Power Inequality for Gaussian quantum systems. This fundamental inequality determines the minimum quantum conditional von Neumann entropy of the output of the beam-splitter or of the squeezing among all the input states where the two inputs are conditionally independent given the memory and have given quantum conditional entropies. We also prove that, for any couple of values of the quantum conditional entropies of the two inputs, the minimum of the quantum conditional entropy of the output given by the conditional Entropy Power Inequality is asymptotically ach…
Concavity of Tsallis Entropy and Tsallis Entropy Power along Heat Flow
研究热流下Tsallis熵的演化，证明在任意维度中Tsallis熵的凹性，并推广到Tsallis熵幂，包括Shannon熵幂和Costa EPI作为端点。
We establish a lower bound on the entropy of weighted sums of (possibly dependent) random variables $(X_1, X_2, \dots, X_n)$ possessing a symmetric joint distribution. Our lower bound is in terms of the joint entropy of $(X_1, X_2, \dots, X_n)$. We show that for $n \geq 3$, the lower bound is tight if and only if $X_i$'s are i.i.d.\ Gaussian random variables. For $n=2$ there are numerous other cases of equality apart from i.i.d.\ Gaussians, which we completely characterize. Going beyond sums, we also present an inequality for certain linear transformations of $(X_1, \dots, X_n)$. Our primary t…
扫码添加微信好友，提出您的宝贵建议 👇
💡 备注请填写：网站反馈
发表机构由 AI 从论文文本抽取，如发现错误，欢迎反馈修正建议。