On a Difference of Jensen Inequality and its Applications to Mean Divergence Measures

概率分布的否定与Jensen不等式之间的相互作用

Interplay between Negation of a Probability Distribution and Jensen Inequality

本文基于Yager提出的概率否定变换，利用Jensen不等式给出否定后不确定性上界，并通过数值例子验证，进而构造概率分布与其否定之间的不相似度函数。

Spatial medians, depth functions and multivariate Jensen's inequality

For any given partial order in a $d$-dimensional euclidean space, under mild regularity assumptions, we show that the intersection of closed (generalized) intervals containing more than 1/2 of the probability mass, is a non-empty compact interval. This property is shared with common intervals on real line, where the intersection is the median set of the underlying probability distribution. So obtained multivariate medians with respect to a partial order, can be observed as special cases of centers of distribution in the sense of type D depth functions introduced by Y. Zuo and R. Serfling, {\em…

Sum decomposition of divergence into three divergences

Divergence functions play a key role as to measure the discrepancy between two points in the field of machine learning, statistics and signal processing. Well-known divergences are the Bregman divergences, the Jensen divergences and the f-divergences. In this paper, we show that the symmetric Bregman divergence can be decomposed into the sum of two types of Jensen divergences and the Bregman divergence. Furthermore, applying this result, we show another sum decomposition of divergence is possible which includes f-divergences explicitly.

Extrinsic Jensen-Shannon Divergence: Applications to Variable-Length Coding

This paper considers the problem of variable-length coding over a discrete memoryless channel (DMC) with noiseless feedback. The paper provides a stochastic control view of the problem whose solution is analyzed via a newly proposed symmetrized divergence, termed extrinsic Jensen-Shannon (EJS) divergence. It is shown that strictly positive lower bounds on EJS divergence provide non-asymptotic upper bounds on the expected code length. The paper presents strictly positive lower bounds on EJS divergence, and hence non-asymptotic upper bounds on the expected code length, for the following two codi…

Generalized Non-Symmetric Divergence Measures and Inequalities

In this paper we consider one parameter generalizations of some non - symmetric divergence measures. Measures are \textit{relative information}, $χ^2 - $\textit{divergence}, \textit{relative J-divergence}, \textit{relative Jensen-Shannon divergence}and \textit{relative arithmetic and geometric divergence}. All the generalizations considered can be written as particular cases of Csiszár \textit{f-divergence}. By conditioning the probability distributions, relationships among the \textit{relative divergence measures}are obtained.

On Mean Divergence Measures

Arithmetic, geometric and harmonic means are the three classical means famous in the literature. Another mean such as square-root mean is also known. In this paper, we have constructed divergence measures based on nonnegative differences among these means, and established an interesting inequality by use of properties of Csiszar's f-divergence. Connections of new mean divergences measures with classical divergence measures such as Jeffreys-Kullback-Leibler J-divergence, Sibson-Burbea-Rao Jensen difference divergence measure and Taneja's arithmetic - geometric mean divergence are also establish…

Inequalities Among Symmetric Divergence Measures and Their Refinement

There are three classical divergence measures in the literature on information theory and statistics, namely, Jeffryes-Kullback-Leiber's J-divergence, Sibson-Burbea-Rao's Jensen-Shannon divegernce and Taneja's arithemtic-geometric mean divergence. These bear an interesting relationship among each other and are based on logarithmic expressions. The divergence measures like Hellinger discrimination, symmetric chi-square divergence, and triangular discrimination are not based on logarithmic expressions. These six divergence measures are symmetric with respect to probability distributions. In this…

Generalized Arithmetic and Geometric Mean Divergence Measure and their Statistical Aspects

Using Blackwell's definition of comparing two experiments, a comparison is made with \textit{generalized AG - divergence} measure having one and two scalar parameters. Connection of \textit{generalized AG - divergence} measure with \textit{Fisher measure of information} is also presented. A unified \textit{generalization of AG - divergence}and\textit{Jensen-Shannon divergence measures} is also presented.

图凸包界作为广义 Jensen 不等式

Graph Convex Hull Bounds as generalized Jensen Inequalities

本文通过研究函数图像的凸包，为任意域上的任意函数给出期望的精确上下界，将 Jensen 不等式推广到有限/无限维及条件期望、Markov 算子情形。

Generalized Exponential Concentration Inequality for Rényi Divergence Estimation

Estimating divergences in a consistent way is of great importance in many machine learning tasks. Although this is a fundamental problem in nonparametric statistics, to the best of our knowledge there has been no finite sample exponential inequality convergence bound derived for any divergence estimators. The main contribution of our work is to provide such a bound for an estimator of Rényi-$α$ divergence for a smooth Hölder class of densities on the $d$-dimensional unit cube $[0, 1]^d$. We also illustrate our theoretical results with a numerical experiment.