arXiv math/05012992005-01-19math.STstat.TH

On Unified Generalizations of Relative Jensen--Shannon and Arithmetic--Geometric Divergence Measures, and Their Properties

Pranesh Kumar, Inder Jeet Taneja

英文摘要：

In this paper we shall consider one parametric generalization of some non-symmetric divergence measures. The \textit{non-symmetric divergence measures} are such as: Kullback-Leibler \textit{relative information}, $χ^2-$\textit{divergence}, \textit{relative J -- divergence}, \textit{relative Jensen -- Shannon divergence} and \textit{relative Arithmetic -- Geometric divergence}. All the generalizations considered can be written as particular cases of Csiszár's \textit{f-divergence}. By putting some conditions on the probability distribution, the aim here is to develop bounds on these measures and their parametric generalizations.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

On a generalization of the Jensen-Shannon divergence

The Jensen-Shannon divergence is a renown bounded symmetrization of the Kullback-Leibler divergence which does not require probability densities to have matching supports. In this paper, we introduce a vector-skew generalization of the scalar $α$-Jensen-Bregman divergences and derive thereof the vector-skew $α$-Jensen-Shannon divergences. We study the properties of these novel divergences and show how to build parametric families of symmetric Jensen-Shannon-type divergences. Finally, we report an iterative algorithm to numerically compute the Jensen-Shannon-type centroids for a set of probabil…

2019-12-19cs.ITmath.ITmath.ST

Nonextensive Generalizations of the Jensen-Shannon Divergence

Convexity is a key concept in information theory, namely via the many implications of Jensen's inequality, such as the non-negativity of the Kullback-Leibler divergence (KLD). Jensen's inequality also underlies the concept of Jensen-Shannon divergence (JSD), which is a symmetrized and smoothed version of the KLD. This paper introduces new JSD-type divergences, by extending its two building blocks: convexity and Shannon's entropy. In particular, a new concept of q-convexity is introduced and shown to satisfy a Jensen's q-inequality. Based on this Jensen's q-inequality, the Jensen-Tsallis q-diff…

2008-04-10cs.ITmath.ITmath.ST

Jensen-Shannon Information Based Characterization of the Generalization Error of Learning Algorithms

Generalization error bounds are critical to understanding the performance of machine learning models. In this work, we propose a new information-theoretic based generalization error upper bound applicable to supervised learning scenarios. We show that our general bound can specialize in various previous bounds. We also show that our general bound can be specialized under some conditions to a new bound involving the Jensen-Shannon information between a random variable modelling the set of training samples and another random variable modelling the hypothesis. We also prove that our bound can be …

2021-01-08cs.ITmath.ITmath.ST

Extrinsic Jensen-Shannon Divergence: Applications to Variable-Length Coding

This paper considers the problem of variable-length coding over a discrete memoryless channel (DMC) with noiseless feedback. The paper provides a stochastic control view of the problem whose solution is analyzed via a newly proposed symmetrized divergence, termed extrinsic Jensen-Shannon (EJS) divergence. It is shown that strictly positive lower bounds on EJS divergence provide non-asymptotic upper bounds on the expected code length. The paper presents strictly positive lower bounds on EJS divergence, and hence non-asymptotic upper bounds on the expected code length, for the following two codi…

2015-01-05cs.ITmath.ITmath.OC

On a Difference of Jensen Inequality and its Applications to Mean Divergence Measures

In this paper we have considered a difference of Jensen's inequality for convex functions and proved some of its properties. In particular, we have obtained results for Csiszár \cite{csi1} $f-$divergence. A result is established that allow us to compare two measures under certain conditions. By the application of this result we have obtained a new inequality for the well known means such as arithmetic, geometric and harmonic. Some divergence measures based on these means are also defined.

2005-01-19math.STstat.TH

Generalized Arithmetic and Geometric Mean Divergence Measure and their Statistical Aspects

Using Blackwell's definition of comparing two experiments, a comparison is made with \textit{generalized AG - divergence} measure having one and two scalar parameters. Connection of \textit{generalized AG - divergence} measure with \textit{Fisher measure of information} is also presented. A unified \textit{generalization of AG - divergence}and\textit{Jensen-Shannon divergence measures} is also presented.

2005-01-19math.PRmath.STstat.TH

更多相关论文收起更多论文

Generalized Non-Symmetric Divergence Measures and Inequalities

In this paper we consider one parameter generalizations of some non - symmetric divergence measures. Measures are \textit{relative information}, $χ^2 - $\textit{divergence}, \textit{relative J-divergence}, \textit{relative Jensen-Shannon divergence}and \textit{relative arithmetic and geometric divergence}. All the generalizations considered can be written as particular cases of Csiszár \textit{f-divergence}. By conditioning the probability distributions, relationships among the \textit{relative divergence measures}are obtained.

2005-01-19math.STstat.TH

On Mean Divergence Measures

Arithmetic, geometric and harmonic means are the three classical means famous in the literature. Another mean such as square-root mean is also known. In this paper, we have constructed divergence measures based on nonnegative differences among these means, and established an interesting inequality by use of properties of Csiszar's f-divergence. Connections of new mean divergences measures with classical divergence measures such as Jeffreys-Kullback-Leibler J-divergence, Sibson-Burbea-Rao Jensen difference divergence measure and Taneja's arithmetic - geometric mean divergence are also establish…

2005-06-13math.STstat.TH

Inequalities Among Symmetric Divergence Measures and Their Refinement

There are three classical divergence measures in the literature on information theory and statistics, namely, Jeffryes-Kullback-Leiber's J-divergence, Sibson-Burbea-Rao's Jensen-Shannon divegernce and Taneja's arithemtic-geometric mean divergence. These bear an interesting relationship among each other and are based on logarithmic expressions. The divergence measures like Hellinger discrimination, symmetric chi-square divergence, and triangular discrimination are not based on logarithmic expressions. These six divergence measures are symmetric with respect to probability distributions. In this…

2005-04-04math.STstat.TH