Arithmetic, geometric and harmonic means are the three classical means famous in the literature. Another mean such as square-root mean is also known. In this paper, we have constructed divergence measures based on nonnegative differences among these means, and established an interesting inequality by use of properties of Csiszar's f-divergence. Connections of new mean divergences measures with classical divergence measures such as Jeffreys-Kullback-Leibler J-divergence, Sibson-Burbea-Rao Jensen difference divergence measure and Taneja's arithmetic - geometric mean divergence are also established.
In this paper we have considered a difference of Jensen's inequality for convex functions and proved some of its properties. In particular, we have obtained results for Csiszár \cite{csi1} $f-$divergence. A result is established that allow us to compare two measures under certain conditions. By the application of this result we have obtained a new inequality for the well known means such as arithmetic, geometric and harmonic. Some divergence measures based on these means are also defined.
Using Blackwell's definition of comparing two experiments, a comparison is made with \textit{generalized AG - divergence} measure having one and two scalar parameters. Connection of \textit{generalized AG - divergence} measure with \textit{Fisher measure of information} is also presented. A unified \textit{generalization of AG - divergence}and\textit{Jensen-Shannon divergence measures} is also presented.
In this paper we consider one parameter generalizations of some non - symmetric divergence measures. Measures are \textit{relative information}, $χ^2 - $\textit{divergence}, \textit{relative J-divergence}, \textit{relative Jensen-Shannon divergence}and \textit{relative arithmetic and geometric divergence}. All the generalizations considered can be written as particular cases of Csiszár \textit{f-divergence}. By conditioning the probability distributions, relationships among the \textit{relative divergence measures}are obtained.
In this paper we shall consider one parametric generalization of some non-symmetric divergence measures. The \textit{non-symmetric divergence measures} are such as: Kullback-Leibler \textit{relative information}, $χ^2-$\textit{divergence}, \textit{relative J -- divergence}, \textit{relative Jensen -- Shannon divergence} and \textit{relative Arithmetic -- Geometric divergence}. All the generalizations considered can be written as particular cases of Csiszár's \textit{f-divergence}. By putting some conditions on the probability distribution, the aim here is to develop bounds on these measures an…
There are three classical divergence measures in the literature on information theory and statistics, namely, Jeffryes-Kullback-Leiber's J-divergence, Sibson-Burbea-Rao's Jensen-Shannon divegernce and Taneja's arithemtic-geometric mean divergence. These bear an interesting relationship among each other and are based on logarithmic expressions. The divergence measures like Hellinger discrimination, symmetric chi-square divergence, and triangular discrimination are not based on logarithmic expressions. These six divergence measures are symmetric with respect to probability distributions. In this…
There are three classical divergence measures known in the literature on information theory and statistics. These are namely, Jeffryes-Kullback-Leiber \cite{jef} \cite{kul} \textit{J-divergence}. Sibson-Burbea-Rao \cite{sib} \cite{bur1, bur2} \textit{Jensen-Shannon divegernce}and Taneja \cite{tan3} \textit{Arithemtic-Geometric divergence}. These three measures bears an interesting relationship among each other. The divergence measures like \textit{Hellinger discrimination}, \textit{symmetric}$χ^2 - $\textit{divergence}, and \textit{triangular discrimination} are also known in the literature. A…
In this paper we provide the asymptotic theory of the general of $ϕ$-divergences measures, which include the most common divergence measures : Renyi and Tsallis families and the Kullback-Leibler measure. We are interested in divergence measures in the discrete case. One sided and two-sided statistical tests are derived as well as symmetrized estimators. Almost sure rates of convergence and asymptotic normality theorem are obtained in the general case, and next particularized for the Renyi and Tsallis families and for the Kullback-Leibler measure as well. Our theorical results are validated by …
For a given parametric probability model, we consider the risk of the maximum likelihood estimator with respect to $α$-divergence, which includes the special cases of Kullback--Leibler divergence, the Hellinger distance and $χ^2$ divergence. The asymptotic expansion of the risk is given with respect to sample sizes of up to order $n^{-2}$. Each term in the expansion is expressed with the geometrical properties of the Riemannian manifold formed by the parametric probability model. We attempt to measure the difficulty of specifying a model through this expansion.
In previous work the authors defined the k-th order simplicial distance between probability distributions which arises naturally from a measure of dispersion based on the squared volume of random simplices of dimension k. This theory is embedded in the wider theory of divergences and distances between distributions which includes Kullback-Leibler, Jensen-Shannon, Jeffreys-Bregman divergence and Bhattacharyya distance. A general construction is given based on defining a directional derivative of a function $ϕ$ from one distribution to the other whose concavity or strict concavity influences the…
Divergence measures have a long association with statistical inference, machine learning and information theory. The density power divergence and related measures have produced many useful (and popular) statistical procedures, which provide a good balance between model efficiency on one hand and outlier stability or robustness on the other. The logarithmic density power divergence, a particular logarithmic transform of the density power divergence, has also been very successful in producing efficient and stable inference procedures; in addition it has also led to significant demonstrated appli…
Wide conditions are provided to guarantee asymptotic unbiasedness and L^2-consistency of the introduced estimates of the Kullback-Leibler divergence for probability measures in R^d having densities w.r.t. the Lebesgue measure. These estimates are constructed by means of two independent collections of i.i.d. observations and involve the specified k-nearest neighbor statistics. In particular, the established results are valid for estimates of the Kullback-Leibler divergence between any two Gaussian measures in R^d with nondegenerate covariance matrices. As a byproduct we obtain new statements co…
Minimizing divergence measures under a constraint is an important problem. We derive a sufficient condition that binary divergence measures provide lower bounds for symmetric divergence measures under a given triangular discrimination or given means and variances. Assuming this sufficient condition, the former bounds are always tight, and the latter bounds are tight when two probability measures have the same variance. An application of these results for nonequilibrium physics is provided.
As a crucial problem in statistics is to decide whether additional variables are needed in a regression model. We propose a new multivariate test to investigate the conditional mean independence of Y given X conditioning on some known effect Z, i.e., E(Y|X, Z) = E(Y|Z). Assuming that E(Y|Z) and Z are linearly related, we reformulate an equivalent notion of conditional mean independence through transformation, which is approximated in practice. We apply the martingale difference divergence (Shao and Zhang, 2014) to measure conditional mean dependence, and show that the estimation error from app…
The concept of varentropy has been recently introduced as a dispersion index of the reliability of measure of information. In this paper, we introduce new measures of variability for two measures of uncertainty, the Kerridge inaccuracy measure and the Kullback-Leibler divergence. These new definitions and related properties, bounds and examples are presented. Finally we show an application of Kullback-Leibler divergence and its dispersion index using the mean-variance rule.
We consider the minimization problem of $ϕ$-divergences between a given probability measure $P$ and subsets $Ω$ of the vector space $\mathcal{M}_\mathcal{F}$ of all signed finite measures which integrate a given class $\mathcal{F}$ of bounded or unbounded measurable functions. The vector space $\mathcal{M}_\mathcal{F}$ is endowed with the weak topology induced by the class $\mathcal{F}\cup \mathcal{B}_b$ where $\mathcal{B}_b$ is the class of all bounded measurable functions. We treat the problems of existence and characterization of the $ϕ$-projections of $P$ on $Ω$. We consider also the dual …
We introduce a new one-parameter family of divergence measures, called bounded Bhattacharyya distance (BBD) measures, for quantifying the dissimilarity between probability distributions. These measures are bounded, symmetric and positive semi-definite and do not require absolute continuity. In the asymptotic limit, BBD measure approaches the squared Hellinger distance. A generalized BBD measure for multiple distributions is also introduced. We prove an extension of a theorem of Bradt and Karlin for BBD relating Bayes error probability and divergence ranking. We show that BBD belongs to the cla…
$f$-divergences are a general class of divergences between probability measures which include as special cases many commonly used divergences in probability, mathematical statistics and information theory such as Kullback-Leibler divergence, chi-squared divergence, squared Hellinger distance, total variation distance etc. In this paper, we study the problem of maximizing or minimizing an $f$-divergence between two probability measures subject to a finite number of constraints on other $f$-divergences. We show that these infinite-dimensional optimization problems can all be reduced to optimizat…
