Conditional Shannon, Réyni, and Tsallis entropies estimation and asymptotic limits: discrete case

Non parametric estimation of joint, Renyi-Stallis entropies and mutual information and asymptotic limits

This paper proposes a new method for estimating the joint probability mass function of a pair of discrete random variables. This estimator is used to construct joint Shannon Rényi-Tsallis entropies, and the mutual information estimates of a pair of discrete random variables. Almost sure consistency and central limit Theorems are established. Our theorical results are validated by simulations.

Minimization Problems Based on Relative $α$-Entropy I: Forward Projection

Minimization problems with respect to a one-parameter family of generalized relative entropies are studied. These relative entropies, which we term relative $α$-entropies (denoted $\mathscr{I}_α$), arise as redundancies under mismatched compression when cumulants of compressed lengths are considered instead of expected compressed lengths. These parametric relative entropies are a generalization of the usual relative entropy (Kullback-Leibler divergence). Just like relative entropy, these relative $α$-entropies behave like squared Euclidean distance and satisfy the Pythagorean property. Minimiz…

关于累积Tsallis熵

On cumulative Tsallis entropies

本文研究累积Tsallis熵，将其视为平均剩余寿命的幂权重扰动，引入对偶熵和相干风险度量族，刻画其有限性与取值范围，并证明其最大化对应Logistic分布的扰动。

A Shannon-Tsallis transformation

We determine a general link between two different solutions of the MaxEnt variational problem, namely, the ones that correspond to using either Shannon's or Tsallis' entropies in the concomitant variational problem. It is shown that the two variations lead to equivalent solutions that take different appearances but contain the same information. These solutions are linked by our transformation.

Nonextensive Generalizations of the Jensen-Shannon Divergence

Convexity is a key concept in information theory, namely via the many implications of Jensen's inequality, such as the non-negativity of the Kullback-Leibler divergence (KLD). Jensen's inequality also underlies the concept of Jensen-Shannon divergence (JSD), which is a symmetrized and smoothed version of the KLD. This paper introduces new JSD-type divergences, by extending its two building blocks: convexity and Shannon's entropy. In particular, a new concept of q-convexity is introduced and shown to satisfy a Jensen's q-inequality. Based on this Jensen's q-inequality, the Jensen-Tsallis q-diff…