In this paper we remark that Shannon entropy can be expressed as a function of the self-information (i.e. the logarithm) and the inverse of the Lambert $W$ function. It means that we consider that Shannon entropy has the trace form: $-k \sum_{i} W^{-1} \circ \mathsf{ln}(p_{i})$. Based on this remark we define a generalized entropy which has as a limit the Shannon entropy. In order to facilitate the reasoning this generalized entropy is obtained by a one-parameter deformation of the logarithmic function.
Introducing a new concept of independence of two systems the Shannon additivity is replaced by a non-commutative and non-associative law which limit is the usual addition. The main properties associated with the generalized entropy are established, particularly those corresponding to statistical ensembles. The Boltzmann-Gibbs statistics is recovered as a limit. The connection with thermodynamics is also studied. We also provide a guideline for systematically defining a deformed algebra which limit is the classical linear algebra. As an illustrative example we study a generalized entropy based on Tsallis self-information. We point out possible connections between deformed algebra and fuzzy logics. Finally, noticing that the new concept of independence is based on t-norm the one-parameter deformation of the logarithm is interpreted as an additive generator of t-norms.
Jaynes & Shannon's Constrained Ignorance and Surprise
在这篇简单的文章中,在理论和应用物理中可能的应用,我们提出了一个原始的方法来推导香农熵的表达从一个纯粹的变分方法,使用约束。在Edwin T. Jaynes,我们的结果从根本上说并不是新的,但是它们导出的背景可能会导致一个非常一致的形式主义,其中最大熵原理自然出现。在此框架中给出了“无知”的一般定义之后,我们使用两种方法推导出熵的某种一般预期表达式。在第一种情况下,人们是有偏见的,对熵函数的形状有一个模糊的概念。在第二种情况下,我们考虑的一般情况下,没有什么是先验已知的。比较了两种思维方式的优点。