关于信息增益、Kullback-Leibler散度、熵产生和对合核
On information gain, Kullback-Leibler divergence, entropy production and the involution kernel
摘要：
众所周知，在信息论和机器学习中，Kullback-Leibler散度起着基础性的作用，它扩展了Shannon熵的概念。给定可测空间$X\times Y$上的{ \it先验}概率核$\hatν$和概率$π$，在前人工作的基础上，我们考虑了$π$相对于$\hatν$的熵的一个合适的定义.利用这个熵的概念，我们得到了一般可测空间的信息增益的自然定义，它与$\hatν$在$X$上以概率$ν$标识的情况下由K-L散度给出的互信息相一致。这将被用来扩展特定的信息增益和动力学熵产生的意义，在一个紧凑的字母表（TFCA模型）的符号动力学的热力学形式主义的模型。在这种情况下，我们表明，对合核是一个自然的工具，更好地理解熵产生的一些重要性质。
英文摘要：
It is well known that in Information Theory and Machine Learning the Kullback-Leibler divergence, which extends the concept of Shannon entropy, plays a fundamental role. Given an {\it a priori} probability kernel $\hatν$ and a probability $π$ on the measurable space $X\times Y$ we consider an appropriate definition of entropy of $π$ relative to $\hatν$, which is based on previous works. Using this concept of entropy we obtain a natural definition of information gain for general measurable spaces which coincides with the mutual information given from the K-L divergence in the case $\hatν$ is identified with a probability $ν$ on $X$. This will be used to extend the meaning of specific information gain and dynamical entropy production to the model of thermodynamic formalism for symbolic dynamics over a compact alphabet (TFCA model). In this case, we show that the involution kernel is a natural tool for better understanding some important properties of entropy production.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
