最佳信念逼近
Optimal Belief Approximation
摘要:
在贝叶斯统计中,概率分布表示信念。然而,对于许多问题的信念不能计算分析和近似的信念是必要的。我们寻求一个损失函数来量化传达一个给定的近似值有多"尴尬"。我们复制并讨论了一个旧的证明,表明只有一个排名的要求下,(1)最好的排名近似是非近似的信念和(2)排名判断近似只通过他们的预测的实际结果。在推导中获得的损失函数等于归一化时的Kullback-Leibler散度。这种损失函数在文献中经常使用。然而,似乎有混乱的正确顺序,其功能的论点,近似和非近似的信念,应该使用。正确的顺序可确保通信的接收者只被剥夺最少量的信息量。我们希望这个基本推导能解决这个明显的混淆。例如,当用高斯分布近似置信时,通过矩匹配给出最优逼近。这与许多建议的计算方案相反。
英文摘要:
In Bayesian statistics probability distributions express beliefs. However, for many problems the beliefs cannot be computed analytically and approximations of beliefs are needed. We seek a loss function that quantifies how "embarrassing" it is to communicate a given approximation. We reproduce and discuss an old proof showing that there is only one ranking under the requirements that (1) the best ranked approximation is the non-approximated belief and (2) that the ranking judges approximations only by their predictions for actual outcomes. The loss function that is obtained in the derivation is equal to the Kullback-Leibler divergence when normalized. This loss function is frequently used in the literature. However, there seems to be confusion about the correct order in which its functional arguments, the approximated and non-approximated beliefs, should be used. The correct order ensures that the recipient of a communication is only deprived of the minimal amount of information. We hope that the elementary derivation settles the apparent confusion. For example when approximating beliefs with Gaussian distributions the optimal approximation is given by moment matching. This is in contrast to many suggested computational schemes.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。