最小描述长度归纳法、贝叶斯主义和柯尔莫哥洛夫复杂性
Minimum Description Length Induction, Bayesianism, and Kolmogorov Complexity
摘要：
建立了贝叶斯方法和最小描述长度方法之间的关系。我们锐化和澄清的一般建模原则MDL和MML，抽象为理想MDL原则，定义从贝叶斯规则的Kolmogorov复杂性。应用理想原理的基本条件被概括为基本不等式，广义地说，当数据相对于每个预期的假设是随机的，并且这些假设相对于（普遍的）先验是随机的时，该原理是有效的。基本上，理想原理指出，与假设相关的先验概率应该由算法普适概率给出，并且模型的对数普适概率加上给定模型的数据概率的对数之和应该最小化。如果我们将模型类限制在有限集上，那么理想原理的应用就变成了柯尔莫哥洛夫最小充分统计量。在一般情况下，我们表明，数据压缩几乎总是最好的策略，无论是在假设识别和预测。
英文摘要：
The relationship between the Bayesian approach and the minimum description length approach is established. We sharpen and clarify the general modeling principles MDL and MML, abstracted as the ideal MDL principle and defined from Bayes's rule by means of Kolmogorov complexity. The basic condition under which the ideal principle should be applied is encapsulated as the Fundamental Inequality, which in broad terms states that the principle is valid when the data are random, relative to every contemplated hypothesis and also these hypotheses are random relative to the (universal) prior. Basically, the ideal principle states that the prior probability associated with the hypothesis should be given by the algorithmic universal probability, and the sum of the log universal probability of the model plus the log of the probability of the data given the model should be minimized. If we restrict the model class to the finite sets then application of the ideal principle turns into Kolmogorov's minimal sufficient statistic. In general we show that data compression is almost always the best strategy, both in hypothesis identification and prediction.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
