学习与行动的最小相对熵原理
A Minimum Relative Entropy Principle for Learning and Acting
摘要：
本文提出了一种方法来构建一个自适应代理，这是普遍的，相对于一个给定的一类专家，其中每个专家是一个代理，已被专门设计为一个特定的环境。这种自适应控制问题被形式化为最小化最适合未知环境的专家的自适应代理的相对熵的问题。如果代理是一个被动的观察者，那么最佳解决方案是众所周知的贝叶斯预测。然而，如果智能体是活动的，那么它过去的动作需要被视为I/O流上的因果干预，而不是正常的概率条件。在这里，它表明，这个新的变分问题的解决方案是由一个随机控制器称为贝叶斯控制规则，实现自适应行为作为一个混合专家模型。此外，它表明，在温和的假设下，贝叶斯控制规则收敛到最合适的专家的控制律。
英文摘要：
This paper proposes a method to construct an adaptive agent that is universal with respect to a given class of experts, where each expert is an agent that has been designed specifically for a particular environment. This adaptive control problem is formalized as the problem of minimizing the relative entropy of the adaptive agent from the expert that is most suitable for the unknown environment. If the agent is a passive observer, then the optimal solution is the well-known Bayesian predictor. However, if the agent is active, then its past actions need to be treated as causal interventions on the I/O stream rather than normal probability conditions. Here it is shown that the solution to this new variational problem is given by a stochastic controller called the Bayesian control rule, which implements adaptive behavior as a mixture of experts. Furthermore, it is shown that under mild assumptions, the Bayesian control rule converges to the control law of the most suitable expert.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
