分支时间主动推理：实证研究与复杂性类别分析
Branching Time Active Inference: empirical study and complexity class analysis
摘要：
主动推理是一种最先进的大脑建模框架，可以解释各种机制，如习惯形成，多巴胺能放电和好奇心。然而，最近的实现遭受的指数复杂度类时，计算所有可能的政策的时间范围内的先验。Fountas等人（2020）使用蒙特卡罗树搜索来解决这个问题，在两个不同的任务中得到了非常好的结果。此外，Champion等人（2021 a）提出了一种基于（时间）结构学习的树搜索方法。这是通过开发一种可变的消息传递方法来实现的，该方法可以实现贝叶斯网络的组合构建以进行主动推理。然而，这种消息传递树搜索方法，我们称之为分支时间主动推理（BTAI），从未进行过经验测试。在本文中，我们提出了一个实验研究的背景下，BTAI的迷宫解决代理。在这种情况下，我们表明，改进的先验偏好和更深入的搜索都有助于减轻局部最小值的脆弱性。然后，我们比较BTAI标准的主动推理（AcI）的图形导航任务。我们表明，对于小图，BTAI和AcI成功地解决了任务。对于较大的图，AcI表现出指数（空间）复杂性类，使该方法难以处理。然而，BTAI更有效地探索了政策空间，成功地扩展到更大的图表。然后，在冰冻湖环境下，将BTAI算法与POMCP算法进行了比较。实验表明，BTAI和POMCP算法积累了类似的奖励金额。此外，我们描述了当BTAI收到更多的奖励比POMCP代理，当相反的是真的。最后，我们将BTAI与Fountas et al（2020）在dSprites数据集上的方法进行了比较，并讨论了每种方法的优缺点。
英文摘要：
Active inference is a state-of-the-art framework for modelling the brain that explains a wide range of mechanisms such as habit formation, dopaminergic discharge and curiosity. However, recent implementations suffer from an exponential complexity class when computing the prior over all the possible policies up to the time horizon. Fountas et al (2020) used Monte Carlo tree search to address this problem, leading to very good results in two different tasks. Additionally, Champion et al (2021a) proposed a tree search approach based on (temporal) structure learning. This was enabled by the development of a variational message passing approach to active inference, which enables compositional construction of Bayesian networks for active inference. However, this message passing tree search approach, which we call branching-time active inference (BTAI), has never been tested empirically. In this paper, we present an experimental study of BTAI in the context of a maze solving agent. In this context, we show that both improved prior preferences and deeper search help mitigate the vulnerability to local minima. Then, we compare BTAI to standard active inference (AcI) on a graph navigation task. We show that for small graphs, both BTAI and AcI successfully solve the task. For larger graphs, AcI exhibits an exponential (space) complexity class, making the approach intractable. However, BTAI explores the space of policies more efficiently, successfully scaling to larger graphs. Then, BTAI was compared to the POMCP algorithm on the frozen lake environment. The experiments suggest that BTAI and the POMCP algorithm accumulate a similar amount of reward. Also, we describe when BTAI receives more rewards than the POMCP agent, and when the opposite is true. Finally, we compared BTAI to the approach of Fountas et al (2020) on the dSprites dataset, and we discussed the pros and cons of each approach.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
