Is there a canonical way to think of agency beyond reward maximisation? In this paper, we show that any type of behaviour complying with physically sound assumptions about how macroscopic biological agents interact with the world canonically integrates exploration and exploitation in the sense of minimising risk and ambiguity about states of the world. This description, known as active inference, refines the free energy principle, a popular descriptive framework for action and perception originating in neuroscience. Active inference provides a normative Bayesian framework to simulate and model agency that is widely used in behavioural neuroscience, reinforcement learning (RL) and robotics. The usefulness of active inference for RL is three-fold. \emph{a}) Active inference provides a principled solution to the exploration-exploitation dilemma that usefully simulates biological agency. \emph{b}) It provides an explainable recipe to simulate behaviour, whence behaviour follows as an explainable mixture of exploration and exploitation under a generative world model, and all differences in behaviour are explicit in differences in world model. \emph{c}) This framework is universal in the sense that it is theoretically possible to rewrite any RL algorithm conforming to the descriptive assumptions of active inference as an active inference algorithm. Thus, active inference can be used as a tool to uncover and compare the commitments and assumptions of more specific models of agency.
Active Inference: A method for Phenotyping Agency in AI systems?
Independent Researcher Laboratoire d'Analyse Cognitive de l'Information, Universit\' e du Qu\' e bec \` a Montr\' e al, Qu\' e bec, Canada Centre of Excellence for AI；Robotics, Sheffield Hallam University, Sheffield, UK Methods；Statistical Computing, Max Planck Institute for Human Cognitive；Brain Sciences, Leipzig, Germany Department of Computer Science, School of Physics, Engineering；Computer Science, University of Hertfordshire, Hatfield, UK Wellcome Centre for Human Neuroimaging, University College London, London, UK Department of Engineering；Informatics, University of Sussex, Falmer, Brighton, BN1 9RH, UK
From Assistance to Agency: Rethinking Autonomy and Control in CI/CD Pipelines
Faculty of Information University of Toronto Toronto Ontario Canada（信息学院多伦多大学多伦多安大略加拿大）；Department of Computer Science Trent University Peterborough Ontario Canada（计算机科学系特伦特大学彼得伯格安大略加拿大）；University of Toronto（多伦多大学）；Trent University（特伦特大学）
pymdp: A Python library for active inference in discrete state spaces
Max Planck Institute of Animal Behavior（马克斯·普朗克动物行为研究所）；Centre for the Advanced Study of Collective Behaviour（集体行为高级研究中心）；University of Konstanz（康斯坦茨大学）；VERSES Research Lab（VERSES研究实验室）；University of Oxford（牛津大学）；Imperial College London（帝国理工学院）；Northeastern University（东北大学）；University College London（伦敦大学学院）；University of Sussex（萨塞克斯大学）
Expanding the Active Inference Landscape: More Intrinsic Motivations in the Perception-Action Loop
Araya Inc.（Araya公司）；Computational Creativity Group, Department of Computing, Goldsmiths, University of London（Goldsmiths大学计算创意小组）；Game Innovation Lab, Department of Computer Science and Engineering, New York University（纽约大学游戏创新实验室）；Sepia Lab, Adaptive Systems Research Group, Department of Computer Science, University of Hertfordshire（赫特福德大学计算机科学系Sepia实验室）；Institute of Perception, Action and Behaviour, School of Informatics, The University of Edinburgh（爱丁堡大学信息学院感知、行为与行为研究所）
Sackler Centre for Consciousness Science（萨克勒意识科学中心）；Evolutionary & Adaptive Systems Research Group（进化与自适应系统研究组）；University of Sussex（萨塞克斯大学）；University of Edinburgh（爱丁堡大学）；Canadian Institute for Advanced Research（加拿大高级研究所）
Institute of Cognitive Sciences and Technologies, National Research Council（国家研究委员会认知科学与技术研究所）；Institute for High Performance Computing and Networking, National Research Council（国家研究委员会高性能计算与网络研究所）；University College London（伦敦大学学院）；VERSES AI Research Lab（VERSES人工智能研究实验室）
Branching Time Active Inference: the theory and its generality
University of Kent（肯特大学）；Imperial College London（帝国理工学院）；Wellcome Centre for Human Neuroimaging, University College London（伦敦大学学院惠康人类神经影像中心）；University of Birmingham（伯明翰大学）
Department of Electrical Engineering（电气工程系）；Eindhoven University of Technology（埃因霍温理工大学）；Eindhoven, the Netherlands（荷兰埃因霍温）；Lazy Dynamics；Utrecht, the Netherlands（荷兰乌得勒支）
Branching Time Active Inference: empirical study and complexity class analysis
University of Kent（肯特大学）；University of Birmingham（伯明翰大学）
主动推理是一种最先进的大脑建模框架，可以解释各种机制，如习惯形成，多巴胺能放电和好奇心。然而，最近的实现遭受的指数复杂度类时，计算所有可能的政策的时间范围内的先验。Fountas等人（2020）使用蒙特卡罗树搜索来解决这个问题，在两个不同的任务中得到了非常好的结果。此外，Champion等人（2021 a）提出了一种基于（时间）结构学习的树搜索方法。这是通过开发一种可变的消息传递方法来实现的，该方法可以实现贝叶斯网络的组合构建以进行主动推理。然而，这种消息传递树搜索方法，我们称之为分支时间主动推理（BTAI），从未进行过经验测试。在本文中，我们提出了一个实验研究的背景下，BTAI的迷宫解决代理。在这种情况下，我们表明，改进的先验偏好和更深入的搜索都有助于减轻局部最小值的脆弱性。然后，我们比较BTAI标准的主动推理（AcI）的图形导航任务。我们表明，对于小图，BTAI和AcI成功地解决了任务。对于较大的图，AcI表现出指数（空间）复杂性类，使该方法难以处理。然而，BTAI更有效地探索了政策空间，成功地扩展到更大的图表。然后，在冰冻湖环境下，将BTAI算法与POMCP算法进行了比较。实验表明，BTAI和POMCP算法积累了类似的奖励金额。此外，我们描述了当BTAI收到更多的奖励比POMCP代理，当相反的是真的。最后，我们将BTAI与Fountas et al（2020）在dSprite…
Designing explainable artificial intelligence with active inference: A framework for transparent introspection and decision-making
VERSES Research Lab（VERSES研究实验室）；Université du Québec à Montréal（魁北克大学蒙特利尔分校）；Humboldt-Universität zu Berlin（柏林洪堡大学）；Macquarie University（麦考瑞大学）；University College London（伦敦大学学院）
Mapping Husserlian phenomenology onto active inference
VERSES Research Lab（VERSES研究实验室）；Université du Québec à Montréal（魁北克大学蒙特利尔分校）；University of Oxford（牛津大学）；University College London（伦敦大学学院）；University of California, Merced（加利福尼亚大学默塞德分校）
Active Inference in String Diagrams: A Categorical Account of Predictive Processing and Free Energy
Quantinuum（昆蒂努姆）；Association for Mathematical Consciousness Science（数学意识科学协会）；Ludwig Maximilian University of Munich（慕尼黑路德维希-马克西米利安大学）；University of Bamberg（班贝格大学）；Topos Institute（托波斯研究所）；VERSES Research（VERSES研究院）
