arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2110.10083cs.LG
收藏

对比主动推理

Contrastive Active Inference

IDLab;Ghent University(根特大学)

Pietro Mazzaglia, Tim Verbelen, Bart Dhoedt

AI总结:

本文提出对比主动推理方法,通过对比目标降低主动推理在生成模型学习与行动规划中的计算负担,在图像任务中超越基于似然的方法,性能接近强化学习,并能有效应对环境干扰和目标泛化。

摘要:

主动推理是一种统一的感知与行动理论,其核心思想是大脑通过最小化自由能来维持对世界的内部模型。从行为角度看,主动推理智能体可被视为自我证明的存在,它们通过行动来实现其乐观预测,即偏好结果或目标。相比之下,强化学习需要人工设计的奖励才能实现任何期望的结果。尽管主动推理可以为控制提供更自然的自监督目标,但由于将该方法扩展到复杂环境方面存在不足,其适用性一直受到限制。在本工作中,我们提出了一种用于主动推理的对比目标,该目标大幅降低了学习智能体生成模型和规划未来行动的计算负担。我们的方法在基于图像的任务中表现明显优于基于似然的主动推理,同时计算成本更低且更易于训练。我们与能够获得人工设计奖励函数的强化学习智能体进行了比较,结果表明我们的方法在性能上与其非常接近。最后,我们还表明,在环境中存在干扰因素的情况下,对比方法的表现显著更好,并且我们的方法能够将目标泛化到背景变化的情形。网站与代码:https://contrastive-aif.github.io/

英文摘要:

Active inference is a unifying theory for perception and action resting upon the idea that the brain maintains an internal model of the world by minimizing free energy. From a behavioral perspective, active inference agents can be seen as self-evidencing beings that act to fulfill their optimistic predictions, namely preferred outcomes or goals. In contrast, reinforcement learning requires human-designed rewards to accomplish any desired outcome. Although active inference could provide a more natural self-supervised objective for control, its applicability has been limited because of the shortcomings in scaling the approach to complex environments. In this work, we propose a contrastive objective for active inference that strongly reduces the computational burden in learning the agent's generative model and planning future actions. Our method performs notably better than likelihood-based active inference in image-based tasks, while also being computationally cheaper and easier to train. We compare to reinforcement learning agents that have access to human-designed reward functions, showing that our approach closely matches their performance. Finally, we also show that contrastive methods perform significantly better in the case of distractors in the environment and that our method is able to generalize goals to variations in the background. Website and code: https://contrastive-aif.github.io/

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑