arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2103.06859cs.LGcs.AI
收藏

从概率控制目标看信息寻求探源

Understanding the Origin of Information-Seeking Exploration in Probabilistic Objectives for Control

University of Edinburgh(爱丁堡大学);University of Sussex(萨塞克斯大学)

Beren Millidge, Anil Seth, Christopher Buckley

摘要:

探索-利用的权衡是描述从机器学习到生物学、经济学等领域的适应行为的核心。虽然已经采取了许多方法,解决这种权衡的一种方法是装备或提出代理拥有内在的“探索驱动”,这通常是在最大限度地提高代理对世界的信息增益方面实现的--这种方法在机器学习和认知科学中得到了广泛的研究。在本文中,我们从数学上研究这种方法的性质和意义,并证明了这种效用最大化和信息寻求行为的组合产生于一个完全不同的目标,我们称之为发散目标类的最小化。我们提出了一个二分法的目标函数之间的自适应行为\emph{evidence}目标,这对应于众所周知的回报或效用最大化的目标在文献中,而\emph{divergence}目标,而不是寻求最大限度地减少代理的预期和期望的未来之间的分歧,并认为,这类新的分歧目标可以形成更丰富的理解探索性的数学基础,自适应和智能行动的组成部分,超越简单的贪婪效用最大化。

英文摘要:

The exploration-exploitation trade-off is central to the description of adaptive behaviour in fields ranging from machine learning, to biology, to economics. While many approaches have been taken, one approach to solving this trade-off has been to equip or propose that agents possess an intrinsic 'exploratory drive' which is often implemented in terms of maximizing the agents information gain about the world -- an approach which has been widely studied in machine learning and cognitive science. In this paper we mathematically investigate the nature and meaning of such approaches and demonstrate that this combination of utility maximizing and information-seeking behaviour arises from the minimization of an entirely difference class of objectives we call divergence objectives. We propose a dichotomy in the objective functions underlying adaptive behaviour between \emph{evidence} objectives, which correspond to well-known reward or utility maximizing objectives in the literature, and \emph{divergence} objectives which instead seek to minimize the divergence between the agent's expected and desired futures, and argue that this new class of divergence objectives could form the mathematical foundation for a much richer understanding of the exploratory components of adaptive and intelligent action, beyond simply greedy utility maximization.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑