arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2302.06083cs.AI
收藏

通用智能体混合与智能的几何结构

Universal Agent Mixtures and the Geometry of Intelligence

Australian National University(澳大利亚国立大学);WooliesX(沃利克斯公司);DeepMind(深度思维公司);The U.S. Securities and Exchange Commission(美国证券交易委员会)

Samuel Allen Alexander, David Quarel, Len Du, Marcus Hutter

AI总结:

本研究引入加权混合运算探究通用智能体的集体智能行为,推导了揭示RL智能体智能几何特性的新定理,并证明满足弱条件的跨环境平均表现智能度量等价于单一环境中的表现。

摘要:

受多智能体强化学习(Reinforcement Learning,RL)近期进展的启发,本研究通过引入加权混合运算,探究理论通用智能体的集体智能行为。给定一组带权重的智能体,它们的加权混合是一个新智能体,其在任意环境中的期望总奖励等于原各智能体在该环境中期望总奖励的对应加权平均值。因此,若用跨环境表现来量化RL智能体的智能水平,加权混合智能体的智能水平就是原智能体智能水平的加权平均值。该运算推导出多个揭示RL智能体智能几何特性的新定理,涉及对称性、凸智能体集和局部极值的相关结果。研究还证明,任何基于跨环境平均表现的RL智能体智能度量,在满足若干弱技术条件的前提下,都(相差一个常数因子地等同于某个依赖于该智能度量的单一环境中的表现。

英文摘要:

Inspired by recent progress in multi-agent Reinforcement Learning (RL), in this work we examine the collective intelligent behaviour of theoretical universal agents by introducing a weighted mixture operation. Given a weighted set of agents, their weighted mixture is a new agent whose expected total reward in any environment is the corresponding weighted average of the original agents' expected total rewards in that environment. Thus, if RL agent intelligence is quantified in terms of performance across environments, the weighted mixture's intelligence is the weighted average of the original agents' intelligences. This operation enables various interesting new theorems that shed light on the geometry of RL agent intelligence, namely: results about symmetries, convex agent-sets, and local extrema. We also show that any RL agent intelligence measure based on average performance across environments, subject to certain weak technical conditions, is identical (up to a constant factor) to performance within a single environment dependent on said intelligence measure.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑