arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2112.13404cs.AIcs.LG
收藏

一般强化学习的抽象

Abstractions of General Reinforcement Learning

Australian National University(澳大利亚国立大学)

Sultan J. Majeed

摘要:

人工智能(AI)领域致力于创造人工决策者,这些决策者可以在感兴趣的领域(至少)与人类同行相提并论。与传统人工智能中的智能体不同,人工通用智能(AGI)中的智能体需要在几乎所有感兴趣的领域复制人类智能。此外,AGI代理应该能够实现这一点,而无需(几乎任何)进一步的更改,重新训练或微调参数。现实世界是非平稳的、非遍历的、非马尔可夫的:我们人类既不能重温我们的过去,最近的观察也不足以统计。然而,我们擅长各种复杂的任务。其中许多任务需要长期规划。我们可以将这种成功与我们从压倒性的感官体验中提取与任务无关的信息的自然能力联系起来。我们不费吹灰之力就能建立起特定于任务的世界心理模型。由于这种抽象的能力,我们可以计划一个非常紧凑的任务表示,而不会损失太多的性能。不仅如此,我们还抽象我们的行动,以产生高层次的计划:行动抽象的水平可以是任何地方之间的小肌肉运动的心理概念“做一个行动”。我们很自然地认为,任何与人类竞争的AGI智能体(在每一个可能的领域)都应该具有抽象其经验和行动的能力。这篇论文是一个调查存在这样的抽象,帮助有效的规划范围广泛的领域,最重要的是,这些抽象来一些最优性保证。

英文摘要:

The field of artificial intelligence (AI) is devoted to the creation of artificial decision-makers that can perform (at least) on par with the human counterparts on a domain of interest. Unlike the agents in traditional AI, the agents in artificial general intelligence (AGI) are required to replicate human intelligence in almost every domain of interest. Moreover, an AGI agent should be able to achieve this without (virtually any) further changes, retraining, or fine-tuning of the parameters. The real world is non-stationary, non-ergodic, and non-Markovian: we, humans, can neither revisit our past nor are the most recent observations sufficient statistics. Yet, we excel at a variety of complex tasks. Many of these tasks require longterm planning. We can associate this success to our natural faculty to abstract away task-irrelevant information from our overwhelming sensory experience. We make task-specific mental models of the world without much effort. Due to this ability to abstract, we can plan on a significantly compact representation of a task without much loss of performance. Not only this, we also abstract our actions to produce high-level plans: the level of action-abstraction can be anywhere between small muscle movements to a mental notion of "doing an action". It is natural to assume that any AGI agent competing with humans (at every plausible domain) should also have these abilities to abstract its experiences and actions. This thesis is an inquiry into the existence of such abstractions which aid efficient planing for a wide range of domains, and most importantly, these abstractions come with some optimality guarantees.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑