arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.12178cs.LGcs.AIstat.ML
收藏

探索无界

Exploration Unbound

Stanford University(斯坦福大学)

Dilip Arumugam, Wanqiao Xu, Benjamin Van Roy

AI总结:

本文研究序贯决策中探索与利用的平衡,提出一个奖励无界且探索收益永不消失的典型环境,证明最优智能体将永远保持探索倾向。

摘要:

一个序贯决策智能体需要在探索环境以获取新知识与利用当前知识以最大化即时奖励之间进行平衡。在传统文献所研究的环境中,随着智能体积累足够的知识且进一步探索的收益逐渐消失,最优决策会随时间推移倾向于利用。然而,如果环境提供了无限量的有用知识,并且无论智能体已学到多少,进一步探索都有巨大收益,那又该如何?我们提供了一个简单且典型的此类复杂环境示例。在该环境中,奖励是无界的,智能体总能通过探索学习更多来增加奖励累积的速率。因此,一个最优智能体会永远保持探索的倾向。

英文摘要:

A sequential decision-making agent balances between exploring to gain new knowledge about an environment and exploiting current knowledge to maximize immediate reward. For environments studied in the traditional literature, optimal decisions gravitate over time toward exploitation as the agent accumulates sufficient knowledge and the benefits of further exploration vanish. What if, however, the environment offers an unlimited amount of useful knowledge and there is large benefit to further exploration no matter how much the agent has learned? We offer a simple, quintessential example of such a complex environment. In this environment, rewards are unbounded and an agent can always increase the rate at which rewards accumulate by exploring to learn more. Consequently, an optimal agent forever maintains a propensity to explore.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑