arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2307.02620cs.LGcs.AI
收藏

观测成本敏感强化学习中的动态观测策略

Dynamic Observation Policies in Observation Cost-Sensitive Reinforcement Learning

National Research Council of Canada(加拿大国家研究委员会);Dalhousie University(达尔豪斯大学);University of Waterloo(滑铁卢大学);University of Ottawa(渥太华大学);Vector Institute for Artificial Intelligence(矢量人工智能研究所)

Colin Bellinger, Mark Crowley, Isaac Tamblyn

AI总结:

本文提出深度动态多步无观测智能体(DMSOA),在观测成本敏感的强化学习场景中,通过动态跳过不必要的测量,以更少的决策步骤和测量学习更优策略,并在OpenAI gym和Atari Pong环境中验证了其有效性。

摘要:

强化学习(RL)已被证明能够为复杂任务学习复杂的控制策略,包括游戏、机器人、供暖和制冷系统以及文本生成。然而,RL中的行动-感知循环通常假设在每个时间步都可以无成本地获得环境状态的测量。然而,在材料设计、深海和行星机器人探索以及医学等应用中,测量甚至近似环境状态可能伴随着高昂的成本。在本文中,我们综述了最近增长的文献,这些文献采纳了RL智能体可能不需要甚至不想要在每个时间步进行昂贵测量的观点。在此背景下,我们提出了深度动态多步无观测智能体(DMSOA),将其与文献中的方法进行对比,并在OpenAI gym和Atari Pong环境中进行实证评估。我们的结果表明,与文献中考虑的替代方法相比,DMSOA以更少的决策步骤和测量学习了更好的策略。

英文摘要:

Reinforcement learning (RL) has been shown to learn sophisticated control policies for complex tasks including games, robotics, heating and cooling systems and text generation. The action-perception cycle in RL, however, generally assumes that a measurement of the state of the environment is available at each time step without a cost. In applications such as materials design, deep-sea and planetary robot exploration and medicine, however, there can be a high cost associated with measuring, or even approximating, the state of the environment. In this paper, we survey the recently growing literature that adopts the perspective that an RL agent might not need, or even want, a costly measurement at each time step. Within this context, we propose the Deep Dynamic Multi-Step Observationless Agent (DMSOA), contrast it with the literature and empirically evaluate it on OpenAI gym and Atari Pong environments. Our results, show that DMSOA learns a better policy with fewer decision steps and measurements than the considered alternative from the literature.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑