arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2308.07795cs.CVcs.AI
收藏

学习从视频中识别强化学习的关键状态

Learning to Identify Critical States for Reinforcement Learning from Videos

King Abdullah University of Science and Technology(阿卜杜拉国王科技大学);The Swiss AI Lab IDSIA/USI/SUPSI(瑞士人工智能实验室(IDSIA/USI/SUPSI));NNAISENSE(NNAISENSE公司)

Haozhe Liu, Mingchen Zhuge, Bing Li, Yuhui Wang, Francesco Faccio, Bernard Ghanem, Jürgen Schmidhuber

AI总结:

本文提出 Deep State Identifier,无需标注即可从强化学习视频中预测回报,并通过基于掩码的敏感性分析识别关键状态,从而理解和改进智能体行为。

摘要:

近期关于深度强化学习(DRL)的研究指出,可以从缺乏已执行动作显式信息的离线数据中提取有关良好策略的算法信息。例如,人类或机器人的视频可能传达大量关于带来回报的动作序列的隐含信息,但希望通过观看此类视频获益的 DRL 机器,首先必须自行学习识别和辨认相关的状态、动作与回报。在不依赖真值标注的情况下,我们名为 Deep State Identifier 的新方法学习从编码为视频的回合中预测回报。随后,它采用一种基于掩码的敏感性分析来提取并识别重要的关键状态。大量实验展示了该方法在理解和改进智能体行为方面的潜力。源代码和生成的数据集可在 https://github.com/AI-Initiative-KAUST/VideoRLCS 获取。

英文摘要:

Recent work on deep reinforcement learning (DRL) has pointed out that algorithmic information about good policies can be extracted from offline data which lack explicit information about executed actions. For example, videos of humans or robots may convey a lot of implicit information about rewarding action sequences, but a DRL machine that wants to profit from watching such videos must first learn by itself to identify and recognize relevant states/actions/rewards. Without relying on ground-truth annotations, our new method called Deep State Identifier learns to predict returns from episodes encoded as videos. Then it uses a kind of mask-based sensitivity analysis to extract/identify important critical states. Extensive experiments showcase our method's potential for understanding and improving agent behavior. The source code and the generated datasets are available at https://github.com/AI-Initiative-KAUST/VideoRLCS.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑