arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2110.02129math.OCcond-mat.stat-mechcs.AIcs.LGmath.DS
收藏

基于Q学习的加热网格世界首次穿越时间最小化研究

A study of first-passage time minimization via Q-learning in heated gridworlds

Center for Computational and Data-intensive Science and Engineering, Skolkovo Institute of Science and Technology(斯科尔科沃科学技术研究所计算与数据密集型科学工程中心)

M. A. Larchenko, P. Osinenko, G. Yaremenko, V. V. Palyulin

摘要:

在从纳米机器人导航到市场交易的应用中,需要优化首次通过时间。在这样的环境中,人们经常会遇到不均匀分布的噪声水平。我们广泛地研究了学习代理票价在1-和2-维加热gridworlds与不均匀的温度分布。结果表明,通过简单的表格Q-学习,SARSA,预期SARSA和双Q-学习训练的智能体中存在一定的偏差效应。虽然高学习率阻止了对具有较高温度的区域的探索,但足够低的学习率增加了这些区域中的代理的存在。在现实世界的物理应用和代理设计中,应该考虑到基于时间差的强化学习方法的特点和偏差。

英文摘要:

Optimization of first-passage times is required in applications ranging from nanobots navigation to market trading. In such settings, one often encounters unevenly distributed noise levels across the environment. We extensively study how a learning agent fares in 1- and 2- dimensional heated gridworlds with an uneven temperature distribution. The results show certain bias effects in agents trained via simple tabular Q-learning, SARSA, Expected SARSA and Double Q-learning. While high learning rate prevents exploration of regions with higher temperature, low enough rate increases the presence of agents in such regions. The discovered peculiarities and biases of temporal-difference-based reinforcement learning methods should be taken into account in real-world physical applications and agent design.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑