基于Q学习的加热网格世界首次穿越时间最小化研究
A study of first-passage time minimization via Q-learning in heated gridworlds
摘要:
在从纳米机器人导航到市场交易的应用中,需要优化首次通过时间。在这样的环境中,人们经常会遇到不均匀分布的噪声水平。我们广泛地研究了学习代理票价在1-和2-维加热gridworlds与不均匀的温度分布。结果表明,通过简单的表格Q-学习,SARSA,预期SARSA和双Q-学习训练的智能体中存在一定的偏差效应。虽然高学习率阻止了对具有较高温度的区域的探索,但足够低的学习率增加了这些区域中的代理的存在。在现实世界的物理应用和代理设计中,应该考虑到基于时间差的强化学习方法的特点和偏差。
英文摘要:
Optimization of first-passage times is required in applications ranging from nanobots navigation to market trading. In such settings, one often encounters unevenly distributed noise levels across the environment. We extensively study how a learning agent fares in 1- and 2- dimensional heated gridworlds with an uneven temperature distribution. The results show certain bias effects in agents trained via simple tabular Q-learning, SARSA, Expected SARSA and Double Q-learning. While high learning rate prevents exploration of regions with higher temperature, low enough rate increases the presence of agents in such regions. The discovered peculiarities and biases of temporal-difference-based reinforcement learning methods should be taken into account in real-world physical applications and agent design.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。