一种基于sojourn的半马尔可夫强化学习方法
A sojourn-based approach to semi-Markov Reinforcement Learning
摘要：
本文基于逗留时间过程，给出了离散半马尔可夫决策过程的一种新方法。利用离散时间半马尔可夫过程的不同特征，并利用它们构造决策过程。通过这种新的方法，代理被允许考虑不同的行动，这也取决于进程在当前状态下的逗留时间。研究了基于$Q$学习算法的有限时域强化学习和随机递归关系的数值方法。最后，我们考虑两个玩具的例子：一个是奖励取决于逗留时间，根据赌徒的谬论;另一个是环境是半马尔可夫的，即使报酬函数不依赖于逗留时间。这些都是用来进行一些数值评估先前提出的$Q$学习算法和不同的朴素的方法基于深度强化学习。
英文摘要：
In this paper we introduce a new approach to discrete-time semi-Markov decision processes based on the sojourn time process. Different characterizations of discrete-time semi-Markov processes are exploited and decision processes are constructed by their means. With this new approach, the agent is allowed to consider different actions depending also on the sojourn time of the process in the current state. A numerical method based on $Q$-learning algorithms for finite horizon reinforcement learning and stochastic recursive relations is investigated. Finally, we consider two toy examples: one in which the reward depends on the sojourn-time, according to the gambler's fallacy; the other in which the environment is semi-Markov even if the reward function does not depend on the sojourn time. These are used to carry on some numerical evaluations on the previously presented $Q$-learning algorithm and on a different naive method based on deep reinforcement learning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
