Bellman Meets Hawkes：基于时间点过程的基于模型的强化学习
Bellman Meets Hawkes: Model-Based Reinforcement Learning via Temporal Point Processes
摘要：
我们考虑一个序贯决策问题，代理面临的环境特征的随机离散事件，并寻求一个最优的干预政策，使其长期回报最大化。这个问题普遍存在于社交媒体、金融和健康信息学中，但在强化学习的传统研究中却很少被研究。为此，我们提出了一个新的框架，基于模型的强化学习代理的行动和意见是异步随机离散事件发生在连续时间。我们通过具有外部干预控制项的Hawkes过程来模拟环境的动态，并开发了一种算法来将这样的过程嵌入到Bellman方程中，该方程指导值梯度的方向。我们证明了我们的方法在合成模拟器和现实世界的问题的优越性。
英文摘要：
We consider a sequential decision making problem where the agent faces the environment characterized by the stochastic discrete events and seeks an optimal intervention policy such that its long-term reward is maximized. This problem exists ubiquitously in social media, finance and health informatics but is rarely investigated by the conventional research in reinforcement learning. To this end, we present a novel framework of the model-based reinforcement learning where the agent's actions and observations are asynchronous stochastic discrete events occurring in continuous-time. We model the dynamics of the environment by Hawkes process with external intervention control term and develop an algorithm to embed such process in the Bellman equation which guides the direction of the value gradient. We demonstrate the superiority of our method in both synthetic simulator and real-world problem.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
