关于强化学习、效果处理器和状态单子
On Reinforcement Learning, Effect Handlers, and the State Monad
摘要：
我们研究的代数效果和处理程序作为一种方式来支持决策抽象的功能性程序，而用户可以要求一个学习算法来解决选择，而无需实现底层的选择机制，并通过奖励的方式给出反馈。从最近提出的一些基于选择单子的问题方法[Abadi和Plotkin，LICS 2021]，我们将底层智能表示为一个强化学习算法，该算法被实现为一组处理程序，用于其中一些代数运算，包括选择和奖励。我们展示了我们如何在实践中使用代数运算和处理程序-在程序设计语言EFF -明确分离的学习算法从它的环境，从而允许一个良好的模块化水平。然后，我们将展示如何将宿主语言视为一个具有处理程序的代数演算，这种方式显示了基本的语言特征。最后，我们暗示类型和效果系统如何确保安全属性，同时指出了一些进一步工作的方向。
英文摘要：
We study the algebraic effects and handlers as a way to support decision-making abstractions in functional programs, whereas a user can ask a learning algorithm to resolve choices without implementing the underlying selection mechanism, and give a feedback by way of rewards. Differently from some recently proposed approach to the problem based on the selection monad [Abadi and Plotkin, LICS 2021], we express the underlying intelligence as a reinforcement learning algorithm implemented as a set of handlers for some of these algebraic operations, including those for choices and rewards. We show how we can in practice use algebraic operations and handlers -- as available in the programming language EFF -- to clearly separate the learning algorithm from its environment, thus allowing for a good level of modularity. We then show how the host language can be taken as a lambda-calculus with handlers, this way showing what the essential linguistic features are. We conclude by hinting at how type and effect systems could ensure safety properties, at the same time pointing at some directions for further work.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
