程序化强化学习：在 Gridworlds 中导航
Programmatic Reinforcement Learning: Navigating Gridworlds
AI总结：
本文针对一类 gridworld 环境研究程序化强化学习，定义程序化策略类别，给出最优策略规模上界与综合算法，并通过原型实现验证理论框架。
摘要：
强化学习（RL）领域研究的是在未知随机环境中学习最优策略的算法。程序化 RL 将策略表示为程序，即涉及控制循环等高阶结构。尽管它在机器学习与形式化方法社区的交叉领域吸引了大量关注，但在理论层面对程序化 RL 仍知之甚少：哪些类别的程序化策略是好的？最优程序化策略有多大？我们如何学习它们？本文的目标是首次回答这些问题，从而开启对程序化 RL 的理论研究。针对一类 gridworld 环境，我们定义了一类程序化策略。我们的主要贡献是给出了最优程序化策略规模的上界，并构造了一种综合这些策略的算法。除这些理论发现外，我们还实现了该算法的原型。
英文摘要：
The field of reinforcement learning (RL) is concerned with algorithms for learning optimal policies in unknown stochastic environments. Programmatic RL studies representations of policies as programs, meaning involving higher order constructs such as control loops. Despite attracting a lot of attention at the intersection of the machine learning and formal methods communities, very little is known on the theoretical front about programmatic RL: what are good classes of programmatic policies? How large are optimal programmatic policies? How can we learn them? The goal of this paper is to give first answers to these questions, initiating a theoretical study of programmatic RL. Considering a class of gridworld environments, we define a class of programmatic policies. Our main contributions are to place upper bounds on the size of optimal programmatic policies, and to construct an algorithm for synthesizing them. These theoretical findings are complemented by a prototype implementation of the algorithm.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
