强化学习中的连续性和连续性
Metrics and continuity in reinforcement learning
摘要：
在强化学习的大多数实际应用中，维护单个状态的直接估计是站不住脚的;在连续状态系统中，这是不可能的。相反，研究人员经常利用状态相似性（无论是显式还是隐式）来构建模型，这些模型可以从有限的样本集中很好地推广。因此，所使用的状态相似性的概念以及它们所诱导的邻域和拓扑结构是至关重要的，因为它将直接影响算法的性能。事实上，一些最近的作品介绍算法假设存在的“行为良好”的邻里，但离开完整的规范，这样的拓扑结构为未来的工作。在本文中，我们介绍了一个统一的形式主义，通过度量的镜头定义这些拓扑结构。我们建立了一个层次之间的这些指标，并证明其理论意义上的马尔可夫决策过程指定的强化学习问题。我们补充我们的理论结果与实证评估显示所考虑的指标之间的差异。
英文摘要：
In most practical applications of reinforcement learning, it is untenable to maintain direct estimates for individual states; in continuous-state systems, it is impossible. Instead, researchers often leverage state similarity (whether explicitly or implicitly) to build models that can generalize well from a limited set of samples. The notion of state similarity used, and the neighbourhoods and topologies they induce, is thus of crucial importance, as it will directly affect the performance of the algorithms. Indeed, a number of recent works introduce algorithms assuming the existence of "well-behaved" neighbourhoods, but leave the full specification of such topologies for future work. In this paper we introduce a unified formalism for defining these topologies through the lens of metrics. We establish a hierarchy amongst these metrics and demonstrate their theoretical implications on the Markov Decision Process specifying the reinforcement learning problem. We complement our theoretical results with empirical evaluations showcasing the differences between the metrics considered.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
