控制理论与POMDPs的交汇：一种混合系统方法
Control Theory Meets POMDPs: A Hybrid Systems Approach
AI总结：
本文提出了一种将控制理论应用于POMDPs的方法，通过混合系统方法确定POMDPs的可达信念空间，并利用Lyapunov函数的子水平集进行过估计，同时提出屏障证书定理以验证安全性和最优性要求。
摘要：
部分可观测马尔可夫决策过程（POMDPs）为人工智能中在不确定性下的序列决策提供建模框架。由于POMDPs中的状态不能直接观测，决策必须基于贝叶斯滤波器（连续信念）的输出。因此，POMDPs通常难以精确求解，研究人员常使用连续信念空间的离散化来近似求解。然而，这些近似解容易产生离散化误差，导致POMDPs在需要安全、最优或性能保证的应用中效果不佳。为克服POMDPs的复杂性挑战，本文引入了控制理论的概念。目标是确定POMDPs的可达信念空间，即在初始信念分布下，给定一组动作和观察，所有可能的演变集合。我们首先将分析POMDPs的问题转化为分析离散时间切换系统的行为。为了估计可达信念空间，我们发现以Lyapunov函数的子水平集形式的上界。此外，为了验证给定POMDPs的安全性和最优性要求，我们提出了一种屏障证书定理，其中我们证明，如果存在一个满足一组不等式并符合POMDPs信念更新方程的屏障证书，则安全性和最优性属性将得到保证。在两种情况下，我们展示了如何将计算分解为可并行解决的较小问题。我们所提出的条件可以作为一组求和平方程序进行计算。我们通过解决主动广告调度和机器教学中的两个问题来展示我们方法的适用性。
英文摘要：
Partially observable Markov decision processes (POMDPs) provide a modeling framework for a variety of sequential decision making under uncertainty scenarios in artificial intelligence (AI). Since the states are not directly observable in a POMDP, decision making has to be performed based on the output of a Bayesian filter (continuous beliefs). Hence, POMDPs are often computationally intractable to solve exactly and researchers resort to approximate methods often using discretizations of the continuous belief space. These approximate solutions are, however, prone to discretization errors, which has made POMDPs ineffective in applications, wherein guarantees for safety, optimality, or performance are required. To overcome the complexity challenge of POMDPs, we apply notions from control theory. The goal is to determine the reachable belief space of a POMDP, that is, the set of all possible evolutions given an initial belief distribution over the states and a set of actions and observations. We begin by casting the problem of analyzing a POMDP into analyzing the behavior of a discrete-time switched system. For estimating the reachable belief space, we find over-approximations in terms of sub-level sets of Lyapunov functions. Furthermore, in order to verify safety and optimality requirements of a given POMDP, we formulate a barrier certificate theorem, wherein we show that if there exists a barrier certificate satisfying a set of inequalities along with the belief update equation of the POMDP, the safety and optimality properties are guaranteed to hold. In both cases, we show how the calculations can be decomposed into smaller problems that can be solved in parallel. The conditions we formulate can be computationally implemented as a set of sum-of-squares programs. We illustrate the applicability of our method by addressing two problems in active ad scheduling and machine teaching.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
