逼近经近似的函数的不动点
Approximating Fixpoints of Approximated Functions
AI总结：
针对单调非扩张函数的最小不动点逼近难题，提出带阻尼因子的Mann迭代变体方案，可保证收敛至最小不动点，适用于强化学习、概率系统等场景。
摘要：
不动点在计算机科学中无处不在，在处理定量语义与验证问题时，人们通常会研究非负实数域上（高维）函数的最小不动点。本文展示了如何逼近这类函数的最小不动点，重点研究函数无法精确获知、而是由一系列收敛于该函数的近似函数表示的场景。我们聚焦于单调且非扩张的函数，这类函数的不动点不具备唯一性保证，标准不动点迭代方案可能会卡在非最小的不动点上。本文的核心贡献是提出了一种迭代方案——带阻尼因子的Mann迭代的变体，在合适的条件下，该方案被证明能够保证收敛到目标函数的最小不动点。随后我们论证了这些结果适用于马尔可夫决策过程的基于模型的强化学习场景，展示了所提迭代方案的具体应用方式，以及它如何推导得到最优期望收益的收敛性。更广泛地说，我们证明了当目标函数可以在给定概率误差界下被近似时（正如可通过采样探索的简单随机博弈等概率系统的情况），我们的结果几乎必然可以用于迭代得到最小不动点。
英文摘要：
Fixpoints are ubiquitous in computer science and when dealing with quantitative semantics and verification one often considers least fixpoints of (higher-dimensional) functions over the non-negative reals. We show how to approximate the least fixpoint of such functions, focusing on the case in which they are not known precisely, but represented by a sequence of approximating functions that converge to them. We concentrate on monotone and non-expansive functions, for which uniqueness of fixpoints is not guaranteed and standard fixpoint iteration schemes might get stuck at a fixpoint that is not the least. Our main contribution is the identification of an iteration scheme, a variation of Mann iteration with a dampening factor, which, under suitable conditions, is shown to guarantee convergence to the least fixpoint of the function of interest. We then argue that these results are relevant in the context of model-based reinforcement learning for Markov decision processes, showing how the proposed iteration scheme instantiates and allows us to derive convergence to the optimal expected return. More generally, we show that our results can be used to iterate to the least fixpoint almost surely for systems where the function of interest can be approximated with given probabilistic error bounds, as it happens for probabilistic systems, such as simple stochastic games, which can be explored via sampling.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
