模拟引理的最优紧界
An Optimal Tightness Bound for the Simulation Lemma
AI总结：
针对强化学习模拟引理在大折扣因子下因累积概率误差处理不佳而失效的问题，本文单独分析该误差，得到含常数因子的最优紧、对转移误设次线性的价值预测误差界，并推广至层次抽象。
摘要：
我们提出了一个关于模型误设下价值预测误差的界，该界是紧的，包括常数因子在内。这是对强化学习中基础性结果“模拟引理”的直接改进。我们证明，由于对累积概率误差的处理不够优，现有界相当宽松，在折扣因子较大时会变得空洞。通过仔细单独考察这一量，而不是将其作为价值误差的子组成部分，我们推导出一个关于转移函数误设呈次线性的界。随后，我们展示了该技术更广泛的适用性，改进了相关子领域层次抽象中的一个类似界。
英文摘要：
We present a bound for value-prediction error with respect to model misspecification that is tight, including constant factors. This is a direct improvement of the "simulation lemma," a foundational result in reinforcement learning. We demonstrate that existing bounds are quite loose, becoming vacuous for large discount factors, due to the suboptimal treatment of compounding probability errors. By carefully considering this quantity on its own, instead of as a subcomponent of value error, we derive a bound that is sub-linear with respect to transition function misspecification. We then demonstrate broader applicability of this technique, improving a similar bound in the related subfield of hierarchical abstraction.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
