软Q学习的有限时间误差分析：切换系统方法
Finite-Time Error Analysis of Soft Q-Learning: Switching System Approach
AI总结：
本文利用切换系统模型对基于log-sum-exp算子和Boltzmann算子的两类软Q学习算法进行统一的有限时间控制理论分析，推导出新的有限时间误差界，为强化学习有限时间分析提供新框架。
摘要：
软Q学习是Q学习的一种变体，旨在解决熵正则化马尔可夫决策问题，其中智能体目标是最大化熵正则化价值函数。尽管软Q学习在经验上取得了成功，但迄今为止对其理论研究仍然有限。本文旨在对软Q学习算法提供一种新颖且统一的有限时间控制理论分析。我们聚焦于两类软Q学习算法：一类使用log-sum-exp算子，另一类使用Boltzmann算子。通过采用动态切换系统模型，我们为这两种软Q学习算法推导出新颖的有限时间误差界。我们希望，通过建立与切换系统模型的联系，本文的分析能够加深当前对软Q学习的理解，并可能为其他强化学习算法的有限时间分析开辟新的框架。
英文摘要：
Soft Q-learning is a variation of Q-learning designed to solve entropy regularized Markov decision problems where an agent aims to maximize the entropy regularized value function. Despite its empirical success, there have been limited theoretical studies of soft Q-learning to date. This paper aims to offer a novel and unified finite-time, control-theoretic analysis of soft Q-learning algorithms. We focus on two types of soft Q-learning algorithms: one utilizing the log-sum-exp operator and the other employing the Boltzmann operator. By using dynamical switching system models, we derive novel finite-time error bounds for both soft Q-learning algorithms. We hope that our analysis will deepen the current understanding of soft Q-learning by establishing connections with switching system models and may even pave the way for new frameworks in the finite-time analysis of other reinforcement learning algorithms.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
