通过2层马尔可夫决策过程学习在团队中的智能体之间进行切换
Learning to Switch Among Agents in a Team via 2-Layer Markov Decision Processes
AI总结：
本文通过2层马尔可夫决策过程定义了团队中智能体控制切换问题，提出一种基于上置信界的在线学习算法，其总后悔值为次线性，并在避障任务仿真中证明优于问题无关算法。
摘要：
强化学习智能体的开发和评估大多基于它们将以完全自主的方式运行——即采取所有行动——的假设。在本研究中，我们的目标是开发算法，通过学习在智能体之间切换控制，使现有的强化学习智能体能够在不同的自动化水平下运行。为此，我们首先通过2层马尔可夫决策过程正式定义了在团队中的智能体之间学习切换控制的问题。然后，我们开发了一种在线学习算法，该算法利用智能体策略和环境转移概率的上置信界来寻找一系列切换策略。我们的算法相对于最优切换策略的总后悔值在学习步数上是次线性的，并且当多个智能体团队在相似环境中运行时，我们的算法通过为环境转移概率维护共享置信界而大大受益，并享有比问题无关算法更好的后悔界。在避障任务中的仿真实验说明了我们的理论发现，并证明通过利用问题的特定结构，我们提出的算法优于问题无关算法。
英文摘要：
Reinforcement learning agents have been mostly developed and evaluated under the assumption that they will operate in a fully autonomous manner -- they will take all actions. In this work, our goal is to develop algorithms that, by learning to switch control between agents, allow existing reinforcement learning agents to operate under different automation levels. To this end, we first formally define the problem of learning to switch control among agents in a team via a 2-layer Markov decision process. Then, we develop an online learning algorithm that uses upper confidence bounds on the agents' policies and the environment's transition probabilities to find a sequence of switching policies. The total regret of our algorithm with respect to the optimal switching policy is sublinear in the number of learning steps and, whenever multiple teams of agents operate in a similar environment, our algorithm greatly benefits from maintaining shared confidence bounds for the environments' transition probabilities and it enjoys a better regret bound than problem-agnostic algorithms. Simulation experiments in an obstacle avoidance task illustrate our theoretical findings and demonstrate that, by exploiting the specific structure of the problem, our proposed algorithm is superior to problem-agnostic algorithms.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
