arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.08906cs.GTcs.AImath.OC
收藏

针对Q学习者的策略制定:一种控制论方法

Strategizing against Q-learners: A Control-theoretical Approach

Bilkent University(毕尔肯大学)

Yuksel Arslantas, Ege Yuceel, Muhammed O. Sayin

AI总结:

本文通过控制论方法研究独立Q学习算法在重复正规博弈中易受策略操纵的脆弱性,提出基于量化的随机博弈近似方案,并分析其性能。

摘要:

本文探讨了独立Q学习算法(一种经典且广泛使用的多智能体强化学习方法)在重复进行的正规博弈中易受老练对手策略操纵的脆弱性。我们量化了如果策略老练的智能体知道对手的Q学习算法,它们能在多大程度上利用天真的Q学习者。为此,我们将策略行为者的交互建模为一个随机博弈(其状态包含Q学习者的Q函数估计),仿佛Q学习算法是底层动态系统。我们还提出了一种基于量化的近似方案来处理连续状态空间,并针对两个竞争的策略行为者和单个策略行为者,从分析和数值两方面评估了该方案的性能。

英文摘要:

In this paper, we explore the susceptibility of the independent Q-learning algorithms (a classical and widely used multi-agent reinforcement learning method) to strategic manipulation of sophisticated opponents in normal-form games played repeatedly. We quantify how much strategically sophisticated agents can exploit naive Q-learners if they know the opponents' Q-learning algorithm. To this end, we formulate the strategic actors' interactions as a stochastic game (whose state encompasses Q-function estimates of the Q-learners) as if the Q-learning algorithms are the underlying dynamical system. We also present a quantization-based approximation scheme to tackle the continuum state space and analyze its performance for two competing strategic actors and a single strategic actor both analytically and numerically.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑