arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.04889cs.GTcs.LGcs.MA
收藏

通过预测其他学习器的行为来最大化多智能体环境中的效用

Maximizing utility in multi-agent environments by anticipating the behavior of other learners

Tel Aviv University(特拉维夫大学);Archimedes AI(阿基米德人工智能公司)

Angelos Assos, Yuval Dagan, Constantinos Daskalakis

AI总结:

研究重复两人博弈中优化器如何预测并引导在线学习器行为;给出对抗 Replicator Dynamics 与 MWU 的高效用算法,并证明一般和博弈下最优响应情形不存在 FPTAS(除非 P=NP)。

摘要:

学习算法常被用于在序贯决策环境中制定决策。在多智能体场景中,每个智能体的决策都会影响其他智能体的效用/损失。因此,如果一个智能体善于预测其他智能体的行为,尤其是它们如何根据截至当时的经验在每一轮做出决策,它就可以尝试在交互的各轮中审慎地制定自己的决策,从而影响其他智能体,使其以最终有利于自身效用的方式行事。本文研究包含两类智能体的重复两人博弈:一类是学习器,采用在线学习算法在每一轮选择策略;另一类是优化器,知道学习器的效用函数及其在线学习算法。优化器希望在考虑学习器行为的同时进行前瞻规划,以最大化自身效用。我们给出两个结果:针对重复零和博弈的正面结果,以及针对重复一般和博弈的负面结果。正面结果是为优化器设计的一种算法,可在面对采用 Replicator Dynamics(复制者动态,即 Multiplicative Weights Update(MWU,乘法权重更新)的连续时间对应形式)的学习器时精确最大化其效用。此外,我们利用该结果给出优化器在离散时间情形下对抗 MWU 的算法,保证优化器的平均效用高于一次博弈的值。负面结果表明,除非 P=NP,否则不存在完全多项式时间近似方案(FPTAS)来最大化优化器面对每轮对历史作出最优响应的学习器时的效用。然而,是否存在可将效用优化至 $o(T)$ 以内的多项式时间算法,仍是一个开放问题。

英文摘要:

Learning algorithms are often used to make decisions in sequential decision-making environments. In multi-agent settings, the decisions of each agent can affect the utilities/losses of the other agents. Therefore, if an agent is good at anticipating the behavior of the other agents, in particular how they will make decisions in each round as a function of their experience that far, it could try to judiciously make its own decisions over the rounds of the interaction so as to influence the other agents to behave in a way that ultimately benefits its own utility. In this paper, we study repeated two-player games involving two types of agents: a learner, which employs an online learning algorithm to choose its strategy in each round; and an optimizer, which knows the learner's utility function and the learner's online learning algorithm. The optimizer wants to plan ahead to maximize its own utility, while taking into account the learner's behavior. We provide two results: a positive result for repeated zero-sum games and a negative result for repeated general-sum games. Our positive result is an algorithm for the optimizer, which exactly maximizes its utility against a learner that plays the Replicator Dynamics -- the continuous-time analogue of Multiplicative Weights Update (MWU). Additionally, we use this result to provide an algorithm for the optimizer against MWU, i.e.~for the discrete-time setting, which guarantees an average utility for the optimizer that is higher than the value of the one-shot game. Our negative result shows that, unless P=NP, there is no Fully Polynomial Time Approximation Scheme (FPTAS) for maximizing the utility of an optimizer against a learner that best-responds to the history in each round. Yet, this still leaves open the question of whether there exists a polynomial-time algorithm that optimizes the utility up to $o(T)$.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑