通过控制理论在游戏中进行在线优化：连接遗憾，被动性和庞加莱递归
Online Optimization in Games via Control Theory: Connecting Regret, Passivity and Poincaré Recurrence
摘要：
我们提出了一个新的控制理论的在线优化和游戏中的学习的理解，通过被动的概念。无源性是控制理论中的一个基本概念，它抽象了物理系统中的能量守恒和耗散。它已成为分析一般反馈系统的标准工具，博弈动力学属于这一类型。我们的出发点是要表明，所有连续时间跟随正则化领导者（FTRL）动态，其中包括著名的复制动态，是无损的，即它是被动的，没有能量耗散。有趣的是，我们证明了被动意味着有界遗憾，连接两个基本的控制理论和在线优化的原语。FTRL中能量守恒的观察激励我们提出一个无损学习动力学家族，每个家族都有一个简单梯度结构的潜在能量函数。该族在凸组合下是封闭的;作为直接推论，FTRL动力学的任何凸组合是无损的，因此具有有界遗憾。这使我们能够扩展Fox和Shamma [Games，2013]的框架，不仅证明游戏动态的全局渐近稳定性结果，而且还证明了Poincaré递归结果。直觉上，当无损游戏（例如图形常和游戏）与无损学习动态耦合时，它们的反馈互连也是无损的，这导致了类似于能量保持的递归行为，推广了Piliouras和Shamma [SODA，2014]以及Mertikopoulos，Papadimitriou和Piliouras [SODA，2018]的结果。
英文摘要：
We present a novel control-theoretic understanding of online optimization and learning in games, via the notion of passivity. Passivity is a fundamental concept in control theory, which abstracts energy conservation and dissipation in physical systems. It has become a standard tool in analysis of general feedback systems, to which game dynamics belong. Our starting point is to show that all continuous-time Follow-the-Regularized-Leader (FTRL) dynamics, which include the well-known Replicator Dynamic, are lossless, i.e. it is passive with no energy dissipation. Interestingly, we prove that passivity implies bounded regret, connecting two fundamental primitives of control theory and online optimization. The observation of energy conservation in FTRL inspires us to present a family of lossless learning dynamics, each of which has an underlying energy function with a simple gradient structure. This family is closed under convex combination; as an immediate corollary, any convex combination of FTRL dynamics is lossless and thus has bounded regret. This allows us to extend the framework of Fox and Shamma [Games, 2013] to prove not just global asymptotic stability results for game dynamics, but Poincaré recurrence results as well. Intuitively, when a lossless game (e.g. graphical constant-sum game) is coupled with lossless learning dynamics, their feedback interconnection is also lossless, which results in a pendulum-like energy-preserving recurrent behavior, generalizing the results of Piliouras and Shamma [SODA, 2014] and Mertikopoulos, Papadimitriou and Piliouras [SODA, 2018].
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
