arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.12830cs.GTcs.MA
收藏

在随机博弈中通过单比特反馈实现去中心化最优均衡学习

Decentralized Optimal Equilibrium Learning in Stochastic Games via Single-bit Feedback

Bilkent University(比尔肯特大学)

Seref Taha Kiremitci, Ahmed Said Donmez, Muhammed O. Sayin

AI总结:

本文提出了一种在随机博弈中通过单比特反馈实现去中心化最优均衡学习的方法,通过探索-承诺和在线变体,实现了对社会福利目标的优化,并建立了有限时间遗憾保证。

摘要:

我们研究了在严重的信息和通信限制下,随机博弈中的去中心化均衡选择。在这种情况下,仅收敛到均衡是不够的,因为随机博弈通常允许多个具有显著不同福利属性的均衡。我们研究了去中心化最优均衡选择,其中代理协调在优化设计指定的社会福利目标的均衡上,同时允许对偏离严格最佳反应的容忍度不同。代理只能观察到全局状态轨迹和他们实现的奖励,并且每轮每个代理交换一个随机化的反馈比特。这种语义内容/不满信号机制隐含地将去中心化学习动态与全局福利目标对齐。我们开发了探索-承诺和在线变体,适用于一般的随机博弈,可以容纳异质的基于模型或无模型方法来解决诱导的马尔可夫决策过程,并建立了显式的有限时间遗憾保证,显示在温和条件下,期望遗憾为对数级。

英文摘要:

We study decentralized equilibrium selection in stochastic games under severe information and communication constraints. In such settings, convergence to equilibrium alone is insufficient, as stochastic games typically admit many equilibria with markedly different welfare properties. We address decentralized optimal equilibrium selection, where agents coordinate on equilibria that optimize a designer-specified social welfare objective while allowing heterogeneous tolerance to deviations from strict best responses. Agents observe only the global state trajectory and their realized rewards, and exchange a single randomized bit of feedback per agent per round. This semantic content/discontent signaling mechanism implicitly aligns decentralized learning dynamics with the global welfare objective. We develop explore-and-commit and online variants applicable to general stochastic games, accommodating heterogeneous model-based or model-free methods for solving the induced Markov decision processes, and establish explicit finite-time regret guarantees, showing logarithmic expected regret under mild conditions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑