Hindsight and Sequential Rationality of Correlated Play
University of Alberta（阿尔伯塔大学）；Amii（阿尔伯塔机器智能研究所）；Université de Montréal（蒙特利尔大学）；Mila（米拉研究所）；Massachusetts Institute of Technology（麻省理工学院）；DeepMind（深度思维）；Brown University（布朗大学）
Dustin Morrill, Ryan D'Orazio, Reca Sarfati, Marc Lanctot, James R. Wright, Amy Greenwald, Michael Bowling
备注：
Corrected technical report for the paper with the same title in the proceedings of the thirty-fifth AAAI Conference on Artificial Intelligence (AAAI-21), February 2-9, 2021, Virtual. Compared to v5, this version fixes the realized terminal history indicators in the diagram describing MacQueen's counterexample. 27 pages and 16 figures
Driven by recent successes in two-player, zero-sum game solving and playing, artificial intelligence work on games has increasingly focused on algorithms that produce equilibrium-based strategies. However, this approach has been less effective at producing competent players in general-sum games or those with more than two players than in two-player, zero-sum games. An appealing alternative is to consider adaptive algorithms that ensure strong performance in hindsight relative to what could have been achieved with modified behavior. This approach also leads to a game-theoretic analysis, but in the correlated play that arises from joint learning dynamics rather than factored agent behavior at equilibrium. We develop and advocate for this hindsight rationality framing of learning in general sequential decision-making settings. To this end, we re-examine mediated equilibrium and deviation types in extensive-form games, thereby gaining a more complete understanding and resolving past misconceptions. We present a set of examples illustrating the distinct strengths and weaknesses of each type of equilibrium in the literature, and prove that no tractable concept subsumes all others. This line of inquiry culminates in the definition of the deviation and equilibrium classes that correspond to algorithms in the counterfactual regret minimization (CFR) family, relating them to all others in the literature. Examining CFR in greater detail further leads to a new recursive definition of rationality in correlated play that extends sequential rationality in a way that naturally applies to hindsight evaluation.
Efficient Deviation Types and Learning for Hindsight Rationality in Extensive-Form Games
University of Alberta（阿尔伯塔大学）；Alberta Machine Intelligence Institute（阿尔伯塔机器智能研究所）；Université de Montréal（蒙特利尔大学）；Mila（米拉研究所）；DeepMind（深度思维）；Brown University（布朗大学）
Relay Hindsight Experience Replay: Self-Guided Continual Reinforcement Learning for Sequential Object Manipulation Tasks with Sparse Rewards
Institute of Intelligent Machines, Hefei Institutes of Physical Science, Chinese Academy of Sciences（中国科学院合肥物质科学研究院智能机械研究所）；University of Science and Technology of China（中国科学技术大学）
Consensus in Motion: A Case of Dynamic Rationality of Sequential Learning in Probability Aggregation
1 Department of Statistics, Ludwig-Maximilians-Universität München\ 2 School of Computing \& Communication, Lancaster University Leipzig \ 3 Munich Center for Mathematical Philosophy, Ludwig-Maximilians-Universität München Correspondence
Human-Aware Robot Navigation via Reinforcement Learning with Hindsight Experience Replay and Curriculum Learning
The Chinese University of Hong Kong（香港中文大学）；Southern University of Science and Technology（南方科技大学）；Shenzhen Research Institute of the Chinese University of Hong Kong（香港中文大学深圳研究院）
University of Pennsylvania（宾夕法尼亚大学）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Max Planck Institute for Biological Cybernetics（马克斯·普朗克生物控制论研究所）；Google Inc.（谷歌公司）；Korea Advanced Institute of Science and Technology（韩国高等科学技术研究院）；The Hebrew University（希伯来大学）
Learning to Infer User Hidden States for Online Sequential Advertising
Alibaba Group（阿里巴巴集团）；University of Southern California（南加州大学）；University College London（伦敦大学学院）；Shanghai Jiao Tong University（上海交通大学）；Univ. of Chinese Academy of Sciences（中国科学院大学）
Rosenthal（1973）引入了拥塞博弈类，并证明了它们在纯策略下总是存在一个纳什均衡。Fotakis et al.（2005）引入了贪婪策略元组的概念，其中参与者顺序地并且可重复地选择一个策略，该策略是对前参与者选择的策略的最佳反应。前一种解决方案的概念是由对参与者的理性及其常识的强烈假设驱动的，而后者对参与者的行为假设很少。从Fotakis \cite{fotakis10}可以得出，对于树可表示拥塞博弈，贪婪行为导致NE。本文给出了这两个解概念等价的充要条件。这种等同性增强了这些概念作为环境的现实结果的可行性。单调对称博弈出现这种等价的条件是策略集具有树形，或者等价地是一个“扩张并行图”。
Game-theoretic LLM: Agent Workflow for Negotiation Games
Rutgers University（罗格斯大学）；University of California, Santa Barbara（加州大学圣巴巴拉分校）；University of Southern California（南加州大学）；University of South Florida（南佛罗里达大学）；Harvard University（哈佛大学）；Institute of Computing Technology（计算技术研究所）
