首页/论文/2001.00602

arXiv 2001.006022020-01cs.LGmath.OCstat.ML

Accelerating Smooth Games by Manipulating Spectral Shapes

SAIT AI Lab, Montreal（SAIT人工智能实验室）

Waïss Azizian, Damien Scieur, Ioannis Mitliagkas, Simon Lacoste-Julien, Gauthier Gidel

备注：: Appears in: Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics (AISTATS 2020). 34 pages

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

英文摘要相关论文

英文摘要：

We use matrix iteration theory to characterize acceleration in smooth games. We define the spectral shape of a family of games as the set containing all eigenvalues of the Jacobians of standard gradient dynamics in the family. Shapes restricted to the real line represent well-understood classes of problems, like minimization. Shapes spanning the complex plane capture the added numerical challenges in solving smooth games. In this framework, we describe gradient-based methods, such as extragradient, as transformations on the spectral shape. Using this perspective, we propose an optimal algorithm for bilinear games. For smooth and strongly monotone operators, we identify a continuum between convex minimization, where acceleration is possible using Polyak's momentum, and the worst case where gradient descent is optimal. Finally, going beyond first-order methods, we propose an accelerated version of consensus optimization.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

Manipulating the Distributions of Experience used for Self-Play Learning in Expert Iteration

Maastricht University（马斯特里赫特大学）

Expert Iteration (ExIt) is an effective framework for learning game-playing policies from self-play. ExIt involves training a policy to mimic the search behaviour of a tree search algorithm - such as Monte-Carlo tree search - and using the trained policy to guide it. The policy and the tree search can then iteratively improve each other, through experience gathered in self-play between instances of the guided tree search algorithm. This paper outlines three different approaches for manipulating the distribution of data collected from self-play, and the procedure that samples batches for learni…

2020-05-30cs.AIcs.LGstat.ML

具有外部性的策略分类

Strategic Classification With Externalities

Harvard University（哈佛大学）；University of Southern California（南加州大学）

我们提出一种考虑代理间外部性的策略分类模型，建模为Stackelberg博弈，证明纯纳什均衡唯一且可计算，并建立PAC学习保证，为多策略行为者环境下的鲁棒分类器奠定理论基础。

2025-02-26cs.AIcs.GTcs.LG

ScaLA: Accelerating Adaptation of Pre-Trained Transformer-Based Language Models via Efficient Large-Batch Adversarial Noise

Microsoft（微软）

In recent years, large pre-trained Transformer-based language models have led to dramatic improvements in many natural language understanding tasks. To train these models with increasing sizes, many neural network practitioners attempt to increase the batch sizes in order to leverage multiple GPUs to improve training speed. However, increasing the batch size often makes the optimization more difficult, leading to slow convergence or poor generalization that can require orders of magnitude more training time to achieve the same model quality. In this paper, we explore the steepness of the loss …

2022-01-29cs.CLcs.LG

数据增强游戏起始用于加速不完全信息游戏的自我对战探索

Data-Augmented Game Starts for Accelerating Self-Play Exploration in Imperfect Information Games

本文提出数据增强游戏起始策略，通过离线数据采样加速策略梯度方法在不完全信息游戏中的探索，降低可利用性并提高策略效果。

2026-05-14cs.AIcs.GTcs.LG

学习带赌博反馈的双人零和博弈的实例相关遗憾界

Instance-Dependent Regret Bounds for Learning Two-Player Zero-Sum Games with Bandit Feedback

The University of Tokyo（东京大学）；RIKEN AIP（理化学研究所人工智能项目）；University of Southern California（南加利福尼亚大学）

本文证明在仅带赌博反馈的双人零和正规形式博弈中，双方采用 Tsallis-INF 算法可实现实例相关的遗憾界，其中常数依赖于博弈间隙和纳什均衡结构，纯策略均衡时达到最优界并支持最后迭代收敛与高效均衡识别。

2025-02-24cs.GTcs.LG

通过基于反事实价值的虚拟博弈加速蒙特卡洛环境中的纳什均衡收敛

Accelerating Nash Equilibrium Convergence in Monte Carlo Settings Through Counterfactual Value Based Fictitious Play

School of Artificial Intelligence and Automation, Huazhong University of Science and Technology（华中科技大学人工智能与自动化学院）

针对多数CFR改进无法用于蒙特卡洛条件的问题，提出MCCFVFP，融合反事实价值与虚拟博弈最优响应，在高比例被占优策略博弈中收敛更快。

2024-10-27cs.AIcs.GTcs.LG

更多相关论文收起更多论文

Accelerating Training in Pommerman with Imitation and Reinforcement Learning

TCS Research（塔塔咨询服务公司研究院）

The Pommerman simulation was recently developed to mimic the classic Japanese game Bomberman, and focuses on competitive gameplay in a multi-agent setting. We focus on the 2$\times$2 team version of Pommerman, developed for a competition at NeurIPS 2018. Our methodology involves training an agent initially through imitation learning on a noisy expert policy, followed by a proximal-policy optimization (PPO) reinforcement learning algorithm. The basic PPO approach is modified for stable transition from the imitation learning phase through reward shaping, action filters based on heuristics, and c…

2019-11-13cs.LGstat.ML

Manipulating a Learning Defender and Ways to Counteract

University of Oxford（牛津大学）；Nanyang Technological University（南洋理工大学）；University of Southampton（南安普顿大学）

In Stackelberg security games when information about the attacker's payoffs is uncertain, algorithms have been proposed to learn the optimal defender commitment by interacting with the attacker and observing their best responses. In this paper, we show that, however, these algorithms can be easily manipulated if the attacker responds untruthfully. As a key finding, attacker manipulation normally leads to the defender learning a maximin strategy, which effectively renders the learning attempt meaningless as to compute a maximin strategy requires no additional information about the other player …

2019-10-31cs.GTcs.LG

Accelerating Deep Neuroevolution on Distributed FPGAs for Reinforcement Learning Problems

IBM Almaden Research Center（IBM阿尔马登研究中心）

Reinforcement learning augmented by the representational power of deep neural networks, has shown promising results on high-dimensional problems, such as game playing and robotic control. However, the sequential nature of these problems poses a fundamental challenge for computational efficiency. Recently, alternative approaches such as evolutionary strategies and deep neuroevolution demonstrated competitive results with faster training time on distributed CPU cores. Here, we report record training times (running at about 1 million frames per second) for Atari 2600 games using deep neuroevoluti…

2020-05-10cs.AIcs.ARcs.DC

Spending Thinking Time Wisely: Accelerating MCTS with Virtual Expansions

Tsinghua University（清华大学）；UC Berkeley（加州大学伯克利分校）；Shanghai Qi Zhi Institute（上海期智研究院）

One of the most important AI research questions is to trade off computation versus performance since ``perfect rationality" exists in theory but is impossible to achieve in practice. Recently, Monte-Carlo tree search (MCTS) has attracted considerable attention due to the significant performance improvement in various challenging domains. However, the expensive time cost during search severely restricts its scope for applications. This paper proposes the Virtual MCTS (V-MCTS), a variant of MCTS that spends more search time on harder states and less search time on simpler states adaptively. We g…

2022-10-23cs.AIcs.LG

Accelerating Reinforcement Learning Agent with EEG-based Implicit Human Feedback

Georgia Institute of Technology（佐治亚理工学院）

Providing Reinforcement Learning (RL) agents with human feedback can dramatically improve various aspects of learning. However, previous methods require human observer to give inputs explicitly (e.g., press buttons, voice interface), burdening the human in the loop of RL agent's learning process. Further, it is sometimes difficult or impossible to obtain the explicit human advise (feedback), e.g., autonomous driving, disabled rehabilitation, etc. In this work, we investigate capturing human's intrinsic reactions as implicit (and natural) feedback through EEG in the form of error-related potent…

2020-10-14cs.LGcs.NEeess.SP

DiffIM：基于代理建模与连续松弛的可微影响最小化

DiffIM: Differentiable Influence Minimization with Surrogate Modeling and Continuous Relaxation

KAIST（韩国科学技术院）

提出DiffIM方法，通过代理建模、连续松弛和梯度驱动选择三种加速方案解决影响最小化问题，在保持或提升有效性的同时实现最高15,160倍的速度提升。

2025-02-03cs.LGcs.SI