In multi-agent reinforcement learning, discovering successful collective behaviors is challenging as it requires exploring a joint action space that grows exponentially with the number of agents. While the tractability of independent agent-wise exploration is appealing, this approach fails on tasks that require elaborate group strategies. We argue that coordinating the agents' policies can guide their exploration and we investigate techniques to promote such an inductive bias. We propose two policy regularization methods: TeamReg, which is based on inter-agent action predictability and CoachReg that relies on synchronized behavior selection. We evaluate each approach on four challenging continuous control tasks with sparse rewards that require varying levels of coordination as well as on the discrete action Google Research Football environment. Our experiments show improved performance across many cooperative multi-agent problems. Finally, we analyze the effects of our proposed methods on the policies that our agents learn and show that our methods successfully enforce the qualities that we propose as proxies for coordinated behaviors.
Inferring Latent Temporal Sparse Coordination Graph for Multi-Agent Reinforcement Learning
Australian Artificial Intelligence Institute (AAII), Faculty of Engineering and Information Technology, University of Technology Sydney（澳大利亚人工智能研究所（AAII），悉尼科技大学工程与信息技术学院）
Institute of Information Systems, University of Liechtenstein（列支敦士登大学信息系统研究所）
This work investigates fundamental questions related to learning features in convolutional neural networks (CNN). Empirical findings across multiple architectures such as VGG, ResNet, Inception, DenseNet and MobileNet indicate that weights near the center of a filter are larger than weights on the outside. Current regularization schemes violate this principle. Thus, we introduce Locality-promoting Regularization (LOCO-Reg), which yields accuracy gains across multiple architectures and datasets. We also show theoretically that the empirical finding is a consequence of maximizing feature cohesio…
Viability of Future Actions: Robust Safety in Reinforcement Learning via Entropy Regularization
Institute for Data Science in Mechanical Engineering, RWTH Aachen University, Germany（机械工程数据科学研究所，亚琛RWTH大学，德国）；Learning Systems and Robotics Lab, Technical University of Munich, Germany（学习系统与机器人实验室，慕尼黑技术大学，德国）
Multi-Agent Deep Reinforcement Learning for Distributed and Autonomous Platoon Coordination via Speed-regulation over Large-scale Transportation Networks
Shanghai Research Institute for Intelligent Autonomous Systems, Tongji University（同济大学上海智能自主系统研究院）；Tongji University（同济大学）
DiRe: Diversity-promoting Regularization for Dataset Condensation
Department of Artificial Intelligence, Indian Institute of Technology Hyderabad（人工智能系，印度理工学院海得拉巴）；Centre for Responsible AI, Wadhwani School of Data Science & AI, Indian Institute of Technology Madras（负责任人工智能中心，Wadhwani数据科学与人工智能学院，印度理工学院马德拉斯）
Robust Convolution Neural ODEs via Contractivity-promoting regularization
Control & Automation Group, Inspire AG（控制与自动化组，Inspire AG）；Automatic Control Laboratory (IfA), ETH Zürich（自动控制实验室（IfA），ETH苏黎世）；School of Future Technology, Shanghai University（未来技术学院，上海大学）；Hangzhou International Innovation Institute, Beihang University（杭州国际创新院，北航）；Laboratoire d’Automatique, EPFL（自动化实验室，EPFL）
A major challenge in multi-agent systems is that the system complexity grows dramatically with the number of agents as well as the size of their action spaces, which is typical in real world scenarios such as autonomous vehicles, robotic teams, network routing, etc. It is hence in imminent need to design decentralized or independent algorithms where the update of each agent is only based on their local observations without the need of introducing complex communication/coordination mechanisms. In this work, we study the finite-time convergence of independent entropy-regularized natural policy g…
The de-facto standard approach of promoting sparsity by means of $\ell_1$-regularization becomes ineffective in the presence of simplex constraints, i.e.,~the target is known to have non-negative entries summing up to a given constant. The situation is analogous for the use of nuclear norm regularization for low-rank recovery of Hermitian positive semidefinite matrices with given trace. In the present paper, we discuss several strategies to deal with this situation, from simple to more complex. As a starting point, we consider empirical risk minimization (ERM). It follows from existing theory …
We present a method for training a deep neural network containing sinusoidal activation functions to fit to time-series data. Weights are initialized using a fast Fourier transform, then trained with regularization to improve generalization. A simple dynamic parameter tuning method is employed to adjust both the learning rate and regularization term, such that stability and efficient training are both achieved. We show how deeper layers can be utilized to model the observed sequence using a sparser set of sinusoid units, and how non-uniform regularization can improve generalization by promotin…
A non-smooth regularization framework for learning over multitask graphs
Université Côte d’Azur, I3S Laboratory, CNRS, France（法国大学皮特里特大学I3S实验室，CNRS）；Machine Learning Genoa Center (MaLGa), Department of Computer Science, University of Genoa（热那亚大学计算机科学系机器学习中心（MaLGa））
We consider the problem of making AI agents that collaborate well with humans in partially observable fully cooperative environments given datasets of human behavior. Inspired by piKL, a human-data-regularized search method that improves upon a behavioral cloning policy without diverging far away from it, we develop a three-step algorithm that achieve strong performance in coordinating with real humans in the Hanabi benchmark. We first use a regularized search algorithm and behavioral cloning to produce a better human model that captures diverse skill levels. Then, we integrate the policy regu…
Guided Deep Generative Model-based Spatial Regularization for Multiband Imaging Inverse Problems
Northwestern Polytechnical University（西北工业大学）；University of Toulouse（图卢兹大学）；IRIT/INPENSEEIHT（图卢兹计算机研究所/国立图卢兹应用科学学院国立高等电力技术、电子学、计算机、水力学与电信学院）；CNRS（法国国家科学研究中心）；Institut Universitaire de France (IUF)（法国大学研究院）
High dimensional regression benefits from sparsity promoting regularizations. Screening rules leverage the known sparsity of the solution by ignoring some variables in the optimization, hence speeding up solvers. When the procedure is proven not to discard features wrongly the rules are said to be \emph{safe}. In this paper we derive new safe rules for generalized linear models regularized with $\ell_1$ and $\ell_1/\ell_2$ norms. The rules are based on duality gap computations and spherical safe regions whose diameters converge to zero. This allows to discard safely more variables, in particul…
CoordLight: Learning Decentralized Coordination for Network-Wide Traffic Signal Control
Department of Mechanical Engineering, National University of Singapore（新加坡国立大学机械工程系）；Chandra Department of Electrical and Computer Engineering, The University of Texas at Austin（德克萨斯大学奥斯汀分校电子与计算机工程系）；Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology（麻省理工学院电子工程与计算机科学系）
本文提出CoordLight框架，通过改进单个交叉口决策和与邻近代理的协调，提升网络级交通优化。引入Queue Dynamic State Encoding和Neighbor-aware Policy Optimization算法，实现更高效的交通信号控制。
Norwegian University of Science and Technology（挪威科技大学）；SINTEF Digital（SINTEF数字研究所）
Deep neural networks have become very popular in modeling complex nonlinear processes due to their extraordinary ability to fit arbitrary nonlinear functions from data with minimal expert intervention. However, they are almost always overparameterized and challenging to interpret due to their internal complexity. Furthermore, the optimization process to find the learned model parameters can be unstable due to the process getting stuck in local minima. In this work, we demonstrate the value of sparse regularization techniques to significantly reduce the model complexity. We demonstrate this for…
