In the last decade, there have been significant advances in multi-agent reinforcement learning (MARL) but there are still numerous challenges, such as high sample complexity and slow convergence to stable policies, that need to be overcome before wide-spread deployment is possible. However, many real-world environments already, in practice, deploy sub-optimal or heuristic approaches for generating policies. An interesting question that arises is how to best use such approaches as advisors to help improve reinforcement learning in multi-agent domains. In this paper, we provide a principled framework for incorporating action recommendations from online sub-optimal advisors in multi-agent settings. We describe the problem of ADvising Multiple Intelligent Reinforcement Agents (ADMIRAL) in nonrestrictive general-sum stochastic game environments and present two novel Q-learning based algorithms: ADMIRAL - Decision Making (ADMIRAL-DM) and ADMIRAL - Advisor Evaluation (ADMIRAL-AE), which allow us to improve learning by appropriately incorporating advice from an advisor (ADMIRAL-DM), and evaluate the effectiveness of an advisor (ADMIRAL-AE). We analyze the algorithms theoretically and provide fixed-point guarantees regarding their learning in general-sum stochastic games. Furthermore, extensive experiments illustrate that these algorithms: can be used in a variety of environments, have performances that compare favourably to other related baselines, can scale to large state-action spaces, and are robust to poor advice from advisors.
Conservative Bias in Multi-Teacher Learning: Why Agents Prefer Low-Reward Advisors
School of Computer Science and Engineering, University of New South Wales(新南威尔士大学计算机科学与工程学院);Escuela de Ingeniería, Universidad Central de Chile(智利中央大学工程学院)
Choose Your Agent: Tradeoffs in Adopting AI Advisors, Coaches, and Delegates in Multi-Party Negotiation
Harvard University(哈佛大学);Google DeepMind Canada(谷歌深Mind加拿大);Google DeepMind United States(谷歌深Mind美国);Google DeepMind New York United States(谷歌深Mind纽约美国)
A Unified Multi-Modal Framework for Intelligent Financial Systems: Integrating Reinforcement Learning, High-Frequency Trading, and Game-Theoretic Approaches with Cross-Modal Sentiment Analysis
Henan University, International Eurasia College(河南大学,国际欧亚学院);City University of Hong Kong, College of Business(香港城市大学,商学院);Northeastern University, School of Electronic and Information Engineering(东北大学,电子与信息工程学院)
Peer-to-Peer Energy Trading in Dairy Farms using Multi-Agent Reinforcement Learning
School of Computer Science, College of Science and Engineering, University of Galway(Galway大学计算机科学学院);School of Engineering, College of Science and Engineering, University of Galway(Galway大学工程学院)
Improving interactive reinforcement learning: What makes a good teacher?
University of Hamburg(汉堡大学);Universidad Central de Chile(智利中央大学);Osaka University(大阪大学);National Institute of Information and Communications Technology(信息通信研究机构)
GenAI-powered Multi-Agent Paradigm for Smart Urban Mobility: Opportunities and Challenges for Integrating Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) with Intelligent Transportation Systems
Oak Ridge National Laboratory(橡树岭国家实验室);University of Washington(华盛顿大学);Texas A&M University(德克萨斯农工大学)
Rule-based High-Level Coaching for Goal-Conditioned Reinforcement Learning in Search-and-Rescue UAV Missions Under Limited-Simulation Training
Automation and Robotics Research Group, Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg.(自动化与机器人研究组,安全、可靠性与信任 interdisciplinary centre(SnT),卢森堡大学)