Solving complex real-world tasks, e.g., autonomous fleet control, often involves a coordinated team of multiple agents which learn strategies from visual inputs via reinforcement learning. Many existing multi-agent reinforcement learning (MARL) algorithms however don't scale to environments where agents operate on visual inputs. To address this issue, algorithmically, recent works have focused on non-stationarity and exploration. In contrast, we study whether scalability can also be achieved via a disentangled representation. For this, we explicitly construct an object-centric intermediate representation to characterize the states of an environment, which we refer to as `semantic tracklets.' We evaluate `semantic tracklets' on the visual multi-agent particle environment (VMPE) and on the challenging visual multi-agent GFootball environment. `Semantic tracklets' consistently outperform baselines on VMPE, and achieve a +2.4 higher score difference than baselines on GFootball. Notably, this method is the first to successfully learn a strategy for five players in the GFootball environment using only visual data.
Object-centric representations have recently enabled significant progress in tackling relational reasoning tasks. By building a strong object-centric inductive bias into neural architectures, recent efforts have improved generalization and data efficiency of machine learning algorithms for these problems. One problem class involving relational reasoning that still remains under-explored is multi-agent reinforcement learning (MARL). Here we investigate whether object-centric representations are also beneficial in the fully cooperative MARL setting. Specifically, we study two ways of incorporati…
Human-centric Reward Optimization for Reinforcement Learning-based Automated Driving using Large Language Models
Faculty of Applied Science & Engineering, University of Toronto（多伦多大学应用科学与工程学院）；University of Minnesota – Twin Cities（明尼苏达大学双城分校）；University of Wisconsin – Madison（威斯康星大学麦迪逊分校）；Transportation Research Institute, University of Toronto（多伦多大学交通研究所）；IRC for Smart Mobility & Logistics, King Fahd University of Petroleum and Minerals（法赫德石油和矿产大学智能移动与物流研究所）
Deep Reinforcement Learning via Object-Centric Attention
Technical University of Darmstadt（达姆施塔特工业大学）；Hessian Center for Artificial Intelligence (hessian.AI)（黑森人工智能中心）；Sorbonne Université（索邦大学）；German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心）；Centre for Cognitive Science, Darmstadt, Germany（达姆施塔特认知科学中心）
Rutgers University（罗格斯大学）；Electronics and Telecommunications Research Institute（电子与电信研究所）；Rutgers Center for Cognitive Science（罗格斯认知科学中心）
Object-centric world models provide structured representation of the scene and can be an important backbone in reinforcement learning and planning. However, existing approaches suffer in partially-observable environments due to the lack of belief states. In this paper, we propose Structured World Belief, a model for learning and inference of object-centric belief states. Inferred by Sequential Monte Carlo (SMC), our belief states provide multiple object-centric scene hypotheses. To synergize the benefits of SMC particles with object representations, we also propose a new object-centric dynamic…
Technical University of Darmstadt（达姆施塔特工业大学）；Hessian Center for Artificial Intelligence (hessian.AI)（黑森人工智能中心）；Centre for Cognitive Science of Darmstadt（达姆施塔特认知科学中心）；German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心）
Computer Science Department, TU Darmstadt（图宾根大学计算机科学系）；National Research Center for Applied Cybersecurity Darmstadt（德意志应用网络安全国家研究中心）；Hessian Center for Artificial Intelligence (hessian.AI)（黑森人工智能中心）；Centre for Cognitive Science, TU Darmstadt（图宾根大学认知科学中心）；German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心）
Towards Data-Centric AI: A Comprehensive Survey of Traditional, Reinforcement, and Generative Approaches for Tabular Data Transformation
University of Kansas（堪萨斯大学）；Southwestern University of Finance and Economics（西南财经大学）；Arizona State University（亚利桑那州立大学）；Portland State University（波特兰州立大学）；Computer Network Information Center, Chinese Academy of Sciences（中国科学院计算机网络信息中心）；University of Macau（澳门大学）；The Hong Kong University of Science and Technology（香港科技大学）
Human-Centric Aware UAV Trajectory Planning in Search and Rescue Missions Employing Multi-Objective Reinforcement Learning with AHP and Similarity-Based Experience Replay
Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg（卢森堡大学安全、可靠性与信任跨学科中心）；University of Luxembourg（卢森堡大学）
TalkToAgent: A Human-centric Explanation of Reinforcement Learning Agents with Large Language Models
Department of Chemical and Biological Engineering, Seoul National University, Republic of Korea（化学与生物工程系，首尔国立大学）；Davidson School of Chemical Engineering, Purdue University, United States（化学工程大卫森学院，普渡大学）
Indraprastha Institute of Information Technology（英迪拉普拉斯信息技术学院）；Delhi Technological University（德里技术大学）
Smart homes require every device inside them to be connected with each other at all times, which leads to a lot of power wastage on a daily basis. As the devices inside a smart home increase, it becomes difficult for the user to control or operate every individual device optimally. Therefore, users generally rely on power management systems for such optimization but often are not satisfied with the results. In this paper, we present a novel multi-objective reinforcement learning framework with two-fold objectives of minimizing power consumption and maximizing user satisfaction. The framework e…
A Survey of Reinforcement Learning for Large Language Models under Data Scarcity: Challenges and Solutions
Peking University（北京大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；National University of Singapore（新加坡国立大学）；Nankai University（南开大学）；Tongji University（同济大学）；Nanjing University（南京大学）；University of Wollongong（沃林根大学）；Shanghai Jiao Tong University（上海交通大学）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）
Google Research（谷歌研究院）；Berkeley AI Research（伯克利人工智能研究院）；UC Berkeley（加州大学伯克利分校）
Reinforcement learning can train policies that effectively perform complex tasks. However for long-horizon tasks, the performance of these methods degrades with horizon, often necessitating reasoning over and chaining lower-level skills. Hierarchical reinforcement learning aims to enable this by providing a bank of low-level skills as action abstractions. Hierarchies can further improve on this by abstracting the space states as well. We posit that a suitable state abstraction should depend on the capabilities of the available lower-level policies. We propose Value Function Spaces: a simple ap…
Representing a scene and its constituent objects from raw sensory data is a core ability for enabling robots to interact with their environment. In this paper, we propose a novel approach for scene understanding, leveraging a hierarchical object-centric generative model that enables an agent to infer object category and pose in an allocentric reference frame using active inference, a neuro-inspired framework for action and perception. For evaluating the behavior of an active vision agent, we also propose a new benchmark where, given a target viewpoint of a particular object, the agent needs to…
OpenRL: A Unified Reinforcement Learning Framework
Paradigm Inc.（第四范式）；Robotics Institute, Carnegie Mellon University（卡内基梅隆大学机器人研究所）；School of Data Science, Fudan University（复旦大学大数据学院）；Beijing University of Posts and Telecommunications（北京邮电大学）
Manipulation tasks can often be decomposed into multiple subtasks performed in parallel, e.g., sliding an object to a goal pose while maintaining contact with a table. Individual subtasks can be achieved by task-axis controllers defined relative to the objects being manipulated, and a set of object-centric controllers can be combined in an hierarchy. In prior works, such combinations are defined manually or learned from demonstrations. By contrast, we propose using reinforcement learning to dynamically compose hierarchical object-centric controllers for manipulation tasks. Experiments in both …
The training of a next-best-view (NBV) planner for visual place recognition (VPR) is a fundamentally important task in autonomous robot navigation, for which a typical approach is the use of visual experiences that are collected in the target domain as training data. However, the collection of a wide variety of visual experiences in everyday navigation is costly and prohibitive for real-time robotic applications. We address this issue by employing a novel {\it domain-invariant} NBV planner. A standard VPR subsystem based on a convolutional neural network (CNN) is assumed to be available, and i…
Discovering Object-Centric Generalized Value Functions From Pixels
École de technologie supérieure（高等技术学院）；Mila-Quebec AI Institute（米拉-魁北克人工智能研究所）；Université de Montréal（蒙特利尔大学）；McGill University（麦吉尔大学）；CIFAR AI Chair（加拿大高等研究院人工智能讲席）；DeepMind（深度思维）
University of Michigan（密歇根大学）；UC Berkeley（加州大学伯克利分校）
First-person object-interaction tasks in high-fidelity, 3D, simulated environments such as the AI2Thor virtual home-environment pose significant sample-efficiency challenges for reinforcement learning (RL) agents learning from sparse task rewards. To alleviate these challenges, prior work has provided extensive supervision via a combination of reward-shaping, ground-truth object-information, and expert demonstrations. In this work, we show that one can learn object-interaction tasks from scratch without supervision by learning an attentive object-model as an auxiliary task during task learning…
Symbolic Relational Deep Reinforcement Learning based on Graph Neural Networks and Autoregressive Policy Decomposition
Czech Technical University in Prague（布拉格捷克技术大学）；Artificial Intelligence Center, Department of Computer Science, Faculty of Electrical Engineering（电气工程学院计算机科学系人工智能中心）
Dept. of Computer Science University of Minnesota, Twin Cities（计算机科学系明尼苏达大学双城分校）；Depts. of Computer Science and Psychology University of Minnesota, Twin Cities（计算机科学与心理学系明尼苏达大学双城分校）
Reinforcement learning agents must generalize beyond their training experience. Prior work has focused mostly on identical training and evaluation environments. Starting from the recently introduced Crafter benchmark, a 2D open world survival game, we introduce a new set of environments suitable for evaluating some agent's ability to generalize on previously unseen (numbers of) objects and to adapt quickly (meta-learning). In Crafter, the agents are evaluated by the number of unlocked achievements (such as collecting resources) when trained for 1M steps. We show that current agents struggle to…
Slot-MPC: Goal-Conditioned Model Predictive Control with Object-Centric Representations
Autonomous Intelligent Systems（自主智能系统）；Computer Science Institute VI（计算机科学研究所VI）；Intelligent Systems and Robotics（智能系统与机器人）；Center for Robotics（机器人中心）；Lamarr Institute for Machine Learning and Artificial Intelligence（拉马尔人工智能学习与智能研究所）；University of Bonn, Germany（波恩大学，德国）
HuMam: Humanoid Motion Control via End-to-End Deep Reinforcement Learning with Mamba
College of Graduate and Professional Studies, Trine University（特灵大学研究生与专业研究学院）；Department of Civil Engineering, University of Hong Kong（香港大学土木工程系）；Faculty of Engineering and Information Technology, University of Technology Sydney（悉尼大学工程与信息技术学院）；National Key Laboratory of Automotive Chassis Integration and Bionics, Jilin University（吉林大学汽车底盘集成与生物仿生国家重点实验室）；School of Computer Science and Statistics, Trinity College Dublin（都柏林信任学院计算机科学与统计学系）
IBM Research（IBM研究院）；MIT-IBM Watson AI Lab（麻省理工学院-IBM沃森人工智能实验室）
Interactive Fiction (IF) games with real human-written natural language texts provide a new natural evaluation for language understanding techniques. In contrast to previous text games with mostly synthetic texts, IF games pose language understanding challenges on the human-written textual descriptions of diverse and sophisticated game worlds and language generation challenges on the action command generation from less restricted combinatorial space. We take a novel perspective of IF game solving and re-formulate it as Multi-Passage Reading Comprehension (MPRC) tasks. Our approaches utilize th…
