A Practical Guide to Studying Emergent Communication through Grounded Language Games
Vrije Universiteit Brussel（布鲁塞尔自由大学）
Jens Nevens, Paul Van Eecke, Katrien Beuls
备注：
This paper was officially published at the 'Language Learning for Artificial Agents (L2A2) Symposium' of the 2019 Artificial Intelligence and Simulation of Behaviour (AISB) Convention
The question of how an effective and efficient communication system can emerge in a population of agents that need to solve a particular task attracts more and more attention from researchers in many fields, including artificial intelligence, linguistics and statistical physics. A common methodology for studying this question consists of carrying out multi-agent experiments in which a population of agents takes part in a series of scripted and task-oriented communicative interactions, called 'language games'. While each individual language game is typically played by two agents in the population, a large series of games allows the population to converge on a shared communication system. Setting up an experiment in which a rich system for communicating about the real world emerges is a major enterprise, as it requires a variety of software components for running multi-agent experiments, for interacting with sensors and actuators, for conceptualising and interpreting semantic structures, and for mapping between these semantic structures and linguistic utterances. The aim of this paper is twofold. On the one hand, it introduces a high-level robot interface that extends the Babel software system, presenting for the first time a toolkit that provides flexible modules for dealing with each subtask involved in running advanced grounded language game experiments. On the other hand, it provides a practical guide to using the toolkit for implementing such experiments, taking a grounded colour naming game experiment as a didactic example.
Multi-agent reinforcement learning offers a way to study how communication could emerge in communities of agents needing to solve specific problems. In this paper, we study the emergence of communication in the negotiation environment, a semi-cooperative model of agent interaction. We introduce two communication protocols -- one grounded in the semantics of the game, and one which is \textit{a priori} ungrounded and is a form of cheap talk. We show that self-interested agents can use the pre-grounded communication channel to negotiate fairly, but are unable to effectively use the ungrounded ch…
Several recent works have found the emergence of grounded compositional language in the communication protocols developed by mostly cooperative multi-agent systems when learned end-to-end to maximize performance on a downstream task. However, human populations learn to solve complex tasks involving communicative behaviors not only in fully cooperative settings but also in scenarios where competition acts as an additional external pressure for improvement. In this work, we investigate whether competition for performance from an external, similar agent team could act as a social influence that e…
New York University（纽约大学）；Facebook AI Research（脸书人工智能研究院）；CIFAR Azrieli Global Scholar（加拿大高级研究院阿兹列利全球学者）
While most machine translation systems to date are trained on large parallel corpora, humans learn language in a different way: by being grounded in an environment and interacting with other humans. In this work, we propose a communication game where two agents, native speakers of their own respective languages, jointly learn to solve a visual referential task. We find that the ability to understand and translate a foreign language emerges as a means to achieve shared goals. The emergent translation is interactive and multimodal, and crucially does not require parallel corpora, but only monoli…
We study the emergence of cooperative behaviors in reinforcement learning agents by introducing a challenging competitive multi-agent soccer environment with continuous simulated physics. We demonstrate that decentralized, population-based training with co-play can lead to a progression in agents' behaviors: from random, to simple ball chasing, and finally showing evidence of cooperation. Our study highlights several of the challenges encountered in large scale multi-agent training in continuous control. In particular, we demonstrate that the automatic optimization of simple shaping rewards, n…
Advances in artificial intelligence often stem from the development of new environments that abstract real-world situations into a form where research can be done conveniently. This paper contributes such an environment based on ideas inspired by elementary Microeconomics. Agents learn to produce resources in a spatially complex world, trade them with one another, and consume those that they prefer. We show that the emergent production, consumption, and pricing behaviors respond to environmental conditions in the directions predicted by supply and demand shifts in Microeconomics. We also demon…
Multi-Agent Reinforcement Learning (MARL) methods find optimal policies for agents that operate in the presence of other learning agents. Central to achieving this is how the agents coordinate. One way to coordinate is by learning to communicate with each other. Can the agents develop a language while learning to perform a common task? In this paper, we formulate and study a MARL problem where cooperative agents are connected to each other via a fixed underlying network. These agents can communicate along the edges of this network by exchanging discrete symbols. However, the semantics of these…
SCARCE: Emergent Institutions in Long-Horizon Multi-Agent Coding Systems
University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）；ELLIS Institute Tübingen（蒂宾根 ELLS 研究所）；MPI for Intelligent Systems（马克斯·普朗克智能系统研究所）；Tübingen AI Center（蒂宾根人工智能中心）
University of Amsterdam（阿姆斯特丹大学）；Facebook A.I. Research（脸书人工智能研究院）；Universitat Pompeu Fabra（庞培法布拉大学）
To cooperate with humans effectively, virtual agents need to be able to understand and execute language instructions. A typical setup to achieve this is with a scripted teacher which guides a virtual agent using language instructions. However, such setup has clear limitations in scalability and, more importantly, it is not interactive. Here, we introduce an autonomous agent that uses discrete communication to interactively guide other agents to navigate and act on a simulated environment. The developed communication protocol is trainable, emergent and requires no additional supervision. The em…
Simon Fraser University（西蒙菲莎大学）；IIT Kanpur（坎pur印度理工学院）；UIUC（伊利诺伊大学厄巴纳-香槟分校）
Communication between embodied AI agents has received increasing attention in recent years. Despite its use, it is still unclear whether the learned communication is interpretable and grounded in perception. To study the grounding of emergent forms of communication, we first introduce the collaborative multi-object navigation task CoMON. In this task, an oracle agent has detailed environment information in the form of a map. It communicates with a navigator agent that perceives the environment visually and is tasked to find a sequence of goals. To succeed at the task, effective communication i…
Previous research into agent communication has shown that a pre-trained guide can speed up the learning process of an imitation learning agent. The guide achieves this by providing the agent with discrete messages in an emerged language about how to solve the task. We extend this one-directional communication by a one-bit communication channel from the learner back to the guide: It is able to ask the guide for help, and we limit the guidance by penalizing the learner for these requests. During training, the agent learns to control this gate based on its current observation. We find that the am…
MILA, Université de Montréal（蒙特利尔大学MILA）；Georgia Institute of Technology（佐治亚理工学院）；Facebook AI Research（Facebook人工智能研究院）
We introduce "Talk The Walk", the first large-scale dialogue dataset grounded in action and perception. The task involves two agents (a "guide" and a "tourist") that communicate via natural language in order to achieve a common goal: having the tourist navigate to a given target location. The task and dataset, which are described in detail, are challenging and their full solution is an open problem that we pose to the community. We (i) focus on the task of tourist localization and develop the novel Masked Attention for Spatial Convolutions (MASC) mechanism that allows for grounding tourist utt…
Enhancing Explainable Cardiac Diagnosis with Guide-Grounded Multimodal LLMs
Business AI Lab, College of Technology, National Economics University（商业人工智能实验室，技术学院，越南国家经济大学）；A2I Lab, Phenikaa School of Computing, Phenikaa University（A2I实验室，费尼卡计算机学院，费尼卡大学）；VNPT AI, VNPT Group（VNPT人工智能，VNPT集团）；FPT University（FPT大学）；Department of Pediatrics, Hospital of University Medicine and Pharmacy, Vietnam National University Hanoi（越南河内国家大学医学与药学院儿科学系）
Princeton University（普林斯顿大学）；WeChat AI（微信人工智能）；MIT-IBM Watson AI Lab（麻省理工学院-IBM沃森人工智能实验室）；MIT（麻省理工学院）
The study of language emergence aims to understand how human languages are shaped by perceptual grounding and communicative intent. Computational approaches to emergent communication (EC) predominantly consider referential games in limited domains and analyze the learned protocol within the game framework. As a result, it remains unclear how the emergent languages from these settings connect to natural languages or provide benefits in real-world language processing tasks, where statistical models trained on large text corpora dominate. In this work, we propose a novel way to establish such a l…
University of Amsterdam（阿姆斯特丹大学）；Pompeu Fabra University（庞培法布拉大学）
We converted the recently developed BabyAI grid world platform to a sender/receiver setup in order to test the hypothesis that established deep reinforcement learning techniques are sufficient to incentivize the emergence of a grounded discrete communication protocol between generalized agents. This is in contrast to previous experiments that employed straight-through estimation or specialized inductive biases. Our results show that these can indeed be avoided, by instead providing proper environmental incentives. Moreover, they show that a longer interval between communications incentivized m…
Propagating construction-time knowledge quality into medical question answering: A framework grounded in clinical guidelines
Nanjing Medical University（南京医科大学）；The First Affiliated Hospital of Nanjing Medical University（南京医科大学第一附属医院）；The Friendship Hospital of Ili Kazakh Autonomous Prefecture（伊犁哈萨克自治州友谊医院）；Nanjing University（南京大学）
Using Grounded Theory for Agent Behavior Analysis at Scale
Purdue University（普渡大学）；Stevens Institute of Technology（史蒂文斯理工学院）；Cornell University（康奈尔大学）；University of Texas at El Paso（德克萨斯大学埃尔帕索分校）；Johns Hopkins University（约翰斯·霍普金斯大学）
Emergent communication research often focuses on optimizing task-specific utility as a driver for communication. However, human languages appear to evolve under pressure to efficiently compress meanings into communication signals by optimizing the Information Bottleneck tradeoff between informativeness and complexity. In this work, we study how trading off these three factors -- utility, informativeness, and complexity -- shapes emergent communication, including compared to human communication. To this end, we propose Vector-Quantized Variational Information Bottleneck (VQ-VIB), a method for t…
AD-CARE: A Guideline-grounded, Modality-agnostic LLM Agent for Real-world Alzheimer's Disease Diagnosis with Multi-cohort Assessment, Fairness Analysis, and Reader Study
The Hong Kong Polytechnic University（香港理工大学）；Xuanwu Hospital, Capital Medical University（首都医科大学宣武医院）；Amazon（亚马逊）；Zhejiang University（浙江大学）；Sany AI（三一人工智能）；Yantai Yuhuangding Hospital, Qingdao University（青岛大学烟台毓璜顶医院）；The University of Hong Kong（香港大学）；The Third Affiliated Hospital of Sun Yat-sen University（中山大学附属第三医院）
University of California, Irvine（加州大学欧文分校）；Mila（米拉研究所）；Québec AI Insitute（魁北克人工智能研究所）
Significant advances have been made in artificial systems by using biological systems as a guide. However, there is often little interaction between computational models for emergent communication and biological models of the emergence of language. Many researchers in language origins and emergent communication take compositionality as their primary target for explaining how simple communication systems can become more like natural language. However, there is reason to think that compositionality is the wrong target on the biological side, and so too the wrong target on the machine-learning si…
University of California, Los Angeles（加利福尼亚大学洛杉矶分校）
Cognitive grammar suggests that the acquisition of language grammar is grounded within visual structures. While grammar is an essential representation of natural language, it also exists ubiquitously in vision to represent the hierarchical part-whole structure. In this work, we study grounded grammar induction of vision and language in a joint learning framework. Specifically, we present VLGrammar, a method that uses compound probabilistic context-free grammars (compound PCFGs) to induce the language grammar and the image grammar simultaneously. We propose a novel contrastive learning framewor…
RetiBridge: Bridging Quantitative Retinal Biomarkers and Qualitative Diagnosis with a Knowledge-Guided Multimodal Large Language Model
University of Liverpool（利物浦大学）；Institute of Life Course & Medical Sciences（生命课程与医学科学研究院）；Department of Eye and Vision Sciences（眼科与视觉科学系）；Computer Science Department（计算机科学系）；Cardiovascular & Metabolic Medicine（心血管与代谢医学）；Liverpool Centre for Cardiovascular Science（利物浦心血管科学中心）
Cyber Academia-Chemical Engineering (CA-ChemE): A Living Digital Town for Self-Directed Research Evolution and Emergent Scientific Discovery
State Key Laboratory of Heavy Oil Processing, College of Carbon Neutrality Future Technology, China University of Petroleum (Beijing), Beijing 102249, China（石油加工国家重点实验室，碳中和未来技术学院，中国石油大学（北京），北京102249）
Arizona State University（亚利桑那州立大学）；Santa Fe Institute（圣塔菲研究所）
The high-dimesionality, non-linearity and emergent properties of complex systems pose a challenge to identifying general laws in the same manner that has been so successful in simpler physical systems. In Anderson's seminal work on why "more is different" he pointed to how emergent, macroscale patterns break symmetries of the underlying microscale laws. Yet, less recognized is that these large-scale, emergent patterns must also retain some symmetries of the microscale rules. Here we introduce a new, relational macrostate theory (RMT) that defines macrostates in terms of symmetries between two …
SANEmerg: An Emergent Communication Framework for Semantic-aware Agentic AI Networking
1 School of Elect. Inform. \& Commun., Huazhong Univ. of Science \& Technology, China；2 Peng Cheng Laboratory, Shenzhen, China；3 Pazhou Laboratory (Huangpu), Guangzhou, China；4 Department of Electrical；Computer Engineering, the University of Arizona
