语言模型如何“思考”？本文提出了一种称为受限语用说话者的概率认知模型，该模型能够刻画不同变体语言模型的运作。具体而言，我们证明了经人类反馈强化学习（Ouyang et al., 2022）微调的大语言模型体现了一种在概念上类似于心理学家归因于人类的快与慢模型（Kahneman, 2011）的思维模型。我们讨论了人类反馈强化学习作为快与慢思维模型的局限性，并提出了扩展该框架的途径。本质上，我们的研究强调了采用认知概率建模方法对于深入理解、评估和推进语言模型的价值。
英文摘要：
How do language models "think"? This paper formulates a probabilistic cognitive model called the bounded pragmatic speaker, which can characterize the operation of different variations of language models. Specifically, we demonstrate that large language models fine-tuned with reinforcement learning from human feedback (Ouyang et al., 2022) embody a model of thought that conceptually resembles a fast-and-slow model (Kahneman, 2011), which psychologists have attributed to humans. We discuss the limitations of reinforcement learning from human feedback as a fast-and-slow model of thought and propose avenues for expanding this framework. In essence, our research highlights the value of adopting a cognitive probabilistic modeling approach to gain insights into the comprehension, evaluation, and advancement of language models.
We show that explicit pragmatic inference aids in correctly generating and following natural language instructions for complex, sequential tasks. Our pragmatics-enabled models reason about why speakers produce certain instructions, and about how listeners will react upon hearing them. Like previous pragmatic models, we use learned base listener and speaker models to build a pragmatic speaker that uses the base listener to simulate the interpretation of candidate descriptions, and a pragmatic listener that reasons counterfactually about alternative descriptions. We extend these models to tasks …
Bielefeld University（比勒费尔德大学）；University of Potsdam（波茨坦大学）
Zero-shot learning in Language & Vision is the task of correctly labelling (or naming) objects of novel categories. Another strand of work in L&V aims at pragmatically informative rather than ``correct'' object descriptions, e.g. in reference games. We combine these lines of research and model zero-shot reference games, where a speaker needs to successfully refer to a novel object in an image. Inspired by models of "rational speech acts", we extend a neural generator to become a pragmatic speaker reasoning about uncertain object categories. As a result of this reasoning, the generator produces…
Computer Science Division（计算机科学系）；University of California, Berkeley（加州大学伯克利分校）
We present a model for pragmatically describing scenes, in which contrastive behavior results from a combination of inference-driven pragmatics and learned semantics. Like previous learned approaches to language generation, our model uses a simple feature-driven architecture (here a pair of neural "listener" and "speaker" models) to ground language in the world. Like inference-driven approaches to pragmatics, our model actively reasons about listener behavior when selecting utterances. For training, our approach requires only ordinary captions, annotated _without_ demonstration of the pragmati…
Human communication is a collaborative process. Speakers, on top of conveying their own intent, adjust the content and language expressions by taking the listeners into account, including their knowledge background, personalities, and physical capabilities. Towards building AI agents with similar abilities in language communication, we propose Pragmatic Rational Speaker (PRS), a framework extending Rational Speech Act (RSA). The PRS attempts to learn the speaker-listener disparity and adjust the speech accordingly, by adding a light-weighted disparity adjustment layer into working memory on to…
We present a model of pragmatic referring expression interpretation in a grounded communication task (identifying colors from descriptions) that draws upon predictions from two recurrent neural network classifiers, a speaker and a listener, unified by a recursive pragmatic reasoning framework. Experiments show that this combined pragmatic model interprets color descriptions more accurately than the classifiers from which it is built, and that much of this improvement results from combining the speaker and listener perspectives. We observe that pragmatic reasoning helps primarily in the hardest…
Non-literal Understanding of Number Words by Language Models
Department of Linguistics, University of Tübingen（语言学系，图宾根大学）；Department of Computer Science, Stanford University（计算机科学系，斯坦福大学）；University of Washington（华盛顿大学）；Department of Psychology, Stanford University（心理学系，斯坦福大学）；Departments of Psychology & Computer Science, Stanford University（心理学与计算机科学系，斯坦福大学）
To be good conversational partners, natural language processing (NLP) systems should be trained to produce contextually useful utterances. Prior work has investigated training NLP systems with communication-based objectives, where a neural listener stands in as a communication partner. However, these systems commonly suffer from semantic drift where the learned language diverges radically from natural language. We propose a method that uses a population of neural listeners to regularize speaker training. We first show that language drift originates from the poor uncertainty calibration of a ne…
Speakers' referential expressions often depart from communicative ideals in ways that help illuminate the nature of pragmatic language use. Patterns of overmodification, in which a speaker uses a modifier that is redundant given their communicative goal, have proven especially informative in this regard. It seems likely that these patterns are shaped by the environment a speaker is exposed to in complex ways. Unfortunately, systematically manipulating these factors during human language acquisition is impossible. In this paper, we propose to address this limitation by adopting neural networks …
Tracing the ongoing emergence of human-like reasoning in Large Language Models
Departament de Filologia Catalana, Universitat Autònoma de Barcelona（加泰罗尼亚语言系系，巴塞罗那自治大学）；Institut für Psychologie, Humboldt-Universitat zu Berlin（柏林洪堡大学心理学研究所）；Institució Catalana de Recerca i Estudis Avançats (ICREA)（加泰罗尼亚高级研究与高级教育研究所）
Does GPT-4 surpass human performance in linguistic pragmatics?
Institute for Artificial Intelligence Research and Development of Serbia（塞尔维亚人工智能研究与发展研究所）；University of Belgrade, Institute for Philosophy and Social Theory, Digital Society Lab（贝尔格莱德大学哲学与社会理论研究所数字社会实验室）；Complexity Science Hub（复杂性科学中心）；University of Novi Sad, Faculty of Philosophy（诺维萨德大学哲学学院）；University of Kragujevac, Faculty of Engineering（克拉古耶瓦茨大学工程学院）
We improve the informativeness of models for conditional text generation using techniques from computational pragmatics. These techniques formulate language production as a game between speakers and listeners, in which a speaker should generate output text that a listener can use to correctly identify the original input that the text describes. While such approaches are widely used in cognitive science and grounded language learning, they have received less attention for more standard language generation tasks. We consider two pragmatic modeling methods for text generation: one where pragmatic…
A hallmark of human language is the ability to effectively and efficiently convey contextually relevant information. One theory for how humans reason about language is presented in the Rational Speech Acts (RSA) framework, which captures pragmatic phenomena via a process of recursive social reasoning (Goodman & Frank, 2016). However, RSA represents ideal reasoning in an unconstrained setting. We explore the idea that speakers might learn to amortize the cost of RSA computation over time by directly optimizing for successful communication with an internal listener model. In simulations with gro…
Language is not only used to transmit neutral information; we often seek to persuade by arguing in favor of a particular view. Persuasion raises a number of challenges for classical accounts of belief updating, as information cannot be taken at face value. How should listeners account for a speaker's "hidden agenda" when incorporating new information? Here, we extend recent probabilistic models of recursive social reasoning to allow for persuasive goals and show that our model provides a pragmatic account for why weakly favorable arguments may backfire, a phenomenon known as the weak evidence …
MUStReason: A Benchmark for Diagnosing Pragmatic Reasoning in Video-LMs for Multimodal Sarcasm Detection
Max Planck Institute for Informatics, Saarland Informatics Campus（马克斯·普朗克研究所信息学研究所，萨尔兰州信息学校园）；Saarland University（萨尔兰州大学）；The University of Edinburgh（爱丁堡大学）；Samsung AI Center, Cambridge（三星人工智能中心，剑桥）
Stanford University（斯坦福大学）；Institute of High Performance Computing, Agency for Science, Technology and Research（科学技术研究局高性能计算研究所）；National University of Singapore（新加坡国立大学）
Grounding language in contextual information is crucial for fine-grained natural language understanding. One important task that involves grounding contextual modifiers is color generation. Given a reference color "green", and a modifier "bluey", how does one generate a color that could represent "bluey green"? We propose a computational pragmatics model that formulates this color generation task as a recursive game between speakers and listeners. In our model, a pragmatic speaker reasons about the inferences that a listener would make, and thus generates a modified color that is maximally inf…
Stanford University（斯坦福大学）；Harvard University（哈佛大学）；San Jose State University（圣何塞州立大学）
Contextual influences on language often exhibit substantial cross-lingual regularities; for example, we are more verbose in situations that require finer distinctions. However, these regularities are sometimes obscured by semantic and syntactic differences. Using a newly-collected dataset of color reference games in Mandarin Chinese (which we release to the public), we confirm that a variety of constructions display the same sensitivity to contextual difficulty in Chinese and English. We then show that a neural speaker agent trained on bilingual data with a simple multitask learning approach d…
Massachusetts Institute of Technology（麻省理工学院）；Center for Brains Minds and Machines, Massachusetts Institute of Technology（麻省理工学院脑、心智与机器中心）
Models of context-sensitive communication often use the Rational Speech Act framework (RSA; Frank & Goodman, 2012), which formulates listeners and speakers in a cooperative reasoning process. However, the standard RSA formulation can only be applied to small domains, and large-scale applications have relied on imitating human behavior. Here, we propose a new approach to scalable pragmatics, building upon recent theoretical results (Zaslavsky et al., 2020) that characterize pragmatic reasoning in terms of general information-theoretic principles. Specifically, we propose an architecture and lea…
University of California, Berkeley（加州大学伯克利分校）；Carnegie Mellon University（卡内基梅隆大学）；Boston University（波士顿大学）
Navigation guided by natural language instructions presents a challenging reasoning problem for instruction followers. Natural language instructions typically identify only a few high-level decisions and landmarks rather than complete low-level motor behaviors; much of the missing information must be inferred based on perceptual context. In machine learning settings, this is doubly challenging: it is difficult to collect enough annotated data to enable learning of this reasoning process from scratch, and also difficult to implement the reasoning process using generic sequence models. Here we d…
What computational principles underlie human pragmatic reasoning? A prominent approach to pragmatics is the Rational Speech Act (RSA) framework, which formulates pragmatic reasoning as probabilistic speakers and listeners recursively reasoning about each other. While RSA enjoys broad empirical support, it is not yet clear whether the dynamics of such recursive reasoning may be governed by a general optimization principle. Here, we present a novel analysis of the RSA framework that addresses this question. First, we show that RSA recursion implements an alternating maximization for optimizing a…
A fine-grained comparison of pragmatic language understanding in humans and language models
Massachusetts Institute of Technology（麻省理工学院）；Sarah Lawrence College（莎拉·劳伦斯学院）；Carleton University（卡尔顿大学）；McGovern Institute for Brain Research, Massachusetts Institute of Technology（麻省理工学院麦戈文脑科学研究所）
Simple reference games are of central theoretical and empirical importance in the study of situated language use. Although language provides rich, compositional truth-conditional semantics to facilitate reference, speakers and listeners may sometimes lack the overall lexical and cognitive resources to guarantee successful reference through these means alone. However, language also has rich associational structures that can serve as a further resource for achieving successful reference. Here we investigate this use of associational information in a setting where only associational information i…
Natural language is an intuitive and expressive way to communicate reward information to autonomous agents. It encompasses everything from concrete instructions to abstract descriptions of the world. Despite this, natural language is often challenging to learn from: it is difficult for machine learning methods to make appropriate inferences from such a wide range of input. This paper proposes a generalization of reward design as a unifying principle to ground linguistic communication: speakers choose utterances to maximize expected rewards from the listener's future behaviors. We first extend …
Universität Osnabrück（奥斯纳布吕克大学）；The University of Edinburgh（爱丁堡大学）；Eberhard Karls Universität Tübingen（蒂宾根大学）
While a large body of work has scrutinized the meaning of conditional sentences, considerably less attention has been paid to formal models of their pragmatic use and interpretation. Here, we take a probabilistic approach to pragmatic reasoning about indicative conditionals which flexibly integrates gradient beliefs about richly structured world states. We model listeners' update of their prior beliefs about the causal structure of the world and the joint probabilities of the consequent and antecedent based on assumptions about the speaker's utterance production protocol. We show that, when su…
University of Washington（华盛顿大学）；Allen Institute for Artificial Intelligence（艾伦人工智能研究所）；Stanford University（斯坦福大学）
Warning: this paper contains content that may be offensive or upsetting. Language has the power to reinforce stereotypes and project social biases onto others. At the core of the challenge is that it is rarely what is stated explicitly, but rather the implied meanings, that frame people's judgments about others. For example, given a statement that "we shouldn't lower our standards to hire more women," most listeners will infer the implicature intended by the speaker -- that "women (candidates) are less qualified." Most semantic formalisms, to date, do not capture such pragmatic implications in…
