随着规模更大、更新的大语言模型在战略性心智理论(Theory of Mind, ToM)任务上的表现持续提升,对这些前沿模型的需求也相应增长。然而,它们的部署在算力和时间方面成本都很高。本文中,我们探究了通过微调创建高性能小型专用算法的可行性。为此,我们首先向一个大型预训练模型呈现20个独特场景——这些场景将不同社会情境与各类社会困境博弈相结合,记录其答案,并将这些答案用于同系列更小模型的问答微调。我们的研究聚焦于上下文内的博弈论决策,这正是人类互动发生的领域,既需要心智理论(或其类似表现),也需要对社会动态的理解。因此,小型模型的训练不仅基于大模型给出的答案,还基于大模型提供的决策动机,这些动机应包含应对战略困境和社会线索的建议与指导。我们发现,经过微调的小型语言模型持续缩小了其预训练版本与更大规模同类模型之间的性能差距,且其性能提升延伸到了训练示例之外的领域和情境中,包括包含完全不同博弈结构的样本外场景。以与大模型行为的对齐度(100%代表行为无法区分)为衡量标准,在所有博弈中,小型模型经微调后平均提升了46%。当面对样本外的社会情境和博弈时,微调后的模型仍表现出显著的对齐水平,分别实现了18%和28%的提升。
英文摘要:
As the performance of larger, newer Large Language Models continues to improve for strategic Theory of Mind (ToM) tasks, the demand for these state-of-the-art models increases commensurately. However, their deployment is costly both in terms of processing power and time. In this paper, we investigate the feasibility of creating smaller, highly-performing specialized algorithms by way of fine-tuning. To do this, we first present a large pre-trained model with 20 unique scenarios that combine different social contexts with games of varying social dilemmas, record its answers, and use them for Q&A fine-tuning on a smaller model of the same family. Our focus is on in-context game-theoretic decision-making, the same domain within which human interaction occurs and that requires both a theory of mind (or a semblance thereof) and an understanding of social dynamics. The smaller model is therefore trained not just on the answers provided, but also on the motivations provided by the larger model, which should contain advice and guidelines to navigate both strategic dilemmas and social cues. We find that the fine-tuned smaller language model consistently bridged the gap in performance between the smaller pre-trained version of the model and its larger relative and that its improvements extended in areas and contexts beyond the ones provided in the training examples, including on out-of-sample scenarios that include completely different game structures. On average for all games, through fine-tuning, the smaller model showed a 46% improvement measured as alignment towards the behavior of the larger model, with 100% representing indistinguishable behavior. When presented with out-of-sample social contexts and games, the fine-tuned model still displays remarkable levels of alignment, reaching an improvement of 18% and 28% respectively.
MindFlow: Mind Supernet Powered Thinking Flows for Research Idea Innovation
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);Peking University(北京大学);DeepWisdom(深度智慧);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Renmin University of China(中国人民大学)
Mind Your Step (by Step): Chain-of-Thought can Reduce Performance on Tasks where Thinking Makes Humans Worse
Department of Computer Science, Princeton University, Princeton, NJ, USA(计算机科学系,普林斯顿大学);NYU Center for Data Science, New York University, New York, NY, USA(纽约大学数据科学中心);Department of Psychology, Princeton University, Princeton, NJ, USA(心理学系,普林斯顿大学)
Mind's Mirror: Distilling Self-Evaluation Capability and Comprehensive Thinking from Large Language Models
Zhejiang University(浙江大学);School of Public Health, Zhejiang University(浙江大学公共卫生学院);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Liangzhu Laboratory and Institute of Wenzhou, Zhejiang University(浙江大学良渚实验室与温州研究院);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)
Meta-Thinking in LLMs via Multi-Agent Reinforcement Learning: A Survey
University of Oklahoma(俄克拉荷马大学);Stanford University(斯坦福大学);School of Electrical and Computer Engineering, Oklahoma State University(电气与计算机工程学院,俄克拉荷马州立大学);University of Glasgow(格拉斯哥大学)
Perspectra: Choosing Your Experts Enhances Critical Thinking in Multi-Agent Research Ideation
Informatics, University of Illinois Urbana-Champaign(信息学,伊利诺伊大学厄巴纳-香槟分校);Siebel School of Computing and Data Science, University of Illinois Urbana-Champaign(Siebel 计算与数据科学学院,伊利诺伊大学厄巴纳-香槟分校);Department of Computer Science, University of Toronto(计算机科学系,多伦多大学);Allen Institute for AI(人工智能研究院);School of Information Sciences, University of Illinois at Urbana-Champaign(信息科学学院,伊利诺伊大学厄巴纳-香槟分校)
CreativityNeuro: Steering Language Model Weights to Improve Divergent Thinking and Reduce Mode Collapse
Center for Brain Science, Harvard University(哈佛大学脑科学中心);CBS-NTT Program in Physics of Intelligence, Harvard University(哈佛大学CBS-NTT智能物理项目);Prior Computers;AI Innovation Institute, Stony Brook University(石溪大学人工智能创新研究所)
On the Thinking-Language Modeling Gap in Large Language Models
Hong Kong Baptist University(香港 Baptist 大学);MBZUAI Carnegie Mellon University(MBZUAI 美国卡内基梅隆大学);The University of Sydney(悉尼大学);The Chinese University of Hong Kong(香港中文大学)
MIND: From Passive Mimicry to Active Reasoning through Capability-Aware Multi-Perspective CoT Distillation
State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, National Engineering Research Center for Visual Information and Applications, and Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University(西安交通大学人机混合增强智能国家重点实验室、视觉信息与应用国家工程研究中心、人工智能与机器人研究所);Nankai University(南开大学);The Hong Kong University of Science and Technology(Guangzhou)(香港科技大学(广州));School of Software Engineering, Xi’an Jiaotong University(西安交通大学软件学院)
The Cultural Psychology of Large Language Models: Is ChatGPT a Holistic or Analytic Thinker?
New York University(纽约大学);University of Chicago(芝加哥大学);Massachusetts Institute of Technology(麻省理工学院);Johns Hopkins University(约翰斯·霍普金斯大学);University of California Los Angeles(加州大学洛杉矶分校)
Codenames as a Benchmark for Large Language Models
College of Science and Engineering, Flinders University(科学与工程学院,弗林德斯大学);Faculty of Engineering and IT, University of Melbourne(工程与信息技术学院,墨尔本大学);ICTEAM, UCLouvain(ICTEAM,UCLouvain)