Recent advancements in large language models (LLMs) revolutionize the field of intelligent agents, enabling collaborative multi-agent systems capable of tackling complex problems across various domains. However, the potential of conformity within these systems, analogous to phenomena like conformity bias and groupthink in human group dynamics, remains largely unexplored, raising concerns about their collective problem-solving capabilities and possible ethical implications. This paper presents a comprehensive study on conformity in LLM-driven multi-agent systems, focusing on three aspects: the existence of conformity, the factors influencing conformity, and potential mitigation strategies. In particular, we introduce BenchForm, a new conformity-oriented benchmark, featuring reasoning-intensive tasks and five distinct interaction protocols designed to probe LLMs' behavior in collaborative scenarios. Several representative LLMs are evaluated on BenchForm, using metrics such as conformity rate and independence rate to quantify conformity's impact. Our analysis delves into factors influencing conformity, including interaction time and majority size, and examines how the subject agent rationalizes its conforming behavior. Furthermore, we explore two strategies to mitigate conformity effects, i.e., developing enhanced personas and implementing a reflection mechanism. Several interesting findings regarding LLMs' conformity are derived from empirical results and case studies. We hope that these insights can pave the way for more robust and ethically-aligned collaborative AI systems. Our benchmark and code are available at BenchForm.
ExpLang: Improved Exploration and Exploitation in LLM Reasoning with On-Policy Thinking Language Selection
National Key Laboratory for Novel Software Technology, Nanjing University, Jiangsu, China(新型软件技术国家重点实验室,南京大学,江苏,中国);Shanghai Artificial Intelligence Laboratory, Shanghai, China(上海人工智能实验室,上海,中国);Department of Linguistics and Translation, City University of Hong Kong, Hong Kong, China(语言学与翻译系,香港城市大学,香港,中国);School of Mathematical Sciences, Dalian University of Technology, Liaoning, China(数学科学学院,大连理工大学,辽宁,中国)
Latent Thinking Optimization: Your Latent Reasoning Language Model Secretly Encodes Reward Signals in Its Latent Thoughts
Department of Computer Science and Engineering, The Ohio State University, USA(计算机科学与工程系,俄亥俄州立大学);Department of Biomedical Informatics, The Ohio State University, USA(生物医学信息学系,俄亥俄州立大学);Translational Data Analytics Institute, The Ohio State University, USA(转化数据分析研究所,俄亥俄州立大学)
BRAINTEASER: Lateral Thinking Puzzles for Large Language Models
Information Sciences Institute, Viterbi School of Engineering, University of Southern California(南加州大学维特比工程学院信息科学研究所);Department of Computer Science, Faculty of Science, Vrije Universiteit Amsterdam(阿姆斯特丹自由大学理学院计算机科学系);Tencent AI Lab(腾讯AI实验室)
Distilling Reasoning Ability from Large Language Models with Adaptive Thinking
College of Computer Science and Technology, National University of Defense Technology(计算机科学与技术学院,国防科技大学);College of Intelligence Science and Technology, National University of Defense Technology(智能科学与技术学院,国防科技大学)
ThinkNote: Enhancing Knowledge Integration and Utilization of Large Language Models via Constructivist Cognition Modeling
School of Computer Science and Engineering, Northeastern University(东北大学计算机科学与工程学院);Department of Computer Science and Technology, Tsinghua University(清华大学计算机科学与技术系);Language Technologies Institute, Carnegie Mellon University(卡内基梅隆大学语言技术研究所)
Conformity Generates Collective Misalignment in AI Agents Societies
University of Konstanz(康斯坦茨大学);Sony Computer Science Laboratories - Rome(索尼计算机科学实验室-罗马);Sapienza University of Rome(罗马大学);Max Planck Institute for Dynamics and Self-Organization(马克斯·普朗克动态与自组织研究所);Institute for the Dynamics of Complex Systems, University of Gottingen(复杂系统动力学研究所,哥廷根大学)
Do LLMs Need to Think in One Language? Correlation between Latent Language and Task Performance
NAIST(日本国立信息与科技研究所);NII LLMC(日本信息处理学会大语言模型中心);MBZUAI(微软亚洲人工智能研究院);RIKEN(日本研究机构);Tohoku University(东北大学);The University of Tokyo(东京大学);Nagoya Institute of Technology(名古屋技术大学)
StreamingThinker: Large Language Models Can Think While Reading
Shanghai Jiao Tong University(上海交通大学);Eastern Institute of Technology, Ningbo(宁波东部技术研究院);Ningbo Key Laboratory of Spatial Intelligence and Digital Derivative, Institute of Digital Twin(宁波空间智能与数字衍生关键实验室,数字孪生研究院);Hong Kong Polytechnic University(香港理工大学);Munich Center for Machine Learning, LMU(慕尼黑机器学习中心,莱茵-慕尼黑大学)
The Price of a Second Thought: On the Evaluation of Reasoning Efficiency in Large Language Models
University of Electronic Science and Technology of China(电子科技大学);Beijing Academy of Artificial Intelligence(北京人工智能研究院);Nanyang Technological University(南洋理工大学)
Ontology-Guided Reverse Thinking Makes Large Language Models Stronger on Knowledge Graph Question Answering
Harbin Institute of Technology(哈尔滨工业大学);Beijing University of Posts and Telecommunications(北京邮电大学);Joint Laboratory of HIT and iFLYTEK(哈工大与科大讯飞联合实验室);University of Science and Technology of China(中国科学技术大学);Pengcheng Laboratory(鹏城实验室)
Multi-Persona Thinking for Bias Mitigation in Large Language Models
Dept. Computing Science, Alberta Machine Intelligence Institute (Amii) University of Alberta, Canada(计算科学系,阿尔伯塔机器智能研究所(Amii),阿尔伯塔大学,加拿大);Canada CIFAR AI Chair, Amii(加拿大 CIFAR 人工智能主席,Amii)