arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2501.13381cs.CL
收藏

行我们所行,非你所想:大语言模型的从众性

Do as We Do, Not as You Think: the Conformity of Large Language Models

Zhejiang University(浙江大学)

Zhiyuan Weng, Guikun Chen, Wenguan Wang

AI总结:

本文针对LLM驱动多智能体系统中未被充分探索的从众性问题,构建BenchForm基准展开研究,揭示其存在性、影响因素并提出缓解策略,为构建更稳健的协作AI系统提供见解。

摘要:

大语言模型(LLMs)的最新进展彻底改变了智能体领域,使协作式多智能体系统能够在各个领域解决复杂问题。然而,这些系统中类似人类群体动力学中的从众偏差和群体思维等现象的从众性潜力在很大程度上仍未被探索,这引发了对其集体问题解决能力和可能的伦理影响的担忧。本文对LLM驱动的多智能体系统中的从众性进行了全面研究,重点关注三个方面:从众性的存在、影响从众性的因素以及潜在的缓解策略。具体而言,我们引入了BenchForm,这是一个新的面向从众性的基准,包含推理密集型任务和五种不同的交互协议,旨在探究LLMs在协作场景中的行为。我们在BenchForm上评估了几个代表性的LLMs,使用从众率和独立率等指标来量化从众性的影响。我们的分析深入探讨了影响从众性的因素,包括交互时间和多数派规模,并研究了主体智能体如何合理化其从众行为。此外,我们探索了两种缓解从众效应的策略,即开发增强型角色和实施反思机制。从实证结果和案例研究中得出了一些关于LLMs从众性的有趣发现。我们希望这些见解能够为更稳健且符合伦理的协作式AI系统铺平道路。我们的基准和代码可在BenchForm获取。

英文摘要:

Recent advancements in large language models (LLMs) revolutionize the field of intelligent agents, enabling collaborative multi-agent systems capable of tackling complex problems across various domains. However, the potential of conformity within these systems, analogous to phenomena like conformity bias and groupthink in human group dynamics, remains largely unexplored, raising concerns about their collective problem-solving capabilities and possible ethical implications. This paper presents a comprehensive study on conformity in LLM-driven multi-agent systems, focusing on three aspects: the existence of conformity, the factors influencing conformity, and potential mitigation strategies. In particular, we introduce BenchForm, a new conformity-oriented benchmark, featuring reasoning-intensive tasks and five distinct interaction protocols designed to probe LLMs' behavior in collaborative scenarios. Several representative LLMs are evaluated on BenchForm, using metrics such as conformity rate and independence rate to quantify conformity's impact. Our analysis delves into factors influencing conformity, including interaction time and majority size, and examines how the subject agent rationalizes its conforming behavior. Furthermore, we explore two strategies to mitigate conformity effects, i.e., developing enhanced personas and implementing a reflection mechanism. Several interesting findings regarding LLMs' conformity are derived from empirical results and case studies. We hope that these insights can pave the way for more robust and ethically-aligned collaborative AI systems. Our benchmark and code are available at BenchForm.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑