arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.14844cs.CLcs.AIcs.MA
收藏

揭示AI多智能体系统中的对话偏见

Unmasking Conversational Bias in AI Multiagent Systems

University of Calabria(卡拉布里亚大学);ICAR-CNR(意大利国家研究委员会信息与自动化研究所);IT University of Copenhagen(哥本哈根信息技术大学)

Erica Coppolillo, Giuseppe Manco, Luca Maria Aiello

AI总结:

该研究针对AI多智能体系统的对话偏见检测空白,提出对话型LLMs多智能体系统偏见量化框架,发现现有问卷式方法无法检测到该场景下的偏见,呼吁开发更复杂的偏见检测与缓解工具包。

摘要:

检测生成式模型输出中的偏见对于降低其在关键场景应用中的潜在风险至关重要。然而,现大多数识别生成文本偏见的方法仅孤立地考虑模型,忽略了其上下文应用场景,具体而言,涉及生成式模型的多智能体系统中可能产生的偏见仍未得到充分研究。为填补这一空白,本文提出一个用于量化对话型大语言模型(LLMs)多智能体系统内偏见的框架。我们的方法涉及模拟小型回音室:将成对的LLMs初始化至对某一极化话题持有一致观点,让它们进行讨论。与预期相反,我们观察到生成消息中表达的立场出现显著转变,尤其是在所有智能体最初均表达保守观点的回音室中,这与诸多LLMs向自由立场倾斜的已被充分记录的政治偏见相符。关键在于,依赖问卷的现有最先进偏见检测方法无法检测到回音室实验中观察到的偏见,这凸显了开发更复杂的AI多智能体系统偏见检测与缓解工具包的迫切需求。用于开展实验的代码已公开可用。

英文摘要:

Detecting biases in the outputs produced by generative models is essential to reduce the potential risks associated with their application in critical settings. However, the majority of existing methodologies for identifying biases in generated text consider the models in isolation and neglect their contextual applications. Specifically, the biases that may arise in multi-agent systems involving generative models remain under-researched. To address this gap, we present a framework designed to quantify biases within multi-agent systems of conversational Large Language Models (LLMs). Our approach involves simulating small echo chambers, where pairs of LLMs, initialized with aligned perspectives on a polarizing topic, engage in discussions. Contrary to expectations, we observe significant shifts in the stance expressed in the generated messages, particularly within echo chambers where all agents initially express conservative viewpoints, in line with the well-documented political bias of many LLMs toward liberal positions. Crucially, the bias observed in the echo-chamber experiment remains undetected by current state-of-the-art bias detection methods that rely on questionnaires. This highlights a critical need for the development of a more sophisticated toolkit for bias detection and mitigation for AI multi-agent systems. The code to perform the experiments is publicly available.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑