arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.05020cs.CLcs.AI
收藏

这是真实生活吗?这只是幻想吗?用大语言模型模拟社会互动的具有误导性的成功

Is this the real life? Is this just fantasy? The Misleading Success of Simulating Social Interactions With LLMs

Carnegie Mellon University(卡内基梅隆大学);Massachusetts Institute of Technology(麻省理工学院);Allen Institute for AI(艾伦人工智能研究所)

Xuhui Zhou, Zhe Su, Tiwalayo Eisape, Hyunwoo Kim, Maarten Sap

AI总结:

本研究针对当前大语言模型社会模拟多采用全知视角、与真实信息不对称互动不符的问题,构建全知与非全知设置下的社会模拟评估框架,实验发现LLM在全知设置表现好但真实场景表现差,指出信息不对称是LLM智能体的核心挑战。

摘要:

大语言模型(LLM)的最新进展催生了更丰富的社会模拟,使得各类社会现象的研究得以开展。然而,近期多数研究在这类模拟中采用了更具全知视角的设置(例如用单个LLM生成所有对话者的内容),这与现实世界中人类和AI智能体参与的非全知、信息不对称的互动存在根本矛盾。为探究这些差异,我们构建了一个评估框架,用于在多种设置(全知、非全知)下用LLM模拟社会互动。实验表明,LLM在不切实际的全知模拟设置中表现更优,但在更准确反映信息不对称的真实世界条件的设置中表现不佳。研究结果显示,解决信息不对称问题仍是基于LLM的智能体面临的一项根本挑战。

英文摘要:

Recent advances in large language models (LLM) have enabled richer social simulations, allowing for the study of various social phenomena. However, most recent work has used a more omniscient perspective on these simulations (e.g., single LLM to generate all interlocutors), which is fundamentally at odds with the non-omniscient, information asymmetric interactions that involve humans and AI agents in the real world. To examine these differences, we develop an evaluation framework to simulate social interactions with LLMs in various settings (omniscient, non-omniscient). Our experiments show that LLMs perform better in unrealistic, omniscient simulation settings but struggle in ones that more accurately reflect real-world conditions with information asymmetry. Our findings indicate that addressing information asymmetry remains a fundamental challenge for LLM-based agents.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑