arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2401.06509cs.CL
收藏

AntEval: 大语言模型驱动智能体的社交互动能力评估

AntEval: Evaluation of Social Interaction Competencies in LLM-Driven Agents

University of Technology Sydney(悉尼科技大学);Zhejiang University(浙江大学)

Yuanzhi Liang, Linchao Zhu, Yi Yang

AI总结:

针对LLM驱动智能体复杂社交互动能力缺乏量化评估的问题,提出包含新型互动框架及IEP、IEG两个指标的AntEval框架,可客观评估互动能力,助力构建更自然的类人互动智能体。

摘要:

大语言模型(LLM)已展现出在广泛场景中复刻人类行为的能力。然而,其处理复杂多角色社交互动的能力尚未得到充分探索,主要原因是缺乏可靠的量化评估方法。这一缺口延缓了擅长闲聊等简单交流之外更细腻互动的智能体的发展。为应对这一挑战,我们提出多智能体互动评估框架(AntEval),包含一套新颖的互动框架与评估方法。该互动框架旨在构建复杂的互动环境,促进社交互动中的信息交换与意图表达。此外,我们提出了评估方法,包含两个指标:信息交换准确率(IEP)和互动表达力差距(IEG),用于量化、客观地评估智能体的互动能力。我们的研究结果凸显了这些评估方法的实用性,并表明其在提升LLM构建能以类人复杂性进行更自然互动的智能体的能力方面具有显著潜力。

英文摘要:

Large Language Models (LLMs) have demonstrated their ability to replicate human behaviors across a wide range of scenarios. However, their capability in handling complex, multi-character social interactions has yet to be fully explored, primarily due to the absence of robust, quantitative evaluation methods. This gap has slowed the development of agents proficient in more nuanced interactions beyond simple exchanges, for example, small talk. To address this challenge, we introduce the Multi-Agent Interaction Evaluation Framework (AntEval), encompassing a novel interaction framework and evaluation methods. The interaction framework aims to foster an complex interaction environment that bolsters information exchange and intention expression within social interactions. Furthermore, we introduce evaluation methods, including two metrics: Information Exchanging Precision (IEP) and Interaction Expressiveness Gap (IEG), designed for the quantitative and objective assessment of agents' interaction competencies. Our findings highlight the utility of these evaluative methods and show significant potential for improving LLMs' ability to construct agents that interact in a more natural manner with human-like intricacy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑