arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2310.16301cs.CL

ChatGPT是一个好的多方对话解决器吗?

Is ChatGPT a Good Multi-Party Conversation Solver?

National Engineering Research Center of Speech and Language Information Processing(国家语音与语言信息处理工程技术研究中心);University of Science and Technology of China(中国科学技术大学)

Chao-Hong Tan, Jia-Chen Gu, Zhen-Hua Ling

AI总结:

本文实证评估了ChatGPT和GPT-4在多方对话任务中的零样本能力,发现GPT-4表现优于ChatGPT,并通过引入说话者与受话者架构提升性能,为构建更鲁棒的MPC智能体提供了参考。

摘要:

大型语言模型(LLMs)已成为自然语言处理领域极具影响力的工具;然而,它们处理多方对话(MPCs)的能力——这是一种涉及多名对话者进行复杂信息交换的场景——仍属未知。本文深入探讨了ChatGPT和GPT-4等生成式LLMs在MPC语境中的潜力。研究通过在三个MPC数据集上对ChatGPT和GPT-4进行评估,涵盖五项代表性任务,实证分析了它们的零样本学习能力。结果显示,ChatGPT在多项评估的MPC任务上表现欠佳,而GPT-4的结果则预示着充满希望的未来。此外,我们尝试通过引入MPC结构(包含说话者与受话者架构)来提升性能。本研究对生成式LLMs在MPC中的应用进行了详尽的评估与分析,为构思和创建更有效、更鲁棒的MPC智能体提供了启示。同时,本工作也强调了利用LLMs处理MPC时面临的挑战,如解析图形信息流和生成风格一致的回复。

英文摘要:

Large Language Models (LLMs) have emerged as influential instruments within the realm of natural language processing; nevertheless, their capacity to handle multi-party conversations (MPCs) -- a scenario marked by the presence of multiple interlocutors involved in intricate information exchanges -- remains uncharted. In this paper, we delve into the potential of generative LLMs such as ChatGPT and GPT-4 within the context of MPCs. An empirical analysis is conducted to assess the zero-shot learning capabilities of ChatGPT and GPT-4 by subjecting them to evaluation across three MPC datasets that encompass five representative tasks. The findings reveal that ChatGPT's performance on a number of evaluated MPC tasks leaves much to be desired, whilst GPT-4's results portend a promising future. Additionally, we endeavor to bolster performance through the incorporation of MPC structures, encompassing both speaker and addressee architecture. This study provides an exhaustive evaluation and analysis of applying generative LLMs to MPCs, casting a light upon the conception and creation of increasingly effective and robust MPC agents. Concurrently, this work underscores the challenges implicit in the utilization of LLMs for MPCs, such as deciphering graphical information flows and generating stylistically consistent responses.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑