arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2401.05033cs.CLcs.AI
收藏

通过自言自语引导基于LLM的任务型对话智能体

Bootstrapping LLM-based Task-Oriented Dialogue Agents via Self-Talk

IT University of Copenhagen(哥本哈根信息技术大学);Pioneer Centre for Artificial Intelligence(先锋人工智能中心);AWS AI Labs(亚马逊云科技人工智能实验室)

Dennis Ulmer, Elman Mansimov, Kaixiang Lin, Justin Sun, Xibin Gao, Yi Zhang

AI总结:

提出基于LLM自言自语的数据生成方法,通过多角色对话产生训练数据并利用自动化指标过滤,有效提升任务型对话智能体在特定工作流中的监督微调效果。

摘要:

大型语言模型(LLMs)是强大的对话智能体,但使其专门用于实现特定功能可能具有挑战性。指令微调,即在指令和人类生成的样本响应上微调模型(Ouyang et al., 2022),已被证明是一种有效的方法,但需要大量数据样本,这些样本可能不可用或生成成本高昂。此外,当目标是让LLM在对话中遵循特定工作流程而非单一指令时,这种成本会进一步增加。受强化学习中自博弈技术及使用LLMs模拟人类智能体的启发,我们提出了一种更有效的数据收集方法,通过LLMs以不同角色参与对话。该方法通过LLMs的“自言自语”生成训练数据,这些数据可被优化并用于监督微调。我们引入了一种自动化方式来衡量对话的(部分)成功程度。该指标用于过滤生成的对话数据,并将其反馈给LLM进行训练。基于我们对对话质量的自动化和人工评估,我们证明此类自言自语数据能改善结果。此外,我们研究了生成对话质量的多种特征,以及它们如何与作为训练数据的潜在效用相关联。

英文摘要:

Large language models (LLMs) are powerful dialogue agents, but specializing them towards fulfilling a specific function can be challenging. Instructing tuning, i.e. tuning models on instruction and sample responses generated by humans (Ouyang et al., 2022), has proven as an effective method to do so, yet requires a number of data samples that a) might not be available or b) costly to generate. Furthermore, this cost increases when the goal is to make the LLM follow a specific workflow within a dialogue instead of single instructions. Inspired by the self-play technique in reinforcement learning and the use of LLMs to simulate human agents, we propose a more effective method for data collection through LLMs engaging in a conversation in various roles. This approach generates a training data via "self-talk" of LLMs that can be refined and utilized for supervised fine-tuning. We introduce an automated way to measure the (partial) success of a dialogue. This metric is used to filter the generated conversational data that is fed back in LLM for training. Based on our automated and human evaluations of conversation quality, we demonstrate that such self-talk data improves results. In addition, we examine the various characteristics that showcase the quality of generated dialogues and how they can be connected to their potential utility as training data.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑