arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2302.03269cs.CLcs.AIcs.IR
收藏

PLACES:为社会会话合成建立语言模型

PLACES: Prompting Language Models for Social Conversation Synthesis

Columbia University(哥伦比亚大学);Amazon Alexa AI(亚马逊Alexa人工智能部门)

Maximillian Chen, Alexandros Papangelis, Chenyang Tao, Seokhwan Kim, Andy Rosenbaum, Yang Liu, Zhou Yu, Dilek Hakkani-Tur

摘要:

收集高质量的会话数据对于大多数应用程序来说可能非常昂贵,并且由于隐私,道德或类似问题而对其他应用程序来说是不可行的。解决这个问题的一个有希望的方向是通过提示大语言模型来生成合成对话。在这项工作中,我们使用一小部分专家撰写的对话作为上下文示例,使用提示合成社交对话数据集。我们对我们的合成对话与人类收集的对话进行了几次彻底的评估。这包括直接对合成对话进行人工评估的对话质量的各个维度,以及对合成生成的数据集进行微调的聊天机器人的交互式人工评估。我们还证明,这种提示方法是可推广到多方对话,为多方任务创建新的合成数据提供了可能。与从人类收集的多方数据集采样的对话摘录相比,我们的合成多方对话在所有测量维度上都得到了更有利的评价。

英文摘要:

Collecting high quality conversational data can be very expensive for most applications and infeasible for others due to privacy, ethical, or similar concerns. A promising direction to tackle this problem is to generate synthetic dialogues by prompting large language models. In this work, we use a small set of expert-written conversations as in-context examples to synthesize a social conversation dataset using prompting. We perform several thorough evaluations of our synthetic conversations compared to human-collected conversations. This includes various dimensions of conversation quality with human evaluation directly on the synthesized conversations, and interactive human evaluation of chatbots fine-tuned on the synthetically generated dataset. We additionally demonstrate that this prompting approach is generalizable to multi-party conversations, providing potential to create new synthetic data for multi-party tasks. Our synthetic multi-party conversations were rated more favorably across all measured dimensions compared to conversation excerpts sampled from a human-collected multi-party dataset.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑