arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2311.05195cs.CL

PRODIGy：一个基于Profile的对话生成数据集

PRODIGy: a PROfile-based DIalogue Generation dataset

Fondazione Bruno Kessler（布鲁诺·凯塞勒基金会）；University of Trento（特伦托大学）

Daniela Occhipinti, Serra Sinem Tekiroglu, Marco Guerini

阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

提出统一框架PRODIGy，结合标准与复杂profile表示构建对话数据集，实验证明基于profile的模型在跨领域泛化能力及人类评估上均优于纯对话模型，并兼顾隐私保护配置。

摘要：

为对话智能体提供profile表示可以提升其一致性和连贯性，从而带来更好的对话体验。然而，当前用于训练此类智能体的基于profile的对话数据集，要么包含简单且特定于对话的显式profile表示，要么包含难以收集的隐式表示。在本研究中，我们提出了一个统一框架，通过创建一种新资源将标准profile表示与更复杂的profile表示结合在一起，在该资源中，每段对话都与所有可能的说话者表示（如沟通风格、传记和性格）相对齐。该框架允许使用生成式语言模型构建多个基线，并测试多种profile配置。自动评估表明，无论是在领域内还是跨领域设置中，基于profile的模型比仅在对话上训练的模型具有更好的泛化能力。这些结果对于微调模型和基于指令的LLMs均是一致的。此外，人类评估表明，人们明显偏好与profile和上下文均保持一致的生成内容。最后，为了应对潜在的隐私问题，所有实验均在两种配置下进行：角色间和角色内。在前者中，LM将关于角色的信息存储在其内部表示中；而在后者中，LM不保留任何个人信息，仅在推理时使用这些信息。

英文摘要：

Providing dialogue agents with a profile representation can improve their consistency and coherence, leading to better conversations. However, current profile-based dialogue datasets for training such agents contain either explicit profile representations that are simple and dialogue-specific, or implicit representations that are difficult to collect. In this work, we propose a unified framework in which we bring together both standard and more sophisticated profile representations by creating a new resource where each dialogue is aligned with all possible speaker representations such as communication style, biographies, and personality. This framework allows to test several baselines built using generative language models with several profile configurations. The automatic evaluation shows that profile-based models have better generalisation capabilities than models trained on dialogues only, both in-domain and cross-domain settings. These results are consistent for fine-tuned models and instruction-based LLMs. Additionally, human evaluation demonstrates a clear preference for generations consistent with both profile and context. Finally, to account for possible privacy concerns, all experiments are done under two configurations: inter-character and intra-character. In the former, the LM stores the information about the character in its internal representation, while in the latter, the LM does not retain any personal information but uses it only at inference time.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑