基于用户画像感知的预训练模型与参数高效微调的用户特定对话生成
User-Specific Dialogue Generation with User Profile-Aware Pre-Training Model and Parameter-Efficient Fine-Tuning
AI总结：
本文针对真实用户特定对话生成，提出结合用户画像感知预训练对话模型与参数高效微调的方法，以少量数据实现高再现性的用户话语生成。
摘要：
本文研究用户特定对话。以往个性化对话研究侧重于实现由人物设定描述所定义的虚拟用户对话，与此不同，用户特定对话旨在再现超越基于人物设定对话的真实用户对话。使用目标用户对话历史进行微调是构建用户特定模型的一种高效学习方法。然而，由于数据量较小，该方法容易出现过拟合和模型破坏。因此，我们提出一种用户特定模型的学习方法，将参数高效微调与包含用户画像的预训练对话模型相结合。参数高效微调只向整个模型添加少量参数，因此即使训练数据很少，也能高效训练，并且对模型破坏具有鲁棒性。此外，通过为自动推断的用户画像添加简单提示而学习得到的预训练模型，即使在微调期间训练数据很少的情况下，也能生成增强用户画像知识的语音。在实验中，我们将所提模型与使用包含用户个人信息提示的大语言模型话语生成进行了比较。再现真实用户话语的实验表明，所提模型即使使用较小的模型，也能生成比对比方法具有更高再现性的话语。
英文摘要：
This paper addresses user-specific dialogs. In contrast to previous research on personalized dialogue focused on achieving virtual user dialogue as defined by persona descriptions, user-specific dialogue aims to reproduce real-user dialogue beyond persona-based dialogue. Fine-tuning using the target user's dialogue history is an efficient learning method for a user-specific model. However, it is prone to overfitting and model destruction due to the small amount of data. Therefore, we propose a learning method for user-specific models by combining parameter-efficient fine-tuning with a pre-trained dialogue model that includes user profiles. Parameter-efficient fine-tuning adds a small number of parameters to the entire model, so even small amounts of training data can be trained efficiently and are robust to model destruction. In addition, the pre-trained model, which is learned by adding simple prompts for automatically inferred user profiles, can generate speech with enhanced knowledge of the user's profile, even when there is little training data during fine-tuning. In experiments, we compared the proposed model with large-language-model utterance generation using prompts containing users' personal information. Experiments reproducing real users' utterances revealed that the proposed model can generate utterances with higher reproducibility than the compared methods, even with a small model.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
