Personalized dialogue agents (DAs) powered by large pre-trained language models (PLMs) often rely on explicit persona descriptions to maintain personality consistency. However, such descriptions may not always be available or may pose privacy concerns. To tackle this bottleneck, we introduce PersonaPKT, a lightweight transfer learning approach that can build persona-consistent dialogue models without explicit persona descriptions. By representing each persona as a continuous vector, PersonaPKT learns implicit persona-specific features directly from a small number of dialogue samples produced by the same persona, adding less than 0.1% trainable parameters for each persona on top of the PLM backbone. Empirical results demonstrate that PersonaPKT effectively builds personalized DAs with high storage efficiency, outperforming various baselines in terms of persona consistency while maintaining good response generation quality. In addition, it enhances privacy protection by avoiding explicit persona descriptions. Overall, PersonaPKT is an effective solution for creating personalized DAs that respect user privacy.
"In Dialogues We Learn": Towards Personalized Dialogue Without Pre-defined Profiles through In-Dialogue Learning
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）；Ant Group（蚂蚁集团）；University of Electronic Science and Technology of China（电子科技大学）；Kunming University of Science and Technology（昆明理工大学）
From Persona to Person: Enhancing the Naturalness with Multiple Discourse Relations Graph Learning in Personalized Dialogue Generation
Department of Computer Science（计算机科学系）；Information Engineering, National Cheng Kung University, Taiwan（信息工程系，国立云林科技大学，台湾）；Department of Computer Science, National Tsing Hua University, Taiwan（计算机科学系，国立清华大学，台湾）
Tsinghua University（清华大学）；Samsung Research China - Beijing (SRC-B)（三星中国研究院-北京（SRC-B））；NetEase Inc.（网易公司）
Endowing dialogue systems with personas is essential to deliver more human-like conversations. However, this problem is still far from well explored due to the difficulties of both embodying personalities in natural languages and the persona sparsity issue observed in most dialogue corpora. This paper proposes a pre-training based personalized dialogue model that can generate coherent responses using persona-sparse dialogue data. In this method, a pre-trained language model is used to initialize an encoder and decoder, and personal attribute embeddings are devised to model richer dialogue cont…
Personalized Large Language Model Assistant with Evolving Conditional Memory
The Hong Kong Polytechnic University（香港理工大学）；INF Technology (Shanghai) co., Ltd.（INF科技（上海）有限公司）；Institute of Artificial Intelligence, Soochow University, China（苏州大学人工智能研究院）
MORPHEUS: Modeling Role from Personalized Dialogue History by Exploring and Utilizing Latent Space
School of New Media and Communication, Tianjin University（天津大学新媒体与传播学院）；College of Intelligence and Computing, Tianjin University（天津大学智能与计算学部）；AI Lab, China Mobile Communication Group Tianjin Co., Ltd.（中国移动通信集团天津有限公司AI实验室）
The generation of personalized dialogue is vital to natural and human-like conversation. Typically, personalized dialogue generation models involve conditioning the generated response on the dialogue history and a representation of the persona/personality of the interlocutor. As it is impractical to obtain the persona/personality representations for every interlocutor, recent works have explored the possibility of generating personalized dialogue by finetuning the model with dialogue examples corresponding to a given persona instead. However, in real-world implementations, a sufficient number …
PsyDT: Using LLMs to Construct the Digital Twin of Psychological Counselor with Personalized Counseling Style for Psychological Counseling
School of Future Technology, South China University of Technology（华南理工大学未来技术学院）；School of Electronic and Information Engineering, South China University of Technology（华南理工大学电子与信息工程学院）；Pazhou Lab（琶洲实验室）
Cue-CoT: Chain-of-thought Prompting for Responding to In-depth Dialogue Questions with LLMs
The Chinese University of Hong Kong（香港中文大学）；Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；Huawei Noah’s Ark Lab（华为诺亚方舟实验室）；National University of Singapore（新加坡国立大学）；Peng Cheng Laboratory（鹏城实验室）
Hong Kong University of Science and Technology（香港科技大学）
Training a personalized dialogue system requires a lot of data, and the data collected for a single user is usually insufficient. One common practice for this problem is to share training dialogues between different users and train multiple sequence-to-sequence dialogue models together with transfer learning. However, current sequence-to-sequence transfer learning models operate on the entire sentence, which might cause negative transfer if different personal information from different users is mixed up. We propose a personalized decoder model to transfer finer granularity phrase-level knowled…
Center for Artificial Intelligence Research (CAiRE)（人工智能研究中心（CAiRE））；The Hong Kong University of Science and Technology（香港科技大学）
Considerable progress has been made towards conversational models that generate coherent and fluent responses by training large language models on large dialogue datasets. These models have little or no control of the generated responses and miss two important features: continuous dialogue skills integration and seamlessly leveraging diverse knowledge sources. In this paper, we propose the Adapter-Bot, a dialogue model that uses a fixed backbone conversational model such as DialGPT (Zhang et al., 2019) and triggers on-demand dialogue skills (e.g., emphatic response, weather information, movie …
Renmin University of China（中国人民大学）；University of Montreal（蒙特利尔大学）
Personalized dialogue systems explore the problem of generating responses that are consistent with the user's personality, which has raised much attention in recent years. Existing personalized dialogue systems have tried to extract user profiles from dialogue history to guide personalized response generation. Since the dialogue history is usually long and noisy, most existing methods truncate the dialogue history to model the user's personality. Such methods can generate some personalized responses, but a large part of dialogue history is wasted, leading to sub-optimal performance of personal…
With the flourish of services on the Internet, a prerequisite for service providers to precisely deliver services to their customers is to capture user requirements comprehensively, accurately, and efficiently. This is called the ``Service Requirement Elicitation (SRE)'' task. Considering the amount of customers is huge, it is an inefficient way for service providers to interact with each user by face-to-face dialog. Therefore, to elicit user requirements with the assistance of virtual intelligent assistants has become a mainstream way. Since user requirements generally consist of different le…
Nanyang Technological University（南洋理工大学）；Institute for Infocomm Research A*STAR（新加坡科技研究局资讯通信研究院）
Conventional approaches to personalized dialogue generation typically require a large corpus, as well as predefined persona information. However, in a real-world setting, neither a large corpus of training data nor persona information are readily available. To address these practical limitations, we propose a novel multi-task meta-learning approach which involves training a model to adapt to new personas without relying on a large corpus, or on any predefined persona information. Instead, the model is tasked with generating personalized responses based on only the dialogue context. Unlike prio…
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）；School of Information, Renmin University of China（中国人民大学信息学院）；Université de Montréal（蒙特利尔大学）
Natural language dialogue systems raise great attention recently. As many dialogue models are data-driven, high-quality datasets are essential to these systems. In this paper, we introduce Pchatbot, a large-scale dialogue dataset that contains two subsets collected from Weibo and Judicial forums respectively. To adapt the raw dataset to dialogue systems, we elaborately normalize the raw dataset via processes such as anonymization, deduplication, segmentation, and filtering. The scale of Pchatbot is significantly larger than existing Chinese datasets, which might benefit the data-driven models.…
University of California, Santa Barbara（加州大学圣塔芭芭拉分校）；Tencent（腾讯）；Carnegie Mellon University（卡内基梅隆大学）
Neural conversational models have long suffered from the problem of inconsistency and lacking coherent personality. To address the issue, persona-based models capturing individual characteristics have been proposed, but they still face the dilemma of model adaption and data privacy. To break this dilemma, we propose a novel Federated Natural Language Generation (FedNLG) framework, which learns personalized representations from various dataset on distributed devices, and thus implements the personalized dialogue system efficiently and safely. FedNLG first pre-trains parameters of standard neura…
Maintaining consistent personas is essential for dialogue agents. Although tremendous advancements have been brought, the limited-scale of annotated persona-dense data are still barriers towards training robust and consistent persona-based dialogue models. In this work, we show how the challenges can be addressed by disentangling persona-based dialogue generation into two sub-tasks with a novel BERT-over-BERT (BoB) model. Specifically, the model consists of a BERT-based encoder and two BERT-based decoders, where one decoder is for response generation, and another is for consistency understandi…
The Hong Kong University of Science and Technology（香港科技大学）
It is difficult to train a personalized task-oriented dialogue system because the data collected from each individual is often insufficient. Personalized dialogue systems trained on a small dataset can overfit and make it difficult to adapt to different user needs. One way to solve this problem is to consider a collection of multiple users' data as a source domain and an individual user's data as a target domain, and to perform a transfer learning from the source to the target domain. By following this idea, we propose "PETAL"(PErsonalized Task-oriented diALogue), a transfer-learning framework…
Dialogue systems without consistent responses are not fascinating. In this study, we build a dialogue system that can respond based on a given character setting (persona) to bring consistency. Considering the trend of the rapidly increasing scale of language models, we propose an approach that uses prompt-tuning, which has low learning costs, on pre-trained large-scale language models. The results of automatic and manual evaluations in English and Japanese show that it is possible to build a dialogue system with more natural and personalized responses using less computational resources than fi…
MIRACLE: Towards Personalized Dialogue Generation with Latent-Space Multiple Personal Attribute Control
Huazhong University of Science and Technology（华中科技大学）；Ping An Property & Casualty Insurance Company of China, Ltd.（中国平安财产保险股份有限公司）；Beijing Institute of Technology（北京理工大学）；Brilliance Technology Co. Ltd.（华晨科技股份有限公司）
