MPCHAT：迈向多模态人格基础的对话
MPCHAT: Towards Multimodal Persona-Grounded Conversation
AI总结：
本文提出首个多模态人格对话数据集MPCHAT，将文本与图像结合表征情景记忆，并通过三项任务验证多模态人格能显著提升对话理解性能。
摘要：
为了构建自洽的个性化对话智能体，以往研究主要关注传递个人事实或个性特征的文本人格。然而，为了充分描述人格的多面性，图像模态能够帮助更好地揭示说话者在情景记忆中的个人特征和经历（Rubin et al., 2003; Conway, 2009）。在本工作中，我们将基于人格的对话扩展到多模态领域，并做出两项主要贡献。首先，我们提出了首个多模态人格基础对话数据集MPCHAT，该数据集将人格扩展为同时包含文本和图像，以涵盖情景记忆。其次，我们通过三个提出的多模态人格基础对话任务（即下一响应预测、基础人格预测和说话人识别）进行实证评估，结果表明引入多模态人格在所有任务上均带来统计显著性的性能提升。因此，我们的工作强调多模态人格对于提升多模态对话理解至关重要，并且我们的MPCHAT为这一研究提供了高质量资源。
英文摘要：
In order to build self-consistent personalized dialogue agents, previous research has mostly focused on textual persona that delivers personal facts or personalities. However, to fully describe the multi-faceted nature of persona, image modality can help better reveal the speaker's personal characteristics and experiences in episodic memory (Rubin et al., 2003; Conway, 2009). In this work, we extend persona-based dialogue to the multimodal domain and make two main contributions. First, we present the first multimodal persona-based dialogue dataset named MPCHAT, which extends persona with both text and images to contain episodic memories. Second, we empirically show that incorporating multimodal persona, as measured by three proposed multimodal persona-grounded dialogue tasks (i.e., next response prediction, grounding persona prediction, and speaker identification), leads to statistically significant performance improvements across all tasks. Thus, our work highlights that multimodal persona is crucial for improving multimodal dialogue comprehension, and our MPCHAT serves as a high-quality resource for this research.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
