arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2003.04679cs.CLcs.CVcs.MM
收藏

学会用贴纸回应:一个统一多轮对话中多模态的框架

Learning to Respond with Stickers: A Framework of Unifying Multi-Modality in Multi-Turn Dialog

WICT, Peking University(北京大学王选计算机研究所);Inception Institute of Artificial Intelligence(Inception人工智能研究所);Beijing Academy of Artificial Intelligence(北京人工智能研究院)

Shen Gao, Xiuying Chen, Chang Liu, Li Liu, Dongyan Zhao, Rui Yan

摘要:

具有生动和吸引人的表情的贴纸在在线消息应用中变得越来越受欢迎,并且一些作品致力于通过将贴纸的文本标签与先前的话语进行匹配来自动选择贴纸响应。然而,由于其数量庞大,要求所有贴纸都有文本标签是不切实际的。因此,在本文中,我们建议推荐一个合适的贴纸给用户的基础上,多轮对话上下文历史没有任何外部标签。这项任务面临两个主要挑战。一种是学习没有相应文本标签的贴纸的语义。另一个挑战是将候选贴纸与多轮对话上下文联合建模。为了应对这些挑战,我们提出了一个标签响应选择器(SRS)模型。具体地,SRS首先采用基于卷积的贴纸图像编码器和基于自我注意的多轮对话编码器来获得贴纸和话语的表示。然后,提出了深度交互网络,用于在贴纸与对话历史中的每个话语之间进行深度匹配。SRS通过融合网络学习所有交互结果之间的短期和长期依赖关系,输出最终的匹配分数。为了评估我们提出的方法,我们收集了一个大规模的真实世界的对话数据集与贴纸从一个最流行的在线聊天平台。在该数据集上进行的大量实验表明,我们的模型在所有常用指标上都达到了最先进的性能。实验也验证了SRS各组成部分的有效性。为了便于在贴纸选择领域的进一步研究,我们发布了340K多回合对话和贴纸对的数据集。

英文摘要:

Stickers with vivid and engaging expressions are becoming increasingly popular in online messaging apps, and some works are dedicated to automatically select sticker response by matching text labels of stickers with previous utterances. However, due to their large quantities, it is impractical to require text labels for the all stickers. Hence, in this paper, we propose to recommend an appropriate sticker to user based on multi-turn dialog context history without any external labels. Two main challenges are confronted in this task. One is to learn semantic meaning of stickers without corresponding text labels. Another challenge is to jointly model the candidate sticker with the multi-turn dialog context. To tackle these challenges, we propose a sticker response selector (SRS) model. Specifically, SRS first employs a convolutional based sticker image encoder and a self-attention based multi-turn dialog encoder to obtain the representation of stickers and utterances. Next, deep interaction network is proposed to conduct deep matching between the sticker with each utterance in the dialog history. SRS then learns the short-term and long-term dependency between all interaction results by a fusion network to output the the final matching score. To evaluate our proposed method, we collect a large-scale real-world dialog dataset with stickers from one of the most popular online chatting platform. Extensive experiments conducted on this dataset show that our model achieves the state-of-the-art performance for all commonly-used metrics. Experiments also verify the effectiveness of each component of SRS. To facilitate further research in sticker selection field, we release this dataset of 340K multi-turn dialog and sticker pairs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑