arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2012.11099cs.CLcs.AI
收藏

基于自定义预训练的多圈响应选择图推理网络

A Graph Reasoning Network for Multi-turn Response Selection via Customized Pre-training

Northeastern University(东北大学);Alibaba Group(阿里巴巴集团)

Yongkang Liu, Shi Feng, Daling Wang, Kaisong Song, Feiliang Ren, Yifei Zhang

摘要:

我们研究了基于检索的聊天机器人中多轮对话的响应选择。现有的研究更多地关注话语与应答之间的匹配,基于学习到的特征计算匹配分数,导致模型推理能力不足。在本文中,我们提出了一个图推理网络(GRN)来解决这个问题。GRN首先基于ALBERT进行预训练,使用专门为响应选择设计的下一个话语预测和话语顺序预测任务。这两个定制的预训练任务可以赋予我们的模型捕捉话语之间的语义和时间依赖的能力。然后,我们微调的模型与序列推理和图形推理结构的集成网络。序列推理模块从全局的角度出发,基于高度概括的话语-应答对的上下文向量进行推理。图推理模块从局部角度对话语级图神经网络进行推理。在两个会话推理数据集上的实验表明,我们的模型可以显着优于强基线方法,并可以达到接近人类水平的性能。

英文摘要:

We investigate response selection for multi-turn conversation in retrieval-based chatbots. Existing studies pay more attention to the matching between utterances and responses by calculating the matching score based on learned features, leading to insufficient model reasoning ability. In this paper, we propose a graph-reasoning network (GRN) to address the problem. GRN first conducts pre-training based on ALBERT using next utterance prediction and utterance order prediction tasks specifically devised for response selection. These two customized pre-training tasks can endow our model with the ability of capturing semantical and chronological dependency between utterances. We then fine-tune the model on an integrated network with sequence reasoning and graph reasoning structures. The sequence reasoning module conducts inference based on the highly summarized context vector of utterance-response pairs from the global perspective. The graph reasoning module conducts the reasoning on the utterance-level graph neural network from the local perspective. Experiments on two conversational reasoning datasets show that our model can dramatically outperform the strong baseline methods and can achieve performance which is close to human-level.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑