arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2301.12400cs.AI
收藏

HeroNet:一种用于会话机器人的混合检索生成网络

HeroNet: A Hybrid Retrieval-Generation Network for Conversational Bots

Harbin Institute of Technology(哈尔滨工业大学)

Bolin Zhang, Yunzhe Xu, Zhiying Tu, Dianhui Chu

摘要:

使用自然语言,会话机器人提供了前所未有的方式,许多领域的挑战,如信息搜索,项目推荐,和问答。现有的机器人通常是通过基于检索或基于生成的方法开发的,但这两种方法都有自己的优点和缺点。为了将这两种方法组合起来,我们提出了一个混合检索生成网络(HeroNet),它具有三个方面的思想:1)。为了生成高质量的句子表示,HeroNet在两个子任务上执行多任务学习:相似搜索发现和查询-响应匹配。具体来说,检索性能得到提高,而模型的大小是通过训练两个轻量级的,特定于任务的适配器模块,共享只有一个底层的T5编码器模型减少。2)。通过引入对抗训练,HeroNet能够同时解决检索和生成任务,同时最大限度地提高彼此的性能。3)。检索结果作为先验知识用于提高生成性能,生成结果由检索器进行评分,并将评分值集成到生成器的交叉熵损失函数中。在开放数据集上的实验结果证明了HeroNet的有效性,我们的代码可以在https://github.com/TempHero/HeroNet.git上获得

英文摘要:

Using natural language, Conversational Bot offers unprecedented ways to many challenges in areas such as information searching, item recommendation, and question answering. Existing bots are usually developed through retrieval-based or generative-based approaches, yet both of them have their own advantages and disadvantages. To assemble this two approaches, we propose a hybrid retrieval-generation network (HeroNet) with the three-fold ideas: 1). To produce high-quality sentence representations, HeroNet performs multi-task learning on two subtasks: Similar Queries Discovery and Query-Response Matching. Specifically, the retrieval performance is improved while the model size is reduced by training two lightweight, task-specific adapter modules that share only one underlying T5-Encoder model. 2). By introducing adversarial training, HeroNet is able to solve both retrieval\&generation tasks simultaneously while maximizing performance of each other. 3). The retrieval results are used as prior knowledge to improve the generation performance while the generative result are scored by the discriminator and their scores are integrated into the generator's cross-entropy loss function. The experimental results on a open dataset demonstrate the effectiveness of the HeroNet and our code is available at https://github.com/TempHero/HeroNet.git

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑