arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2303.08599cs.CLcs.AI
收藏

基于高斯过程的可靠对话响应检索的有效不确定性估计

Efficient Uncertainty Estimation with Gaussian Process for Reliable Dialog Response Retrieval

Ping An Technology (Shenzhen) Co., Ltd.(平安科技(深圳)有限公司);University of Science and Technology of China(中国科学技术大学)

Tong Ye, Zhitao Li, Jianzong Wang, Ning Cheng, Jing Xiao

摘要:

深度神经网络在基于检索的对话系统中取得了卓越的性能,但它们被证明是校准不良的。虽然像Monte Carlo Dropout和Entrance这样的基本校准方法可以很好地校准,但这些方法在训练或推理阶段非常耗时。为了应对这些挑战,我们提出了一种用于基于BERT的对话式搜索的高效不确定性校准框架GPF-BERT,该框架在BERT架构之上采用高斯过程层和焦点损失来实现高质量的神经排序器。大量的实验验证了该方法的有效性。与基本标定方法相比,GPF-BERT在三个域内数据集和分布移位任务中实现了最低的经验标定误差(ECE),而在大多数情况下产生了最高的$R_{10}@1$和MAP性能。在时间消耗方面,我们的GPF-BERT具有8 $\times$的加速比。

英文摘要:

Deep neural networks have achieved remarkable performance in retrieval-based dialogue systems, but they are shown to be ill calibrated. Though basic calibration methods like Monte Carlo Dropout and Ensemble can calibrate well, these methods are time-consuming in the training or inference stages. To tackle these challenges, we propose an efficient uncertainty calibration framework GPF-BERT for BERT-based conversational search, which employs a Gaussian Process layer and the focal loss on top of the BERT architecture to achieve a high-quality neural ranker. Extensive experiments are conducted to verify the effectiveness of our method. In comparison with basic calibration methods, GPF-BERT achieves the lowest empirical calibration error (ECE) in three in-domain datasets and the distributional shift tasks, while yielding the highest $R_{10}@1$ and MAP performance on most cases. In terms of time consumption, our GPF-BERT has an 8$\times$ speedup.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑