arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2310.03518cs.CLcs.AIcs.DS
收藏

面向鲁棒性与泛化性训练:针对输入扰动的噪声槽填充实证研究

Towards Robust and Generalizable Training: An Empirical Study of Noisy Slot Filling for Input Perturbations

Beijing University of Posts and Telecommunications(北京邮电大学);Huazhong University of Science and Technology(华中科技大学);University of Surrey(萨里大学);Meituan Group(美团集团)

Jiachi Liu, Liwen Wang, Guanting Dong, Xiaoshuai Song, Zechen Wang, Zhengyang Wang, Shanglin Lei, Jinzheng Zhao, Keqing He, Bo Xiao, Weiran Xu

AI总结:

针对现有槽填充模型抗输入噪声能力弱、鲁棒性研究受限于规则合成数据集的问题,本文构建含五类真实人工标注噪声的Noise-SF数据集,整合鲁棒训练方法并通过实验验证了框架的有效性,还提出了前瞻性研究建议。

摘要:

在真实对话场景中,由于话语中存在未知输入噪声,现有监督式槽填充模型在实际应用中往往表现不佳。尽管已有部分关于噪声鲁棒模型的研究,但这些工作仅在基于规则的合成数据集上进行评估,存在局限性,难以推动噪声鲁棒方法的研究。本文提出了一个面向槽填充任务的噪声鲁棒性评估数据集Noise-SF。该数据集包含五种人工标注的噪声类型,且所有噪声均真实存在于实际场景中;本文还将大量槽填充鲁棒训练方法纳入所提出的框架。通过在Noise-SF上开展全面的实证评估实验,我们发现基线模型的鲁棒性评估表现较差,而所提框架能有效提升模型的鲁棒性。基于实证实验结果,我们提出了若干前瞻性建议,以推动该方向的研究。我们的数据集Noise-SF将发布于https://github.com/dongguanting/Noise-SF。

英文摘要:

In real dialogue scenarios, as there are unknown input noises in the utterances, existing supervised slot filling models often perform poorly in practical applications. Even though there are some studies on noise-robust models, these works are only evaluated on rule-based synthetic datasets, which is limiting, making it difficult to promote the research of noise-robust methods. In this paper, we introduce a noise robustness evaluation dataset named Noise-SF for slot filling task. The proposed dataset contains five types of human-annotated noise, and all those noises are exactly existed in real extensive robust-training methods of slot filling into the proposed framework. By conducting exhaustive empirical evaluation experiments on Noise-SF, we find that baseline models have poor performance in robustness evaluation, and the proposed framework can effectively improve the robustness of models. Based on the empirical experimental results, we make some forward-looking suggestions to fuel the research in this direction. Our dataset Noise-SF will be released at https://github.com/dongguanting/Noise-SF.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑