arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1702.08563cs.CL
收藏

软标记推理--自然语言推理的推广

Soft Label Memorization-Generalization for Natural Language Inference

University of Massachusetts(马萨诸塞大学);Vanderbilt University(范德堡大学);Bedford VA(贝德福德退伍军人管理局)

John P. Lalor, Hao Wu, Hong Yu

摘要:

通常,当为训练样本获得多个标签时,假设存在必须考虑的噪声元素。已经表明,这种不一致可以被认为是信号而不是噪声。在这项工作中,我们研究使用软标签的训练数据,以提高泛化机器学习模型。然而,使用软标签来训练深度神经网络(DNN)是不切实际的,这是由于为大型数据集获得多个标签所涉及的成本。我们提出了软标签记忆泛化(SLMG),这是一种使用软标签训练DNN的微调方法。我们假设由人类注释者提供的标签的差异表示关于真实标签而不是噪声的模糊性。SLMG的实验表明,改进的泛化性能的自然语言推理(NLI)任务。我们的实验表明,通过注入一小部分的软标签训练数据(0.03%的训练集大小),我们可以提高泛化性能超过几个基线。

英文摘要:

Often when multiple labels are obtained for a training example it is assumed that there is an element of noise that must be accounted for. It has been shown that this disagreement can be considered signal instead of noise. In this work we investigate using soft labels for training data to improve generalization in machine learning models. However, using soft labels for training Deep Neural Networks (DNNs) is not practical due to the costs involved in obtaining multiple labels for large data sets. We propose soft label memorization-generalization (SLMG), a fine-tuning approach to using soft labels for training DNNs. We assume that differences in labels provided by human annotators represent ambiguity about the true label instead of noise. Experiments with SLMG demonstrate improved generalization performance on the Natural Language Inference (NLI) task. Our experiments show that by injecting a small percentage of soft label training data (0.03% of training set size) we can improve generalization performance over several baselines.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑