基于虚拟对抗离散扰动的一致性训练
Consistency Training with Virtual Adversarial Discrete Perturbation
摘要：
一致性训练通过强制原始输入和扰动输入的预测相似来正则化模型。以前的研究已经提出了各种增强方法的扰动，但有限的，因为它们是不可知的训练模型。因此，扰动样本可能不会帮助正则化，因为它们易于从模型中分类。在这种情况下，我们提出了一种增加离散噪声的增强方法，该方法将导致预测之间的最高分歧。这种虚拟对抗性离散噪声通过替换一小部分标记而获得，同时尽可能多地保持原始语义，从而有效地推动训练模型的决策边界。实验结果表明，我们提出的方法优于其他一致性训练基线与文本编辑，释义，或连续噪声的半监督文本分类任务和鲁棒性基准
英文摘要：
Consistency training regularizes a model by enforcing predictions of original and perturbed inputs to be similar. Previous studies have proposed various augmentation methods for the perturbation but are limited in that they are agnostic to the training model. Thus, the perturbed samples may not aid in regularization due to their ease of classification from the model. In this context, we propose an augmentation method of adding a discrete noise that would incur the highest divergence between predictions. This virtual adversarial discrete noise obtained by replacing a small portion of tokens while keeping original semantics as much as possible efficiently pushes a training model's decision boundary. Experimental results show that our proposed method outperforms other consistency training baselines with text editing, paraphrasing, or a continuous noise on semi-supervised text classification tasks and a robustness benchmark
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
