通过最小化最大期望损失来重新加权增强样本
Reweighting Augmented Samples by Minimizing the Maximal Expected Loss
摘要：
数据增强是提高深度神经网络泛化能力的有效技术。然而，以往的数据增强方法通常对增强后的样本一视同仁，没有考虑它们对模型的影响。为了解决这个问题，对于来自相同训练样本的增强样本，我们建议为它们分配不同的权重。我们构造了最大期望损失，这是增广样本上任何重新加权损失的上确界。受对抗训练的启发，我们最小化了这种最大预期损失（MMEL），并获得了一个简单且可解释的封闭形式解决方案：应该更多地关注具有大损失值的增强样本（即，更难的例子）。最小化这个最大预期损失使模型能够在任何重新加权策略下表现良好。所提出的方法通常可以应用于任何数据增强方法之上。实验进行自然语言理解任务与令牌级数据增强，和图像分类任务与常用的图像增强技术，如随机裁剪和水平翻转。实验结果表明，该方法提高了模型的泛化性能。
英文摘要：
Data augmentation is an effective technique to improve the generalization of deep neural networks. However, previous data augmentation methods usually treat the augmented samples equally without considering their individual impacts on the model. To address this, for the augmented samples from the same training example, we propose to assign different weights to them. We construct the maximal expected loss which is the supremum over any reweighted loss on augmented samples. Inspired by adversarial training, we minimize this maximal expected loss (MMEL) and obtain a simple and interpretable closed-form solution: more attention should be paid to augmented samples with large loss values (i.e., harder examples). Minimizing this maximal expected loss enables the model to perform well under any reweighting strategy. The proposed method can generally be applied on top of any data augmentation methods. Experiments are conducted on both natural language understanding tasks with token-level data augmentation, and image classification tasks with commonly-used image augmentation techniques like random crop and horizontal flip. Empirical results show that the proposed method improves the generalization performance of the model.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
