通过主动生成成对反事实提升分类器鲁棒性
Improving Classifier Robustness through Active Generation of Pairwise Counterfactuals
AI总结：
提出利用反事实生成模型从不确定区域主动采样生成反事实，并用成对分类器自动标注，仅需10%人工标注即可显著提升分类器鲁棒性并降低错误率。
摘要：
反事实数据增强（CDA）是提升自然语言分类器鲁棒性的常用技术。然而，一个根本挑战是如何发现有意义的反事实并高效标注它们，同时最小化人工标注成本。现有方法要么完全依赖人工标注标签，这一昂贵过程限制了反事实数据的规模；要么隐式假设标签不变性，这可能用错误标签误导模型。在本文中，我们提出了一个新颖框架，利用反事实生成模型通过主动从不确定区域采样生成大量多样的反事实，然后用学习到的成对分类器自动标注它们。我们的关键洞察是，通过训练一个插值原始样本与反事实之间关系的成对分类器，我们可以更正确地标注生成的反事实。我们证明，仅需少量人工标注的反事实数据（10%），我们就能生成带有学习标签的反事实增强数据集，在6个域外数据集上提供18-20%的鲁棒性提升和14-21%的错误率降低，在情感分类和问题复述任务上均与完全人工标注的反事实数据集效果相当。
英文摘要：
Counterfactual Data Augmentation (CDA) is a commonly used technique for improving robustness in natural language classifiers. However, one fundamental challenge is how to discover meaningful counterfactuals and efficiently label them, with minimal human labeling cost. Most existing methods either completely rely on human-annotated labels, an expensive process which limits the scale of counterfactual data, or implicitly assume label invariance, which may mislead the model with incorrect labels. In this paper, we present a novel framework that utilizes counterfactual generative models to generate a large number of diverse counterfactuals by actively sampling from regions of uncertainty, and then automatically label them with a learned pairwise classifier. Our key insight is that we can more correctly label the generated counterfactuals by training a pairwise classifier that interpolates the relationship between the original example and the counterfactual. We demonstrate that with a small amount of human-annotated counterfactual data (10%), we can generate a counterfactual augmentation dataset with learned labels, that provides an 18-20% improvement in robustness and a 14-21% reduction in errors on 6 out-of-domain datasets, comparable to that of a fully human-annotated counterfactual dataset for both sentiment classification and question paraphrase tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
