arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2109.04746cs.LG
收藏

基于表示插值的反事实对抗学习

Counterfactual Adversarial Learning with Representation Interpolation

Alibaba Group(阿里巴巴集团);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)

Wei Wang, Boxin Wang, Ning Shi, Jinfeng Li, Bingyu Zhu, Xiangyu Liu, Rong Zhang

摘要:

与逻辑推理相比,深度学习模型更倾向于统计拟合。当训练数据中存在统计偏差时,可能会记忆虚假的相关性,这严重限制了模型的性能,特别是在小数据场景中。在这项工作中,我们引入了反事实对抗训练框架(CAT),从因果关系的角度来解决这个问题。特别地,对于特定样本,CAT首先以对抗的方式通过潜在空间插值生成反事实表示,然后对每个原始-反事实对执行反事实风险最小化(CRM),以动态调整样本损失权重,这鼓励模型探索真实的因果效应。大量的实验表明,CAT在不同的下游任务,包括句子分类,自然语言推理和问答,实现了显着的性能改善SOTA。

英文摘要:

Deep learning models exhibit a preference for statistical fitting over logical reasoning. Spurious correlations might be memorized when there exists statistical bias in training data, which severely limits the model performance especially in small data scenarios. In this work, we introduce Counterfactual Adversarial Training framework (CAT) to tackle the problem from a causality perspective. Particularly, for a specific sample, CAT first generates a counterfactual representation through latent space interpolation in an adversarial manner, and then performs Counterfactual Risk Minimization (CRM) on each original-counterfactual pair to adjust sample-wise loss weight dynamically, which encourages the model to explore the true causal effect. Extensive experiments demonstrate that CAT achieves substantial performance improvement over SOTA across different downstream tasks, including sentence classification, natural language inference and question answering.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑