去除伪特征可能会损害准确性并影响组不均衡
Removing Spurious Features can Hurt Accuracy and Affect Groups Disproportionately
摘要:
虚假特征的存在干扰了获得在人群中的许多组中表现良好的鲁棒模型的目标。一个自然的补救办法是从模型中删除虚假的特征。然而,在这项工作中,我们表明,去除虚假的功能,可以降低准确性,由于过度参数化模型的归纳偏见。我们完全描述了去除虚假特征如何影响无噪声过参数化线性回归中不同群体(更一般地说,测试分布)的准确性。此外,我们还表明,即使在平衡的数据集中,去除虚假特征也会降低准确性-每个目标与每个虚假特征同时出现;并且它会无意中使模型更容易受到其他虚假特征的影响。最后,我们证明了鲁棒的自训练可以在不影响整体准确性的情况下去除虚假特征。在Toxic-Comment-Detectoin和CelebA数据集上的实验表明,我们的结果在非线性模型中成立。
英文摘要:
The presence of spurious features interferes with the goal of obtaining robust models that perform well across many groups within the population. A natural remedy is to remove spurious features from the model. However, in this work we show that removal of spurious features can decrease accuracy due to the inductive biases of overparameterized models. We completely characterize how the removal of spurious features affects accuracy across different groups (more generally, test distributions) in noiseless overparameterized linear regression. In addition, we show that removal of spurious feature can decrease the accuracy even in balanced datasets -- each target co-occurs equally with each spurious feature; and it can inadvertently make the model more susceptible to other spurious features. Finally, we show that robust self-training can remove spurious features without affecting the overall accuracy. Experiments on the Toxic-Comment-Detectoin and CelebA datasets show that our results hold in non-linear models.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。