使用特征筛克服深度网络中的简单性偏差
Overcoming Simplicity Bias in Deep Networks using a Feature Sieve
AI总结：
本文提出 feature sieve，通过自动识别并抑制网络低层虚假简单特征、增强高层有效表示来缓解 DNN 的简单性偏差，并在多个去偏基准上无需先验信息即取得显著提升。
摘要：
简单性偏差是指深度网络过度依赖简单但预测能力较弱的特征，而排除更强、更复杂特征的令人担忧的倾向。在现实应用中，有限的训练数据和虚假的特征—标签关联会加剧这一问题，从而导致有偏差且错误的预测。我们提出一种直接的干预式方法来解决 DNN 中的简单性偏差，称之为 feature sieve（特征筛）。我们的目标是自动识别并抑制网络较低层中易于计算的虚假特征，从而使网络较高层能够提取并利用更丰富、更有意义的表示。我们在受控数据集和真实世界图像上提供了这种对相关特征进行差异化抑制与增强的具体证据，并报告了在许多现实世界去偏基准上的显著提升（Imagenet-A 上相对提升 11.4%；BAR 上为 3.2%，等等）。关键在于，我们不依赖关于虚假属性或特征的先验知识，而且实际上优于许多显式纳入此类信息的基线方法。我们相信，关于 feature sieve 的工作为深度网络的自动化对抗特征提取和表示学习开辟了令人振奋的新研究方向。
英文摘要：
Simplicity bias is the concerning tendency of deep networks to over-depend on simple, weakly predictive features, to the exclusion of stronger, more complex features. This is exacerbated in real-world applications by limited training data and spurious feature-label correlations, leading to biased, incorrect predictions. We propose a direct, interventional method for addressing simplicity bias in DNNs, which we call the feature sieve. We aim to automatically identify and suppress easily-computable spurious features in lower layers of the network, thereby allowing the higher network levels to extract and utilize richer, more meaningful representations. We provide concrete evidence of this differential suppression & enhancement of relevant features on both controlled datasets and real-world images, and report substantial gains on many real-world debiasing benchmarks (11.4% relative gain on Imagenet-A; 3.2% on BAR, etc). Crucially, we do not depend on prior knowledge of spurious attributes or features, and in fact outperform many baselines that explicitly incorporate such information. We believe that our feature sieve work opens up exciting new research directions in automated adversarial feature extraction and representation learning for deep networks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
