DeNetDM：通过网络深度调制进行去偏
DeNetDM: Debiasing by Network Depth Modulation
AI总结：
DeNetDM通过利用网络深度作为隐式正则化器，基于专家乘积训练范式构建偏置与去偏分支并蒸馏知识，无需偏置标注或数据增强，在合成和真实数据集上超越现有去偏方法5%。
摘要：
在偏置数据集上训练的神经网络往往会无意中学习到虚假的相关性，从而阻碍泛化。我们正式证明了：(1) 表现出虚假相关性的样本位于相对于不具有此类相关性的样本更低秩的流形上；(2) 网络的深度充当了对其表示中编码的属性子空间秩的隐式正则化器。利用这些见解，我们提出了DeNetDM，一种新颖的去偏方法，它使用网络深度调制作为发展对虚假相关性鲁棒性的一种方式。利用源自专家乘积（Product of Experts）的训练范式，我们创建了具有深层和浅层架构的偏置分支和去偏分支，然后通过知识蒸馏来产生目标去偏模型。我们的方法不需要偏置标注或显式数据增强，同时性能与需要其中一种或两种的方法相当。我们证明，DeNetDM在合成和真实世界数据集上均优于现有去偏技术，性能提升5%。项目页面可在https://vssilpa.github.io/denetdm/获取。
英文摘要：
Neural networks trained on biased datasets tend to inadvertently learn spurious correlations, hindering generalization. We formally prove that (1) samples that exhibit spurious correlations lie on a lower rank manifold relative to the ones that do not; and (2) the depth of a network acts as an implicit regularizer on the rank of the attribute subspace that is encoded in its representations. Leveraging these insights, we present DeNetDM, a novel debiasing method that uses network depth modulation as a way of developing robustness to spurious correlations. Using a training paradigm derived from Product of Experts, we create both biased and debiased branches with deep and shallow architectures and then distill knowledge to produce the target debiased model. Our method requires no bias annotations or explicit data augmentation while performing on par with approaches that require either or both. We demonstrate that DeNetDM outperforms existing debiasing techniques on both synthetic and real-world datasets by 5\%. The project page is available at https://vssilpa.github.io/denetdm/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
