arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2006.10032cs.LGstat.ML
收藏

自训练避免在域偏移下使用虚假特征

Self-training Avoids Using Spurious Features Under Domain Shift

Stanford University(斯坦福大学)

Yining Chen, Colin Wei, Ananya Kumar, Tengyu Ma

摘要:

在无监督的领域自适应中,现有的理论集中在源域和目标域接近的情况下。在实践中,条件熵最小化和伪标记的工作,即使在域的位移是远远大于现有的理论分析。我们确定并分析了一个特定的设置,其中域偏移可能很大,但这些算法可证明是有效的:某些虚假特征与源域中的标签相关,但与目标中的标签无关。我们的分析认为,线性分类的虚假功能是高斯和非虚假的功能是一个混合的对数凹分布。对于这种设置,我们证明,熵最小化未标记的目标数据将避免使用虚假的功能,如果初始化一个体面的准确的源分类,即使目标是非凸的,并包含多个坏的局部极小值使用虚假的功能。我们验证了我们的理论虚假域转移任务半合成Celeb-A和MNIST数据集。我们的研究结果表明,从业者收集和自我训练大型,多样化的数据集,以减少分类器的偏见,即使标签是不切实际的。

英文摘要:

In unsupervised domain adaptation, existing theory focuses on situations where the source and target domains are close. In practice, conditional entropy minimization and pseudo-labeling work even when the domain shifts are much larger than those analyzed by existing theory. We identify and analyze one particular setting where the domain shift can be large, but these algorithms provably work: certain spurious features correlate with the label in the source domain but are independent of the label in the target. Our analysis considers linear classification where the spurious features are Gaussian and the non-spurious features are a mixture of log-concave distributions. For this setting, we prove that entropy minimization on unlabeled target data will avoid using the spurious feature if initialized with a decently accurate source classifier, even though the objective is non-convex and contains multiple bad local minima using the spurious features. We verify our theory for spurious domain shift tasks on semi-synthetic Celeb-A and MNIST datasets. Our results suggest that practitioners collect and self-train on large, diverse datasets to reduce biases in classifiers even if labeling is impractical.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑