生成分类器避免捷径解法
Generative Classifiers Avoid Shortcut Solutions
浏览论文内容
中文总结 AI 辅助
生成分类器通过建模所有特征避免捷径解法，提升在分布偏移下的性能。
中文摘要 AI 辅助
判别方法在分类中常常学习出在分布内有效的捷径，但即使在微小分布偏移下也会失败。这种失败模式源于对与标签 spuriously 相关的特征过度依赖。我们证明生成分类器，它们使用类条件生成模型，可以通过建模所有特征，包括核心和 spurious 特征，而不是主要 spurious 特征，从而避免这一问题。这些生成分类器易于训练，避免了需要专门的增强、强正则化、额外超参数或对特定 spurious 相关性知识的需要。我们发现基于扩散和自回归的生成分类器在五个标准图像和文本分布偏移基准上实现了最先进的性能，并减少了实际应用中的 spurious 相关性影响，如医学或卫星数据集。最后，我们仔细分析了一个高斯玩具设置，以理解生成分类器的归纳偏置，以及决定生成分类器优于判别分类器的数据属性。
英文摘要
Discriminative approaches to classification often learn shortcuts that hold in-distribution but fail even under minor distribution shift. This failure mode stems from an overreliance on features that are spuriously correlated with the label. We show that generative classifiers, which use class-conditional generative models, can avoid this issue by modeling all features, both core and spurious, instead of mainly spurious ones. These generative classifiers are simple to train, avoiding the need for specialized augmentations, strong regularization, extra hyperparameters, or knowledge of the specific spurious correlations to avoid. We find that diffusion-based and autoregressive generative classifiers achieve state-of-the-art performance on five standard image and text distribution shift benchmarks and reduce the impact of spurious correlations in realistic applications, such as medical or satellite datasets. Finally, we carefully analyze a Gaussian toy setting to understand the inductive biases of generative classifiers, as well as the data properties that determine when generative classifiers outperform discriminative ones.
发表机构
- Carnegie Mellon University（卡内基梅隆大学）
- Stanford University（斯坦福大学）
机构由 AI 辅助整理，请以论文原文为准。
