长尾图像分类的小批量采样策略：CIFAR-100-LT 上的实证研究
Mini-batch Sampling Strategies for Long-Tailed Image Classification: An Empirical Study on CIFAR-100-LT
浏览论文内容
中文总结 AI 辅助
本研究系统比较四种小批量采样策略，发现渐进均衡采样在长尾分类中稳健提升尾部准确率，而重平衡时机与程度同等重要。
中文摘要 AI 辅助
真实世界的数据集通常呈现长尾类别分布，其中少数头部类别包含大量训练样本，而大量尾部类别仅有少量样本。由采样策略决定的每个小批量的组成，控制了哪些类别对随机梯度估计做出贡献，因此影响整个类别谱系上的收敛行为和泛化能力。我们对用于长尾图像分类的四种小批量采样策略进行了系统的理论和实证比较：均匀实例采样、类别均衡采样、平方根采样和渐进均衡采样。我们将这四种策略置于一个统一的偏差-方差框架中，描述它们对梯度估计的影响，该框架揭示了经验损失的无偏优化与稀有类别公平表示之间的张力。然后，我们使用 ResNet-32 在 CIFAR-100-LT 上，在三个不平衡比率（rho = 10, 50, 100）下，于受控条件下评估这些策略，每种策略在同一个种子内共享相同的长尾子集和初始化。在 rho = 100 时，渐进采样相对于均匀基线将尾部类别准确率提高了 25%（13.5% 对比 10.8%），并且在所有三个种子上结果一致，而其总体准确率与均匀采样在种子间变异下没有显著差异（40.0% 对比 39.7%）；尾部类别的提升，而非总体提升，是稳健的效果。在 rho = 100 时，类别均衡采样在每一类别的准确率上均有所下降，包括它旨在帮助的尾部类别，我们将此归因于对稀缺数据的极端过采样导致的过拟合；在 rho = 50 时，这种失败仅限于头部和中等类别。这些结果表明，在训练过程中何时应用重平衡与重平衡的程度同样重要。
英文摘要
Real-world datasets often exhibit long-tailed class distributions, where a few head classes contain a large number of training samples while a large number of tail classes have only a few. The composition of each mini-batch, determined by the sampling strategy, governs which classes contribute to the stochastic gradient estimate, and therefore affects convergence behaviour and generalisation across the whole class spectrum. We provide a systematic theoretical and empirical comparison of four mini-batch sampling strategies for long-tailed image classification: uniform instance sampling, class-balanced sampling, square-root sampling, and progressively balanced sampling. We place all four in a unified bias-variance framework describing their effect on gradient estimation, which exposes the tension between unbiased optimisation of the empirical loss and fair representation of rare classes. We then evaluate them under controlled conditions using ResNet-32 on CIFAR-100-LT at three imbalance ratios (rho = 10, 50, 100), with every strategy sharing the same long-tailed subsets and initialisation within a seed. Progressive sampling improves tail-class accuracy by 25% relative to the uniform baseline at rho = 100 (13.5% versus 10.8%), consistently across all three seeds, while its overall accuracy is not distinguishable from that of uniform sampling given the seed-to-seed variation (40.0% versus 39.7%); the tail-class gain, not the overall gain, is the robust effect. At rho = 100, class-balanced sampling degrades accuracy on every class group, including the tail classes it is designed to help, which we attribute to overfitting caused by extreme oversampling of scarce data; at rho = 50 this failure is confined to head and medium classes. These results indicate that when rebalancing is applied during training matters as much as how much rebalancing is applied.
发表机构
- University of Bristol（布里斯托大学）
机构由 AI 辅助整理，请以论文原文为准。
