发表机构
Beijing Jiaotong University; AntGroup; Northwestern Polytechnical University; University of Southampton（北京交通大学; 蚂蚁集团; 西北工业大学; 南安普顿大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究针对长尾半监督学习中伪标签偏向多数类的问题，从动态学习视角揭示对数去偏机制，提出DyTrim动态剪枝框架，经实验验证可提升模型泛化性能。
AI 中文摘要
长尾分布在现实世界的半监督学习（SSL）中普遍存在，其中伪标签往往偏向多数类，导致泛化能力下降。尽管已提出多种长尾半监督学习（LTSSL）方法，但它们隐式对对数进行去偏的机制仍鲜为人知。本研究从动态学习的视角重新审视LTSSL，并对对数去偏进行理论刻画。具体而言，我们推导了对数更新的逐步分解，表明预测由类不平衡偏差主导，该偏差可靠地反映了标签先验。为揭示这一效应，我们使用与任务无关的基准图像的对数作为累积偏差的指标，并证明它们收敛于类先验。这提供了一个统一的视角，其中对数调整、重加权和重采样等LTSSL补救措施对应于重塑梯度动态。基于这一见解，我们提出DyTrim，一个基于原理的动态剪枝框架，通过对标记数据进行类感知剪枝和对未标记数据进行基于置信度的软剪枝来重新分配梯度预算。我们提供理论保证，证明DyTrim可减少类偏差并提高泛化能力。在标准LTSSL基准上的大量实验表明，该方法在不同架构和方法中均取得了一致的性能提升。代码可在此URL获取。
英文摘要
Long-tailed distributions are prevalent in real-world semi-supervised learning (SSL), where pseudo-labels tend to favor majority classes, leading to degraded generalization. While many long-tailed semi-supervised learning (LTSSL) methods have been proposed, the mechanisms by which they implicitly debias logits remain poorly understood. In this work, we revisit LTSSL through the lens of learning dynamics and provide a theoretical characterization of logits debiasing. Specifically, we derive a step-wise decomposition of the logits updates, showing that predictions are dominated by class-imbalance bias that reliably reflects label priors. To expose this effect, we use the logits of a task-irrelevant baseline image as an indicator of accumulated bias and prove that they converge to the class prior. This provides a unified view where LTSSL remedies such as logit adjustment, reweighting, and resampling correspond to reshaping gradient dynamics. Based on this insight, we propose DyTrim, a principle-based dynamic pruning framework that reallocates gradient budget through class-aware pruning on labeled data and confidence-based soft pruning on unlabeled data. We provide theoretical guarantees that DyTrim reduces class bias and improves generalization. Extensive experiments on standard LTSSL benchmarks show consistent gains across architectures and methods. Code available at: https://jiajun0425.github.io/DyTrim
Comments32 pages, 19 figures
Journal refInternational Conference on Learning Representations. 2026, 2026: 48426-48457
