Frozen pretrained features are often treated as a safe interface for downstream learning: only a small linear readout is trained, while the backbone is fixed. We show that this readout can still overfit noisy labels in a structured way. A label-blind PCA rank sweep reveals a sharp spectral pattern: under label noise, exposing all pretrained directions can hurt clean accuracy, and intermediate ranks often recover much of the lost performance. Rank-matched random projections help less, and measured between-class signal is strongly concentrated in leading PCs. The pattern appears across three ImageNet-pretrained backbones on CIFAR-10, with gains up to $36.0\pm0.8$ points over the default full-rank probe at 40\% noise. Tuned full-rank probes outperform validation-selected PCA probes, so we present the sweep as a diagnostic of spectral overfitting rather than a competitive noisy-label method.
Transductive Linear Probing: A Novel Framework for Few-Shot Node Classification
Arizona State University(亚利桑那州立大学);University of Virginia(弗吉尼亚大学)
Few-Shot节点分类的任务是提供准确的预测节点从新的类只有几个代表性的标记节点。这个问题已经引起了极大的关注,因为它的投影到流行的现实世界的应用,如产品分类的新添加的商品类别的电子商务平台上的稀缺记录或罕见疾病的诊断上的患者相似性图。为了解决非欧几里德图域中具有挑战性的标签稀缺问题,元学习已经成为一种成功和主导的范式。最近,受图自监督学习发展的启发,为Few-Shot节点分类转移预训练的节点嵌入可能是元学习的一个有前途的替代方案,但尚未公开。在这项工作中,我们经验性地证明了另一种框架\textit{Transductive Linear Probing}的潜力,该框架可以传输从图对比学习方法中学习的预训练节点嵌入。我们进一步将Few-Shot节点分类的设置从标准的完全监督扩展到更现实的自监督设置,其中由于缺乏来自训练类的监督,元学习方法无法轻松部署。令人惊讶的是,即使没有任何地面事实标签,具有自监督图对比预训练的直推线性探测也可以在相同协议下优于最先进的完全监督元学习方法。我们希望这项工作可以揭示新的光在Few-Shot节点分类问题,并促进未来的研究,从几乎没有标记的实例图的学习。
Optimal Representation Size: High-Dimensional Analysis of Pretraining and Linear Probing
Gatsby Computational Neuroscience Unit(Gatsby计算神经科学单元);University College London(伦敦大学学院);Institute of Science and Technology Austria(奥地利科学与技术研究所);Sainsbury Wellcome Centre(萨金斯-韦尔科姆中心)
Linear probing enables Ship-Radiated Noise recognition with pretrained audio embeddings
Centre of Mathematics and Computer Science, Stochastics group(数学与计算机科学中心,随机组);Vrije Universiteit, Department of Mathematics(自由大学,数学系);Naturalis Biodiversity Center(自然多样性中心)
LiFT: Local Search via Linear Programming for Overfitting-Controlled Transformers
Department of Management Sciences, Indian Institute of Technology Kanpur(印度理工学院坎普尔分校管理科学系);Department of Civil Engineering, Indian Institute of Technology Kanpur(印度理工学院坎普尔分校土木工程系);Operations and Decision Sciences, Indian Institute of Management Ahmedabad(印度管理学院艾哈迈达巴德分校运营与决策科学系);Brij Disa Centre for Data Science and AI, Indian Institute of Management Ahmedabad(印度管理学院艾哈迈达巴德分校Brij Disa数据科学与人工智能中心)