在野外收集的训练数据往往包含噪声标签和异常值,这些会显著降低深度神经网络的性能和可靠性。虽然数据清洗通常被用作单独的预处理阶段,但这种两阶段流程既不能完全利用下游模型的反馈,也不能适应未知的噪声模式。我们提出了Step-E,一个简单的框架,将样本选择和模型学习整合到一个优化过程中。在每个epoch中,Step-E通过损失对样本进行排序,并逐渐增加高损失样本在梯度更新中被排除的比例,在短暂的预热阶段后,从而产生一个在线课程,专注于简单且一致的样本,最终忽略持续的异常值。在CIFAR-100N上,Step-E将ResNet-18模型的测试准确率从43.3%(±0.7%)提升到50.4%(±0.9%),明显优于损失截断、自 paced学习和单次过滤,接近干净标签 oracle 的60.5%(±0.2%)。在CIFAR-10N(aggre)上,Step-E也优于噪声基线(85.3% vs. 83.9%),并且几乎达到干净标签 oracle(85.9%),仅需适度的训练时间开销。
英文摘要:
Training data collected in the wild often contain noisy labels and outliers that substantially degrade the performance and reliability of deep neural networks. While data cleaning is commonly applied as a separate preprocessing stage, such two-stage pipelines neither fully exploit feedback from the downstream model nor adapt to unknown noise patterns. We propose Step-E, a simple framework that integrates sample selection and model learning into a single optimization process. At each epoch, Step-E ranks samples by loss and gradually increases the fraction of high-loss examples that are excluded from gradient updates after a brief warm-up stage, yielding an online curriculum that focuses on easy and consistent examples and eventually ignores persistent outliers. On CIFAR-100N, Step-E improves the test accuracy of a ResNet-18 model from 43.3% (+/- 0.7%) to 50.4% (+/- 0.9%), clearly outperforming loss truncation, self-paced learning, and one-shot filtering while approaching the clean-label oracle at 60.5% (+/- 0.2%). On CIFAR-10N (aggre), Step-E also improves over the noisy baseline (85.3% vs. 83.9%) and nearly matches the clean-label oracle (85.9%), with only moderate training-time overhead.
Graph convolutional networks for learning with few clean and many noisy labels
Google Research(谷歌研究院);VRG, Faculty of Electrical Engineering, Czech Technical University in Prague(布拉格捷克理工大学电气工程学院视觉识别组);Inria, Univ Rennes, CNRS, IRISA(法国国家信息与自动化研究所、雷恩大学、法国国家科学研究中心、布列塔尼大西洋信息与通信技术研究所)
Weak-PDE-Net: Discovering Open-Form PDEs via Differentiable Symbolic Networks and Weak Formulation
School of Mathematical Sciences, East China Normal University(东华大学数学科学学院);Institute of Applied Physics and Computational Mathematics(应用物理与计算数学研究所);Institute of Artificial Intelligence, Beihang University(北航人工智能研究所);Shanghai Zhangjiang Institute of Mathematics(上海张江数学研究所);Academy for Advanced Interdisciplinary Studies, Northeast Normal University(东北师范大学先进交叉学科研究院)
Active Label Refinement for Robust Training of Imbalanced Medical Image Classification Tasks in the Presence of High Label Noise
Center for Imaging Science, Rochester Institute of Technology(罗切斯特理工学院成像科学中心);University of Aberdeen(阿伯丁大学);Biomedical Engineering, Rochester Institute of Technology(罗切斯特理工学院生物医学工程系)
Group Benefits Instances Selection for Data Purification
Nanjing University of Science and Technology(南京理工大学);Hohai University(河海大学);Beijing Institute of Technology(北京理工大学);Beijing Research Institute of Mechanical and Electrical Technology(北京机电工程研究所);China Ordnance Industrial Standardization Research Institute(中国兵器工业标准化研究所)
Robust Federated Learning with Confidence-Weighted Filtering and GAN-Based Completion under Noisy and Incomplete Data
Computer Engineering Department, İstanbul Commerce University(伊斯坦布尔商业大学计算机工程系);Turkcell Teknoloji(Turkcell技术公司);Grid Communications and Security Group, Electrification and Energy Infrastructures Division, Oak Ridge National Laboratory(橡树岭国家实验室电网安全与能源基础设施部门)
针对联邦学习中的噪声标签、类别缺失与不均衡问题,提出结合自适应噪声清洗、条件 GAN 合成补全和鲁棒训练的隐私保护框架,并在 MNIST 与 Fashion-MNIST 上验证其性能提升。
Auto-differentiable data assimilation: Co-learning of states, dynamics, and filtering algorithms
School of Mechanical Engineering, Purdue University(普渡大学机械工程学院);Department of Statistics, The University of Chicago(芝加哥大学统计学系);Department of Computer Science, The University of Chicago(芝加哥大学计算机科学系);NSF-Simons National Institute for Theory and Mathematics in Biology, Chicago, IL(国家理论与生物数学研究所)
A Differentiable Framework for Full and Phaseless Data Inversion Using Neural Implicit Contrast-Source Representation
Department of Electronic Engineering, Beijing National Research Center for Information Science and Technology (BNRist), and State Key laboratory of Space Network and Communications(电子工程系,北京信息科学与技术国家研究中心(BNRist),空间网络与通信国家重点实验室)
Gaussian processes meet NeuralODEs: A Bayesian framework for learning the dynamics of partially observed systems from scarce and noisy data
University of Pennsylvania(宾夕法尼亚大学)
本文提出了一个机器学习框架(GP-NODE),用于从非线性动态系统的部分、噪声和不规则观测中识别贝叶斯系统。该方法利用微分编程的最新发展,通过常微分方程求解器传播梯度信息,并使用Hamilton Monte Carlo采样和高斯过程先验对所观察到的系统状态进行贝叶斯推断。这使我们能够利用观测数据中的时间相关性,并有效地推断具有量化不确定性的合理模型的后验分布。此外,使用稀疏促进先验,如芬兰马蹄免费模型参数,使潜在的潜在动态的可解释和简约的表示发现。一系列的数值研究表明,建议的GP-NODE方法的有效性,包括捕食系统,系统生物学,和50维人体运动动力系统。总之,我们的研究结果提出了一种新的,灵活的和强大的工作流程,数据驱动的模型发现下的不确定性。本手稿随附的所有代码和数据可在\url{https://github.com/PredictiveIntelligenceLab/GP-NODEs}在线获取。
School of Information Science and Engineering, Southeast University(东南大学信息科学与工程学院);School of Economics and Management, Southeast University(东南大学经济管理学院)