Mitigating Memorization of Noisy Labels by Clipping the Model Prediction
Southern University of Science and Technology（南方科技大学）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；South China University of Technology（华南理工大学）；Nanyang Technological University（南洋理工大学）；RIKEN AIP（理化学研究所人工智能研究中心）
Hongxin Wei, Huiping Zhuang, Renchunzi Xie, Lei Feng, Gang Niu, Bo An, Yixuan Li
In the presence of noisy labels, designing robust loss functions is critical for securing the generalization performance of deep neural networks. Cross Entropy (CE) loss has been shown to be not robust to noisy labels due to its unboundedness. To alleviate this issue, existing works typically design specialized robust losses with the symmetric condition, which usually lead to the underfitting issue. In this paper, our key idea is to induce a loss bound at the logit level, thus universally enhancing the noise robustness of existing losses. Specifically, we propose logit clipping (LogitClip), which clamps the norm of the logit vector to ensure that it is upper bounded by a constant. In this manner, CE loss equipped with our LogitClip method is effectively bounded, mitigating the overfitting to examples with noisy labels. Moreover, we present theoretical analyses to certify the noise-tolerant ability of LogitClip. Extensive experiments show that LogitClip not only significantly improves the noise robustness of CE loss, but also broadly enhances the generalization performance of popular robust losses.
Understanding and Mitigating Memorization in Generative Models via Sharpness of Probability Landscapes
Department of Computer Science, Yonsei University, Seoul, Korea（计算机科学系，延世大学，首尔，韩国）；Department of Artificial Intelligence, Yonsei University, Seoul, Korea（人工智能系，延世大学，首尔，韩国）
Mitigating Disparate Impact of Differentially Private Learning through Bounded Adaptive Clipping
Department of Computer Science, University of Helsinki（计算机科学系，赫尔辛基大学）；Department of Electrical and Computer Engineering, American University of Beirut（电气与计算机工程系，贝鲁特美国大学）
University of Chicago（芝加哥大学）；Argonne National Laboratory（阿贡国家实验室）；Lawrence Berkeley National Laboratory（劳伦斯伯克利国家实验室）；Dartmouth College（达特茅斯学院）；International Computer Science Institute（国际计算机科学研究所）；University of California, Berkeley（加州大学伯克利分校）
CrossSplit: Mitigating Label Noise Memorization through Data Splitting
Samsung Advanced Institute of Technology (SAIT)（三星高级技术研究院（SAIT））；SAIT AI Lab（三星高级技术研究院人工智能实验室）；Mila（米拉研究所）；Université de Montreal（蒙特利尔大学）；Canada CIFAR AI Chair（加拿大CIFAR人工智能讲席）
Understanding and Mitigating Memorization in Diffusion Models for Tabular Data
Department of Computer；Data Sciences, Case Western Reserve University, Cleveland, USA；Department of Computer Science \& Engineering, Texas A\&M University, College Station, USA；Department of Biochemistry, Case Western Reserve University, Cleveland, USA；Center for RNA Science；Therapeutics, Case Western Reserve University, Cleveland, USA；Department of Biomedical Engineering, Case Western Reserve University, Cleveland, USA
