arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2301.06013cs.CV
收藏

重新思考伪标签的精度:基于互补学习的测试时自适应

Rethinking Precision of Pseudo Label: Test-Time Adaptation via Complementary Learning

Tentcent Inc.(腾讯公司);Fudan University(复旦大学)

Jiayi Han, Longbin Zeng, Liang Du, Weiyang Ding, Jianfeng Feng

AI总结:

本文提出一种基于互补标签的测试时自适应学习方法,通过利用低概率类别降低伪标签噪声风险,在多个数据集上取得最先进性能。

摘要:

在这项工作中,我们提出了一种新颖的互补学习方法,以增强测试时自适应(TTA),该方法已被证明在存在分布偏移(如损坏)的测试数据上表现出良好性能。在测试时自适应任务中,源域信息通常不可用,模型必须在没有监督的情况下针对测试时样本进行优化。因此,通常的方法是在无监督学习框架下,利用训练良好的源模型的预测为未标注数据分配标签。以往研究采用无监督目标(如模型预测的熵)作为优化目标,以有效学习测试时样本的特征。然而,模型的性能很容易受到伪标签质量的影响,因为伪标签的不准确会给模型引入噪声。因此,我们提出利用“较低概率类别”来降低错误伪标签的风险。引入互补标签来指定这些类别。我们强调,在传统真实标签分布下,互补标签的风险函数与其Vanilla损失公式一致。实验表明,所提出的学习算法在不同数据集和实验设置下均取得了最先进的性能。

英文摘要:

In this work, we propose a novel complementary learning approach to enhance test-time adaptation (TTA), which has been proven to exhibit good performance on testing data with distribution shifts such as corruptions. In test-time adaptation tasks, information from the source domain is typically unavailable and the model has to be optimized without supervision for test-time samples. Hence, usual methods assign labels for unannotated data with the prediction by a well-trained source model in an unsupervised learning framework. Previous studies have employed unsupervised objectives, such as the entropy of model predictions, as optimization targets to effectively learn features for test-time samples. However, the performance of the model is easily compromised by the quality of pseudo-labels, since inaccuracies in pseudo-labels introduce noise to the model. Therefore, we propose to leverage the "less probable categories" to decrease the risk of incorrect pseudo-labeling. The complementary label is introduced to designate these categories. We highlight that the risk function of complementary labels agrees with their Vanilla loss formula under the conventional true label distribution. Experiments show that the proposed learning algorithm achieves state-of-the-art performance on different datasets and experiment settings.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑