arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2012.07477cs.CV
收藏

有限样本下的聚合自监督特征学习

Aggregative Self-Supervised Feature Learning from a Limited Sample

Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);Jarvis Lab, Tencent(腾讯贾维斯实验室)

Jiuwen Zhu, Yuexiang Li, S. Kevin Zhou

摘要:

自监督学习是解决训练数据有限和标注不足问题的有效方法。SSL的关键部分是它的代理任务,它定义了监督信号,并驱动学习向有效的特征表示。然而,大多数SSL方法通常集中在一个单一的代理任务,这极大地限制了学习的功能的表达能力,从而恶化了网络的泛化能力。在这方面,我们在此提出了两种聚合策略,根据各种形式的互补性来提高自监督学习特征的鲁棒性。我们首先提出了一个原则性的框架,多任务聚合自监督学习从一个有限的样本,形成一个统一的表示,利用不同的任务之间的特征互补性的意图。然后,在自聚合SSL中,我们提出了一个辅助损失函数的基础上的线性中心内核对齐度量,这显式地促进了探索在哪里被发现的功能,从一个代理任务的学习,进一步提高建模能力,以自我补充现有的代理任务。我们在有限的数据和注释场景下对2D自然图像和3D医学图像分类任务进行了广泛的实验,证实了所提出的聚合策略成功地提高了分类精度。

英文摘要:

Self-supervised learning (SSL) is an efficient approach that addresses the issue of limited training data and annotation shortage. The key part in SSL is its proxy task that defines the supervisory signals and drives the learning toward effective feature representations. However, most SSL approaches usually focus on a single proxy task, which greatly limits the expressive power of the learned features and therefore deteriorates the network generalization capacity. In this regard, we hereby propose two strategies of aggregation in terms of complementarity of various forms to boost the robustness of self-supervised learned features. We firstly propose a principled framework of multi-task aggregative self-supervised learning from a limited sample to form a unified representation, with an intent of exploiting feature complementarity among different tasks. Then, in self-aggregative SSL, we propose to self-complement an existing proxy task with an auxiliary loss function based on a linear centered kernel alignment metric, which explicitly promotes the exploring of where are uncovered by the features learned from a proxy task at hand to further boost the modeling capability. Our extensive experiments on 2D natural image and 3D medical image classification tasks under limited data and annotation scenarios confirm that the proposed aggregation strategies successfully boost the classification accuracy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑