重新审视持续表示学习中的监督
Revisiting Supervision for Continual Representation Learning
AI总结：
本研究重新审视持续表示学习中监督的作用，发现配备多层感知机头部的监督模型可超越自监督模型，凸显投影器对特征可迁移性的重要性。
摘要：
在持续学习领域，模型被设计为逐个任务地学习。虽然大多数研究集中在监督持续学习上，但人们对利用大量无标签数据的无监督持续学习的兴趣日益增长。近期研究强调了无监督方法（尤其是自监督学习）在提供稳健表示方面的优势。这些通过自监督方法构建的表示具有更好的可迁移性，这通常与多层感知机投影器所起的作用相关。在本工作中，我们从这一观察出发，重新审视监督在持续表示学习中的作用。我们认为，额外的信息（如人工标注）不应降低表示的质量。我们的研究结果表明，当监督模型配备多层感知机头部时，在持续表示学习中能够超越自监督模型。这凸显了多层感知机投影器在持续学习任务序列中塑造特征可迁移性的重要性。代码可在GitHub上获取：https://github.com/danielm1405/sl-vs-ssl-cl。
英文摘要：
In the field of continual learning, models are designed to learn tasks one after the other. While most research has centered on supervised continual learning, there is a growing interest in unsupervised continual learning, which makes use of the vast amounts of unlabeled data. Recent studies have highlighted the strengths of unsupervised methods, particularly self-supervised learning, in providing robust representations. The improved transferability of those representations built with self-supervised methods is often associated with the role played by the multi-layer perceptron projector. In this work, we depart from this observation and reexamine the role of supervision in continual representation learning. We reckon that additional information, such as human annotations, should not deteriorate the quality of representations. Our findings show that supervised models when enhanced with a multi-layer perceptron head, can outperform self-supervised models in continual representation learning. This highlights the importance of the multi-layer perceptron projector in shaping feature transferability across a sequence of tasks in continual learning. The code is available on github: https://github.com/danielm1405/sl-vs-ssl-cl.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
