NTKCPL：通过估计真实覆盖率在自监督模型之上进行主动学习
NTKCPL: Active Learning on Top of Self-Supervised Model by Estimating True Coverage
AI总结：
针对自监督模型上主动学习相变点提前、策略难选的问题，提出NTKCPL，利用NTK近似、伪标签与聚类生成更准确估计经验风险，并在五个数据集上验证其优势。
摘要：
训练机器学习分类器所需的高昂标注成本推动了主动学习和自监督学习的广泛研究。近期研究表明，在监督学习背景下，需要在训练过程的不同阶段采用不同的主动学习策略，才能确保性能优于随机基线。我们将可用标注数量改变适用主动学习策略的节点称为相变点。本文证明，当将主动学习与自监督模型相结合以获得更好性能时，相变点会更早出现。对于此前未见过的数据集，确定应使用哪种策略变得具有挑战性。我们认为，现有主动学习算法受到相变的严重影响，因为这些算法所估计的整个主动学习池上的经验风险并不准确，并且会受到已标注样本数量的影响。为解决这一问题，我们提出了一种新颖的主动学习策略，即神经正切核聚类-伪标签（neural tangent kernel clustering-pseudo-labels，NTKCPL）。该方法基于伪标签以及采用 NTK 近似的模型预测来估计经验风险。我们分析了影响该近似误差的因素，并设计了一种伪标签聚类生成方法以降低近似误差。我们在五个数据集上验证了所提方法，实验结果表明，该方法在大多数情况下优于基线方法，并且在更广泛的训练预算范围内均有效。
英文摘要：
High annotation cost for training machine learning classifiers has driven extensive research in active learning and self-supervised learning. Recent research has shown that in the context of supervised learning different active learning strategies need to be applied at various stages of the training process to ensure improved performance over the random baseline. We refer to the point where the number of available annotations changes the suitable active learning strategy as the phase transition point. In this paper, we establish that when combining active learning with self-supervised models to achieve improved performance, the phase transition point occurs earlier. It becomes challenging to determine which strategy should be used for previously unseen datasets. We argue that existing active learning algorithms are heavily influenced by the phase transition because the empirical risk over the entire active learning pool estimated by these algorithms is inaccurate and influenced by the number of labeled samples. To address this issue, we propose a novel active learning strategy, neural tangent kernel clustering-pseudo-labels (NTKCPL). It estimates empirical risk based on pseudo-labels and the model prediction with NTK approximation. We analyze the factors affecting this approximation error and design a pseudo-label clustering generation method to reduce the approximation error. We validate our method on five datasets, empirically demonstrating that it outperforms the baseline methods in most cases and is valid over a wider range of training budgets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
