arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2206.04838cs.LGcs.AI
收藏

保护核心集:一种基于密度的主动学习核心集选择算法

In Defense of Core-set: A Density-aware Core-set Selection for Active Learning

Deargen Inc.(Deargen公司);Deargen USA, Inc.(Deargen美国公司)

Yeachan Kim, Bonggun Shin

摘要:

主动学习通过从未标记数据集中标记信息样本来实现标记数据集的有效构造。在真实世界的主动学习场景中,考虑所选样本的多样性是至关重要的,因为存在许多冗余或高度相似的样本。核心集方法是一种很有前途的基于多样性的方法,它根据样本之间的距离来选择不同的样本。然而,与基于不确定性的方法相比,该方法表现不佳,该方法选择了神经模型显示置信度较低的最困难的样本。在这项工作中,我们通过密度的镜头分析特征空间,有趣的是,观察到局部稀疏区域往往比密集区域具有更多的信息样本。基于我们的分析,我们赋予核心集的方法与密度意识,并提出了一个密度意识的核心集(DACS)。该策略是估计未标记样本的密度,并主要从稀疏区域中选择不同的样本。为了减少密度估计中的计算瓶颈,我们还引入了一种新的基于局部敏感哈希的密度近似。实验结果清楚地证明了DACS在分类和回归任务中的有效性,并特别表明DACS可以在实际场景中产生最先进的性能。由于DACS是弱依赖于神经架构,我们提出了一个简单而有效的组合方法,以表明现有的方法可以有益地结合DACS。

英文摘要:

Active learning enables the efficient construction of a labeled dataset by labeling informative samples from an unlabeled dataset. In a real-world active learning scenario, considering the diversity of the selected samples is crucial because many redundant or highly similar samples exist. Core-set approach is the promising diversity-based method selecting diverse samples based on the distance between samples. However, the approach poorly performs compared to the uncertainty-based approaches that select the most difficult samples where neural models reveal low confidence. In this work, we analyze the feature space through the lens of the density and, interestingly, observe that locally sparse regions tend to have more informative samples than dense regions. Motivated by our analysis, we empower the core-set approach with the density-awareness and propose a density-aware core-set (DACS). The strategy is to estimate the density of the unlabeled samples and select diverse samples mainly from sparse regions. To reduce the computational bottlenecks in estimating the density, we also introduce a new density approximation based on locality-sensitive hashing. Experimental results clearly demonstrate the efficacy of DACS in both classification and regression tasks and specifically show that DACS can produce state-of-the-art performance in a practical scenario. Since DACS is weakly dependent on neural architectures, we present a simple yet effective combination method to show that the existing methods can be beneficially combined with DACS.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑