arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.01101cs.LGcs.AI
收藏

特征对齐:在预训练模型背景下重新思考基于代理的高效主动学习

Feature Alignment: Rethinking Efficient Active Learning via Proxy in the Context of Pre-trained Models

Australian Centre for Robotics, University of Sydney(澳大利亚悉尼大学澳大利亚机器人中心);Norwegian University of Science and Technology(挪威科技大学)

Ziting Wen, Oscar Pizarro, Stefan Williams

AI总结:

本文针对预训练模型下基于代理的主动学习性能下降问题,提出一种通过更新预计算特征和选择合适训练方法的对齐选择新方法,在保持计算效率的同时有效降低了总成本。

摘要:

使用主动学习微调预训练模型有望降低标注成本。然而,这种结合带来了显著的计算成本,特别是随着预训练模型规模的不断增长。最近的研究提出了基于代理的主动学习,通过预计算特征来降低计算成本。然而,这种方法通常会导致主动学习性能的显著下降,有时甚至超过了计算成本的节省。本文表明,并非所有的样本选择差异都会导致性能下降。此外,我们展示了合适的训练方法可以缓解由某些选择差异引起的主动学习性能下降。基于详细分析,我们提出了一种新方法,即通过代理进行对齐选择,该方法通过更新预计算特征和选择合适的训练方法来提升基于代理的主动学习性能。大量实验验证了我们的方法在保持计算效率的同时,改善了高效主动学习的总成本。代码可在 https://github.com/ZiTingW/asvp 获取。

英文摘要:

Fine-tuning the pre-trained model with active learning holds promise for reducing annotation costs. However, this combination introduces significant computational costs, particularly with the growing scale of pre-trained models. Recent research has proposed proxy-based active learning, which pre-computes features to reduce computational costs. Yet, this approach often incurs a significant loss in active learning performance, sometimes outweighing the computational cost savings. This paper demonstrates that not all sample selection differences result in performance degradation. Furthermore, we show that suitable training methods can mitigate the decline of active learning performance caused by certain selection discrepancies. Building upon detailed analysis, we propose a novel method, aligned selection via proxy, which improves proxy-based active learning performance by updating pre-computed features and selecting a proper training method. Extensive experiments validate that our method improves the total cost of efficient active learning while maintaining computational efficiency. The code is available at \url{https://github.com/ZiTingW/asvp}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑