arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.13690cs.LG
收藏

理解模型失配下基于不确定性的主动学习

Understanding Uncertainty-based Active Learning Under Model Mismatch

Texas A&M University(德克萨斯农工大学);University of California, Los Angeles(加州大学洛杉矶分校);Brookhaven National Laboratory(布鲁克海文国家实验室)

Amir Hossein Rahmati, Mingzhou Fan, Ruida Zhou, Nathan M. Urban, Byung-Jun Yoon, Xiaoning Qian

AI总结:

本文研究了模型容量对基于不确定性的主动学习(UAL)有效性的影响,发现当模型容量低且无法覆盖真实分布时,UAL性能可能劣于随机采样,此时应采用直接估计预测性能的获取函数以改善效果。

摘要:

基于不确定性的主动学习(UAL)并非随机获取训练数据点,而是通过查询从无标签池中基于预测不确定性选出的关键样本的标签来运作,旨在最小化模型训练的标注成本。UAL的有效性关键取决于模型容量以及所采用的基于不确定性的获取函数。在本研究的背景下,我们的分析重点是理解机器学习模型的容量如何影响UAL的有效性。通过理论分析、全面仿真和实证研究,我们得出结论:当机器学习模型类别容量较低且无法覆盖潜在的真实分布时,UAL可能导致比随机采样更差的性能。在这种情况下,采用直接以估计预测性能为目标的获取函数可能有助于改善UAL的性能。

英文摘要:

Instead of randomly acquiring training data points, Uncertainty-based Active Learning (UAL) operates by querying the label(s) of pivotal samples from an unlabeled pool selected based on the prediction uncertainty, thereby aiming at minimizing the labeling cost for model training. The efficacy of UAL critically depends on the model capacity as well as the adopted uncertainty-based acquisition function. Within the context of this study, our analytical focus is directed toward comprehending how the capacity of the machine learning model may affect UAL efficacy. Through theoretical analysis, comprehensive simulations, and empirical studies, we conclusively demonstrate that UAL can lead to worse performance in comparison with random sampling when the machine learning model class has low capacity and is unable to cover the underlying ground truth. In such situations, adopting acquisition functions that directly target estimating the prediction performance may be beneficial for improving the performance of UAL.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑