用可操作的策略重建主动学习中的信任
Rebuilding Trust in Active Learning with Actionable Metrics
摘要：
主动学习（AL）是一个活跃的研究领域，但很少在工业中使用，尽管迫切需要。这在一定程度上是由于目标的不一致，虽然研究努力在选定的数据集上获得最佳结果，但行业希望保证主动学习的表现始终如一，至少比随机标记更好。主动学习的一次性性质使得了解如何进行策略选择以及导致性能不佳的原因（缺乏探索，选择太难分类的样本......）至关重要。为了帮助重建行业从业者对主动学习的信任，我们提出了各种可操作的指标。通过对CIFAR 100，Fashion-MNIST和20个新闻组等参考数据集的广泛实验，我们表明这些指标为从业者可以利用的AL策略带来了可解释性。
英文摘要：
Active Learning (AL) is an active domain of research, but is seldom used in the industry despite the pressing needs. This is in part due to a misalignment of objectives, while research strives at getting the best results on selected datasets, the industry wants guarantees that Active Learning will perform consistently and at least better than random labeling. The very one-off nature of Active Learning makes it crucial to understand how strategy selection can be carried out and what drives poor performance (lack of exploration, selection of samples that are too hard to classify, ...). To help rebuild trust of industrial practitioners in Active Learning, we present various actionable metrics. Through extensive experiments on reference datasets such as CIFAR100, Fashion-MNIST, and 20Newsgroups, we show that those metrics brings interpretability to AL strategies that can be leveraged by the practitioner.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
