arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2005.11442cs.LGstat.ML
收藏

倾斜数据集的主动学习

Active Learning for Skewed Data Sets

Stanford University(斯坦福大学);Google(谷歌)

Abbas Kazerouni, Qi Zhao, Jing Xie, Sandeep Tata, Marc Najork

摘要:

考虑一个顺序主动学习问题,在每一轮,代理选择一批未标记的数据点,查询它们的标签并更新二元分类器。虽然存在着丰富的身体的主动学习在这种一般形式的工作,在本文中,我们专注于两个显着特点的问题:严重的类不平衡(偏斜)和少量的初始训练数据。这两个问题在许多Web应用程序中以惊人的频率出现。例如,检测在线社区中的攻击性或敏感内容(色情,暴力和仇恨言论)正受到行业和研究界的极大关注。这类问题具有我们所描述的两个特征--绝大多数内容都不是冒犯性的,因此这类内容的正面例子数量比负面例子少几个数量级。此外,在构建机器学习模型来解决此类问题时,通常只有少量的初始训练数据可用。为了解决这两个问题,我们提出了一种混合主动学习算法(HAL),该算法平衡了利用当前标记的训练样本与探索大量未标记数据的知识。通过模拟结果,我们表明,HAL使显着更好的选择什么样的点标记相比,强大的基线,如边缘采样。在HAL选择用于标记的示例上训练的分类器很容易在目标指标(如精确率-召回率曲线下的面积)上超过基线,给出相同的标记示例预算。我们相信HAL提供了一种简单、直观、计算上易于处理的方法来为广泛的机器学习应用构建主动学习。

英文摘要:

Consider a sequential active learning problem where, at each round, an agent selects a batch of unlabeled data points, queries their labels and updates a binary classifier. While there exists a rich body of work on active learning in this general form, in this paper, we focus on problems with two distinguishing characteristics: severe class imbalance (skew) and small amounts of initial training data. Both of these problems occur with surprising frequency in many web applications. For instance, detecting offensive or sensitive content in online communities (pornography, violence, and hate-speech) is receiving enormous attention from industry as well as research communities. Such problems have both the characteristics we describe -- a vast majority of content is not offensive, so the number of positive examples for such content is orders of magnitude smaller than the negative examples. Furthermore, there is usually only a small amount of initial training data available when building machine-learned models to solve such problems. To address both these issues, we propose a hybrid active learning algorithm (HAL) that balances exploiting the knowledge available through the currently labeled training examples with exploring the large amount of unlabeled data available. Through simulation results, we show that HAL makes significantly better choices for what points to label when compared to strong baselines like margin-sampling. Classifiers trained on the examples selected for labeling by HAL easily out-perform the baselines on target metrics (like area under the precision-recall curve) given the same budget for labeling examples. We believe HAL offers a simple, intuitive, and computationally tractable way to structure active learning for a wide range of machine learning applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑