arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2104.09015cs.LGcs.AI
收藏

标签、信息与计算:使用足够标签的有效学习

Labels, Information, and Computation: Efficient Learning Using Sufficient Labels

University of Florida(佛罗里达大学)

Shiyu Duan, Spencer Chang, Jose C. Principe

摘要:

在监督学习中,获得大量完全标记的训练数据是昂贵的。我们表明,我们并不总是需要完整的标签信息的每一个训练样本,训练一个称职的分类。具体来说,受统计充分性原则的启发,我们提出了一个完全标记的训练集的统计(摘要),它捕获了几乎所有用于分类的相关信息,但同时更容易直接获得。我们称之为统计“随机标记的数据”,并证明其充分性和效率,找到最佳的隐藏表示,主管分类头可以训练使用尽可能少的一个随机选择的完全标记的例子,每个类。充分标记的数据可以直接从注释器中获得,而不需要先收集完全标记的数据。我们证明了直接获得完全标记数据比获得完全标记数据更容易。此外,标记的数据自然更安全,因为它存储的是相对信息,而不是绝对信息。大量的实验结果支持我们的理论。

英文摘要:

In supervised learning, obtaining a large set of fully-labeled training data is expensive. We show that we do not always need full label information on every single training example to train a competent classifier. Specifically, inspired by the principle of sufficiency in statistics, we present a statistic (a summary) of the fully-labeled training set that captures almost all the relevant information for classification but at the same time is easier to obtain directly. We call this statistic "sufficiently-labeled data" and prove its sufficiency and efficiency for finding the optimal hidden representations, on which competent classifier heads can be trained using as few as a single randomly-chosen fully-labeled example per class. Sufficiently-labeled data can be obtained from annotators directly without collecting the fully-labeled data first. And we prove that it is easier to directly obtain sufficiently-labeled data than obtaining fully-labeled data. Furthermore, sufficiently-labeled data is naturally more secure since it stores relative, instead of absolute, information. Extensive experimental results are provided to support our theory.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑