arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 0905.4022cs.LG
收藏

基于特征选择的迁移学习

Transfer Learning Using Feature Selection

University of Pennsylvania(宾夕法尼亚大学);University of Colorado (Boulder)(科罗拉多大学博尔德分校);Columbia University(哥伦比亚大学);Swarthmore College(斯沃斯莫尔学院)

Paramveer S. Dhillon, Dean Foster, Lyle Ungar

摘要:

我们提出了三种使用迁移学习来改进特征选择的相关方法。这三种方法解决了不同的问题,因此在任务或特征类之间共享不同类型的信息,但这三种方法都基于信息理论的最小描述长度(MDL)原则,并共享相同的基本贝叶斯解释。第一种方法MIC适用于为共享同一组特征的多个任务("同时转移“)同时建立预测模型的情况。MIC允许将每个特征添加到任何、一些或所有任务模型中,并且对于从大量特征池中选择一小组预测特征是最有益的,这在基因组和生物数据集中很常见。我们的第二种方法,TPC(三部分编码),使用类似的方法的情况下,功能可以分为功能类。我们的第三种方法,Transfer-TPC,解决了“顺序转移”问题,其中我们想要转移知识的任务可能事先不知道,并且可能具有与其他任务不同的数据量。当我们想要在具有不等量标记数据的任务之间传输知识时,Transfer-TPC是最有益的,例如用于消除不同动词意义的数据。我们证明了这些方法的有效性与实验结果有关的基因组学和词义消歧(WSD)的真实世界的数据。

英文摘要:

We present three related ways of using Transfer Learning to improve feature selection. The three methods address different problems, and hence share different kinds of information between tasks or feature classes, but all three are based on the information theoretic Minimum Description Length (MDL) principle and share the same underlying Bayesian interpretation. The first method, MIC, applies when predictive models are to be built simultaneously for multiple tasks (``simultaneous transfer'') that share the same set of features. MIC allows each feature to be added to none, some, or all of the task models and is most beneficial for selecting a small set of predictive features from a large pool of features, as is common in genomic and biological datasets. Our second method, TPC (Three Part Coding), uses a similar methodology for the case when the features can be divided into feature classes. Our third method, Transfer-TPC, addresses the ``sequential transfer'' problem in which the task to which we want to transfer knowledge may not be known in advance and may have different amounts of data than the other tasks. Transfer-TPC is most beneficial when we want to transfer knowledge between tasks which have unequal amounts of labeled data, for example the data for disambiguating the senses of different verbs. We demonstrate the effectiveness of these approaches with experimental results on real world data pertaining to genomics and to Word Sense Disambiguation (WSD).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑