arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2101.01918cs.LGstat.ML
收藏

高维感知器迁移学习中的相变

Phase Transitions in Transfer Learning for High-Dimensional Perceptrons

Harvard University(哈佛大学)

Oussama Dhifallah, Yue M. Lu

摘要:

迁移学习通过利用从相关源任务中学习到的知识来提高目标任务的泛化性能。核心问题包括决定应该传递哪些信息以及何时传递信息是有益的。后一个问题与所谓的负迁移现象有关,即转移的源信息实际上降低了目标任务的泛化性能。当两个任务足够不相似时,就会发生这种情况。在本文中,我们提出了一个理论分析的迁移学习通过研究一对相关的感知器学习任务。尽管我们的模型很简单,但它再现了实践中观察到的几个关键现象。具体来说,我们的渐近分析揭示了一个阶段的过渡,从负迁移到正迁移的两个任务的相似性移动过去一个明确定义的阈值。

英文摘要:

Transfer learning seeks to improve the generalization performance of a target task by exploiting the knowledge learned from a related source task. Central questions include deciding what information one should transfer and when transfer can be beneficial. The latter question is related to the so-called negative transfer phenomenon, where the transferred source information actually reduces the generalization performance of the target task. This happens when the two tasks are sufficiently dissimilar. In this paper, we present a theoretical analysis of transfer learning by studying a pair of related perceptron learning tasks. Despite the simplicity of our model, it reproduces several key phenomena observed in practice. Specifically, our asymptotic analysis reveals a phase transition from negative transfer to positive transfer as the similarity of the two tasks moves past a well-defined threshold.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑