arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1611.01714cs.LGcs.CL
收藏

超越微调:小数据学习的模块化方法

Beyond Fine Tuning: A Modular Approach to Learning on Small Data

Eastern Washington University(东华盛顿大学);Pacific Northwest National Laboratory(太平洋西北国家实验室)

Ark Anderson, Kyle Shaffer, Artem Yankov, Court D. Corley, Nathan O. Hodas

摘要:

在本文中,我们提出了一种技术,训练神经网络模型的少量数据。目前用于在少量丰富数据上训练神经网络的方法通常依赖于诸如微调预训练的神经网络或使用特定于领域的手工设计特征的策略。在这里,我们将网络层或整个网络视为模块,并将预训练模块与未训练模块相结合,以学习数据集之间的分布变化。使用模块化方法的核心影响来自于向网络添加新的表示,而不是通过微调来替换表示。使用这种技术,我们能够超越使用标准微调迁移学习方法的结果,并且在使用少量数据时,我们还能够显着提高这些方法的性能。

英文摘要:

In this paper we present a technique to train neural network models on small amounts of data. Current methods for training neural networks on small amounts of rich data typically rely on strategies such as fine-tuning a pre-trained neural network or the use of domain-specific hand-engineered features. Here we take the approach of treating network layers, or entire networks, as modules and combine pre-trained modules with untrained modules, to learn the shift in distributions between data sets. The central impact of using a modular approach comes from adding new representations to a network, as opposed to replacing representations via fine-tuning. Using this technique, we are able surpass results using standard fine-tuning transfer learning approaches, and we are also able to significantly increase performance over such approaches when using smaller amounts of data.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑