arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2202.10629cs.LGcs.AI
收藏

模型重编程:资源高效的跨领域机器学习

Model Reprogramming: Resource-Efficient Cross-Domain Machine Learning

IBM Research(IBM研究院)

Pin-Yu Chen

AI总结:

本文综述 model reprogramming(模型重编程),通过复用源领域预训练模型且免微调解决目标领域任务,以应对资源受限场景下的数据、成本和预训练模型不足问题。

摘要:

在视觉、语言和语音等数据丰富的领域,深度学习普遍用于交付高性能的特定任务模型,甚至能够学习通用的、与任务无关的表示,以便高效微调到下游任务。然而,资源受限领域中的深度学习仍面临多重挑战,包括:(i)数据有限,(ii)模型开发成本受限,以及(iii)缺乏用于有效微调的充足预训练模型。本文概述了 model reprogramming(模型重编程),以弥合这一差距。模型重编程通过重新调整用途并复用来自源领域的成熟预训练模型来解决目标领域中的任务,且无需模型微调,从而实现资源高效的跨领域机器学习;其中源领域与目标领域可能差异巨大。在许多应用中,模型重编程的表现优于迁移学习和从零开始训练。本文阐明了模型重编程的方法论,总结了现有用例,对模型重编程成功的原因给出了理论解释,并在最后讨论了开放式研究问题与机遇。有关模型重编程研究的列表正在 https://github.com/IBM/model-reprogramming 上持续维护和更新。

英文摘要:

In data-rich domains such as vision, language, and speech, deep learning prevails to deliver high-performance task-specific models and can even learn general task-agnostic representations for efficient finetuning to downstream tasks. However, deep learning in resource-limited domains still faces multiple challenges including (i) limited data, (ii) constrained model development cost, and (iii) lack of adequate pre-trained models for effective finetuning. This paper provides an overview of model reprogramming to bridge this gap. Model reprogramming enables resource-efficient cross-domain machine learning by repurposing and reusing a well-developed pre-trained model from a source domain to solve tasks in a target domain without model finetuning, where the source and target domains can be vastly different. In many applications, model reprogramming outperforms transfer learning and training from scratch. This paper elucidates the methodology of model reprogramming, summarizes existing use cases, provides a theoretical explanation of the success of model reprogramming, and concludes with a discussion on open-ended research questions and opportunities. A list of model reprogramming studies is actively maintained and updated at https://github.com/IBM/model-reprogramming.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑