arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2102.07121cs.LGcs.AI
收藏

多目标元学习

Multi-Objective Meta Learning

Southern University of Science and Technology(南方科技大学)

Feiyang Ye, Baijiong Lin, Zhixiong Yue, Pengxin Guo, Qiao Xiao, Yu Zhang

摘要:

多目标的Meta学习可以表示为一个多目标双层优化问题(MOBLP),其中上层子问题是解决元学习者的几个可能冲突的目标。然而,现有的研究要么采用低效的进化算法,要么将多个目标线性组合为单目标问题,需要调整组合权重。在本文中,我们提出了一个统一的基于梯度的多目标元学习(MOML)的框架,并设计了第一个基于梯度的优化算法来解决MOBLP交替解决低层次和高层次的子问题,通过梯度下降方法和基于梯度的多目标优化方法,分别。理论上,我们证明了所提出的基于梯度的优化算法的收敛性。经验上,我们提出的MOML框架的有效性在几个元学习问题,包括少样本学习,神经结构搜索,领域自适应,多任务学习。

英文摘要:

Meta learning with multiple objectives can be formulated as a Multi-Objective Bi-Level optimization Problem (MOBLP) where the upper-level subproblem is to solve several possible conflicting targets for the meta learner. However, existing studies either apply an inefficient evolutionary algorithm or linearly combine multiple objectives as a single-objective problem with the need to tune combination weights. In this paper, we propose a unified gradient-based Multi-Objective Meta Learning (MOML) framework and devise the first gradient-based optimization algorithm to solve the MOBLP by alternatively solving the lower-level and upper-level subproblems via the gradient descent method and the gradient-based multi-objective optimization method, respectively. Theoretically, we prove the convergence properties of the proposed gradient-based optimization algorithm. Empirically, we show the effectiveness of the proposed MOML framework in several meta learning problems, including few-shot learning, neural architecture search, domain adaptation, and multi-task learning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑