基于几何自适应预条件器的元学习
Meta-Learning with a Geometry-Adaptive Preconditioner
AI总结：
本文提出几何自适应预条件梯度下降（GAP），通过元学习任务特定且路径依赖的黎曼度量预条件器，改进MAML内层循环优化，并在多种少样本学习任务中超越现有MAML及PGD-MAML系列方法。
摘要：
模型无关元学习（MAML）是最成功的元学习算法之一。它具有双层优化结构，其中外层循环过程学习共享初始化，内层循环过程优化任务特定权重。尽管MAML在内层循环中依赖标准梯度下降，但近期研究表明，使用元学习的预条件器来控制内层循环的梯度下降可能是有益的。然而，现有的预条件器无法同时以任务特定和路径依赖的方式进行自适应。此外，它们不满足黎曼度量条件，而该条件能够通过预条件梯度实现最速下降学习。在本研究中，我们提出了几何自适应预条件梯度下降（GAP），它可以克服MAML中的这些局限性；GAP能够高效地元学习一个依赖于任务特定参数的预条件器，并且其预条件器可以被证明是一个黎曼度量。得益于这两个特性，几何自适应预条件器能够有效改善内层循环优化。实验结果表明，GAP在各种少样本学习任务中优于最先进的MAML系列和预条件梯度下降-MAML（PGD-MAML）系列。代码可在以下地址获取：https://github.com/Suhyun777/CVPR23-GAP。
英文摘要：
Model-agnostic meta-learning (MAML) is one of the most successful meta-learning algorithms. It has a bi-level optimization structure where the outer-loop process learns a shared initialization and the inner-loop process optimizes task-specific weights. Although MAML relies on the standard gradient descent in the inner-loop, recent studies have shown that controlling the inner-loop's gradient descent with a meta-learned preconditioner can be beneficial. Existing preconditioners, however, cannot simultaneously adapt in a task-specific and path-dependent way. Additionally, they do not satisfy the Riemannian metric condition, which can enable the steepest descent learning with preconditioned gradient. In this study, we propose Geometry-Adaptive Preconditioned gradient descent (GAP) that can overcome the limitations in MAML; GAP can efficiently meta-learn a preconditioner that is dependent on task-specific parameters, and its preconditioner can be shown to be a Riemannian metric. Thanks to the two properties, the geometry-adaptive preconditioner is effective for improving the inner-loop optimization. Experiment results show that GAP outperforms the state-of-the-art MAML family and preconditioned gradient descent-MAML (PGD-MAML) family in a variety of few-shot learning tasks. Code is available at: https://github.com/Suhyun777/CVPR23-GAP.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
