MAML 与 ANIL 可证明地学习表征
MAML and ANIL Provably Learn Representations
AI总结：
本文从理论上证明 MAML、ANIL 及其一阶近似能在多任务线性表征学习中以指数速率恢复真实共享表征，并解释最后一层适配利用任务多样性是关键机制。
摘要：
近期的经验证据促使普遍看法认为，基于梯度的元学习（GBML）方法之所以在少样本学习中表现良好，是因为它们学习了一种在各任务间共享的、富有表达力的数据表征。然而，从理论角度看，GBML 的作用机制在很大程度上仍然神秘。本文证明，两种著名的 GBML 方法——MAML 和 ANIL，以及它们的一阶近似——都能够在一组给定任务中学习共同表征。具体而言，在著名的多任务线性表征学习设定中，它们能够以指数级快的速率恢复真实表征。此外，我们的分析阐明，促使 MAML 和 ANIL 恢复底层表征的驱动力在于它们会调整模型的最后一层，该层利用底层任务多样性，在所有感兴趣的方向上改进表征。据我们所知，这些是首批表明 MAML 和/或 ANIL 学习富有表达力的表征，并严格解释其为何这样做的结果。
英文摘要：
Recent empirical evidence has driven conventional wisdom to believe that gradient-based meta-learning (GBML) methods perform well at few-shot learning because they learn an expressive data representation that is shared across tasks. However, the mechanics of GBML have remained largely mysterious from a theoretical perspective. In this paper, we prove that two well-known GBML methods, MAML and ANIL, as well as their first-order approximations, are capable of learning common representation among a set of given tasks. Specifically, in the well-known multi-task linear representation learning setting, they are able to recover the ground-truth representation at an exponentially fast rate. Moreover, our analysis illuminates that the driving force causing MAML and ANIL to recover the underlying representation is that they adapt the final layer of their model, which harnesses the underlying task diversity to improve the representation in all directions of interest. To the best of our knowledge, these are the first results to show that MAML and/or ANIL learn expressive representations and to rigorously explain why they do so.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
