arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2009.03228cs.LGcs.AIstat.ML
收藏

基于高斯过程的信息论元学习

Information Theoretic Meta Learning with Gaussian Processes

DeepMind(深度思维);Athens Univ. of Econ. and Business(雅典经济与商业大学)

Michalis K. Titsias, Francisco J. R. Ruiz, Sotirios Nikoloutsopoulos, Alexandre Galashov

摘要:

我们制定元学习使用信息理论的概念,即,互信息和信息瓶颈。这个想法是学习任务描述的随机表示或编码,由训练集给出,这是关于预测验证集的高度信息。利用变分逼近的互信息,我们得到了一个通用的和易于处理的元学习的框架。这个框架统一了现有的基于梯度的算法,也允许我们派生新的算法。特别是,我们开发了一个基于内存的算法,使用高斯过程来获得非参数编码表示。我们证明了我们的方法在一个Few-Shot回归问题和四个Few-Shot分类问题,获得竞争力的准确性相比,现有的基线。

英文摘要:

We formulate meta learning using information theoretic concepts; namely, mutual information and the information bottleneck. The idea is to learn a stochastic representation or encoding of the task description, given by a training set, that is highly informative about predicting the validation set. By making use of variational approximations to the mutual information, we derive a general and tractable framework for meta learning. This framework unifies existing gradient-based algorithms and also allows us to derive new algorithms. In particular, we develop a memory-based algorithm that uses Gaussian processes to obtain non-parametric encoding representations. We demonstrate our method on a few-shot regression problem and on four few-shot classification problems, obtaining competitive accuracy when compared to existing baselines.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑