关于使用条件批归一化的辅助任务调制在多模态元学习中的局限性
On the Limits of Multi-modal Meta-Learning with Auxiliary Task Modulation Using Conditional Batch Normalization
AI总结：
本研究探讨了一种基于条件批归一化的多模态少样本学习架构，发现桥接网络引入的额外计算和参数是改进主因，且效果无法跨基准复现，为多模态元学习提供了见解与建议。
摘要：
少样本学习旨在学习能够利用少量样本解决新任务的表示。最近的研究表明，跨模态学习可以改善少样本分类的表示。更具体地说，语言是一种丰富的模态，可用于指导视觉学习。在本研究中，我们实验了一种用于少样本学习的多模态架构，该架构由三个组件组成：分类器、辅助网络和桥接网络。分类器执行主分类任务，辅助网络学习从相同输入预测语言表示，桥接网络则使用条件批归一化将辅助网络的高级特征转换为少样本分类器各层的调制参数。桥接网络应促成语言与视觉之间一种轻量级的语义对齐，这可能对分类器有用。然而，在两个流行的少样本分类基准上评估所提方法后，我们发现：a) 改进效果无法在不同基准间复现；b) 即使有改进，也是由桥接网络引入的额外计算和参数所致。我们为未来在多模态元学习中的工作提供了见解和建议，尤其是在使用语言表示时。
英文摘要：
Few-shot learning aims to learn representations that can tackle novel tasks given a small number of examples. Recent studies show that cross-modal learning can improve representations for few-shot classification. More specifically, language is a rich modality that can be used to guide visual learning. In this work, we experiment with a multi-modal architecture for few-shot learning that consists of three components: a classifier, an auxiliary network, and a bridge network. While the classifier performs the main classification task, the auxiliary network learns to predict language representations from the same input, and the bridge network transforms high-level features of the auxiliary network into modulation parameters for layers of the few-shot classifier using conditional batch normalization. The bridge should encourage a form of lightweight semantic alignment between language and vision which could be useful for the classifier. However, after evaluating the proposed approach on two popular few-shot classification benchmarks we find that a) the improvements do not reproduce across benchmarks, and b) when they do, the improvements are due to the additional compute and parameters introduced by the bridge network. We contribute insights and recommendations for future work in multi-modal meta-learning, especially when using language representations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
