arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1506.00511cs.LGcs.CVcs.NE
收藏

基于文本描述的深度Zero-Shot卷积神经网络预测

Predicting Deep Zero-Shot Convolutional Neural Networks using Textual Descriptions

University of Toronto(多伦多大学)

Jimmy Ba, Kevin Swersky, Sanja Fidler, Ruslan Salakhutdinov

摘要:

视觉类别的零样本学习的主要挑战之一是收集伴随图像的语义属性。最近的研究表明,从文本描述(如维基百科文章)中学习可以避免显式定义这些属性的问题。我们提出了一个新的模型,可以分类看不见的类别,从他们的文本描述。具体来说,我们使用文本特征来预测深度卷积神经网络(CNN)中卷积层和全连接层的输出权重。我们利用CNN的架构并学习不同层的特征,而不仅仅是学习两种模态的嵌入空间,这与现有方法一样常见。所提出的模型还允许我们为每个视觉类别自动生成一个伪属性列表,该列表由来自维基百科文章的单词组成。我们训练我们的模型端到端使用加州理工大学-加州大学圣地亚哥分校的鸟和花的数据集,并评估ROC和精度召回曲线。我们的实证结果表明,该模型显着优于以前的方法。

英文摘要:

One of the main challenges in Zero-Shot Learning of visual categories is gathering semantic attributes to accompany images. Recent work has shown that learning from textual descriptions, such as Wikipedia articles, avoids the problem of having to explicitly define these attributes. We present a new model that can classify unseen categories from their textual description. Specifically, we use text features to predict the output weights of both the convolutional and the fully connected layers in a deep convolutional neural network (CNN). We take advantage of the architecture of CNNs and learn features at different layers, rather than just learning an embedding space for both modalities, as is common with existing approaches. The proposed model also allows us to automatically generate a list of pseudo- attributes for each visual category consisting of words from Wikipedia articles. We train our models end-to-end us- ing the Caltech-UCSD bird and flower datasets and evaluate both ROC and Precision-Recall curves. Our empirical results show that the proposed model significantly outperforms previous methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑