arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1807.08133cs.LGcs.AIcs.CLcs.NEstat.ML
收藏

什么不在哪里:将空间表示集成到深度学习架构中的挑战

What is not where: the challenge of integrating spatial representations into deep learning architectures

ADAPT Centre for Digital Content Technology(ADAPT数字内容技术中心);Dublin Institute of Technology(都柏林理工学院);CLASP and FLOV(CLASP与FLOV);University of Gotenburg(哥德堡大学)

John D. Kelleher, Simon Dobnik

摘要:

本文研究了当前用于图像字幕生成的深度学习架构在多大程度上捕获了空间语言。从文献中生成的字幕的例子的评价的基础上,我们认为,系统捕获的对象是在图像数据中,但不是这些对象的位置:这些系统生成的字幕是输出的语言模型的条件下,输出的对象检测器,不能捕获细粒度的位置信息。虽然语言模型为图像字幕提供了有用的知识,但我们认为深度学习图像描述架构也应该对对象之间的几何关系进行建模。

英文摘要:

This paper examines to what degree current deep learning architectures for image caption generation capture spatial language. On the basis of the evaluation of examples of generated captions from the literature we argue that systems capture what objects are in the image data but not where these objects are located: the captions generated by these systems are the output of a language model conditioned on the output of an object detector that cannot capture fine-grained location information. Although language models provide useful knowledge for image captions, we argue that deep learning image captioning architectures should also model geometric relations between objects.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑