arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1903.07669cs.CV
收藏

神经顺序短语基础(SeqGROUND)

Neural Sequential Phrase Grounding (SeqGROUND)

ETH Zürich(苏黎世联邦理工学院);Disney Research(迪士尼研究院);University of British Columbia(不列颠哥伦比亚大学)

Pelin Dogan, Leonid Sigal, Markus Gross

摘要:

我们提出了一个端到端的方法,在图像中的短语接地。与以前的方法,通常试图地面每个短语独立地建立一个图像-文本嵌入,我们的架构制定接地多个短语作为一个顺序和上下文的过程。具体来说,我们将区域建议和所有短语编码到两个LSTM单元堆栈中,以及迄今为止接地的短语-区域对。这些LSTM堆栈共同捕获上下文,用于下一个短语的基础。由此产生的架构,我们称之为Seqlane,通过允许图像区域与多个短语匹配,反之亦然,支持多对多匹配。我们在Flickr 30 K基准数据集上显示出具有竞争力的性能,并通过消融研究,验证了顺序接地的有效性以及我们模型架构中的个性化设计选择。

英文摘要:

We propose an end-to-end approach for phrase grounding in images. Unlike prior methods that typically attempt to ground each phrase independently by building an image-text embedding, our architecture formulates grounding of multiple phrases as a sequential and contextual process. Specifically, we encode region proposals and all phrases into two stacks of LSTM cells, along with so-far grounded phrase-region pairs. These LSTM stacks collectively capture context for grounding of the next phrase. The resulting architecture, which we call SeqGROUND, supports many-to-many matching by allowing an image region to be matched to multiple phrases and vice versa. We show competitive performance on the Flickr30K benchmark dataset and, through ablation studies, validate the efficacy of sequential grounding as well as individual design choices in our model architecture.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑