用于语义分割、视觉蕴涵和释义的语段短语表
Segment-Phrase Table for Semantic Segmentation, Visual Entailment and Paraphrasing
摘要：
我们介绍段短语表（SPT），文本短语和它们相应的分割之间的双射关联的大集合。利用对象识别和自然语言语义的最新进展，我们展示了如何成功地建立一个高质量的段短语表，使用最少的人的监督。更重要的是，我们展示了这种丰富的双峰资源所释放的独特价值，无论是视觉还是自然语言理解。首先，我们表明，细粒度的文本标签促进上下文推理，有助于满足跨图像段的语义约束。此功能使我们能够在基准数据集上实现最先进的分割结果。接下来，我们表明，高质量的分割文本短语的关联有助于更丰富的语义理解和推理这些文本短语。利用这一功能，我们激发了视觉蕴涵和视觉释义的问题，并展示了它在大型数据集上的实用性。
英文摘要：
We introduce Segment-Phrase Table (SPT), a large collection of bijective associations between textual phrases and their corresponding segmentations. Leveraging recent progress in object recognition and natural language semantics, we show how we can successfully build a high-quality segment-phrase table using minimal human supervision. More importantly, we demonstrate the unique value unleashed by this rich bimodal resource, for both vision as well as natural language understanding. First, we show that fine-grained textual labels facilitate contextual reasoning that helps in satisfying semantic constraints across image segments. This feature enables us to achieve state-of-the-art segmentation results on benchmark datasets. Next, we show that the association of high-quality segmentations to textual phrases aids in richer semantic understanding and reasoning of these textual phrases. Leveraging this feature, we motivate the problem of visual entailment and visual paraphrasing, and demonstrate its utility on a large dataset.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
