使用语言自适应推理的指代表达理解
Referring Expression Comprehension Using Language Adaptive Inference
AI总结：
本文针对REC中不同表达只需少量模型组件的问题，提出LADS框架按语言动态提取自适应子网，在四个基准上同时提升推理速度与准确率。
摘要：
与通用目标检测不同，指代表达理解（REC）旨在定位自然语言表达所指代的特定对象。该表达提供了相关视觉和上下文模式的高级概念，这些概念随不同表达而显著变化，并且仅占REC模型中编码内容的一小部分。这使我们提出一个问题：对于各种指代表达，我们是否真的需要具有固定结构的整个网络？理想情况下，给定一个表达，只需要REC模型中与表达相关的组件。这些组件的数量应当较少，因为每个表达仅包含极少的视觉和上下文线索。本文探索表达与REC模型之间的适配，以实现动态推理。具体而言，我们提出了一个简洁而高效的框架，名为Language Adaptive Dynamic Subnets（LADS，语言自适应动态子网），它能够以指代表达为条件，从REC模型中提取语言自适应子网。通过使用紧凑子网，推理可以更加经济高效。在RefCOCO、RefCOCO+、RefCOCOg和Referit上的大量实验表明，与最先进方法相比，所提方法实现了更快的推理速度和更高的准确率。
英文摘要：
Different from universal object detection, referring expression comprehension (REC) aims to locate specific objects referred to by natural language expressions. The expression provides high-level concepts of relevant visual and contextual patterns, which vary significantly with different expressions and account for only a few of those encoded in the REC model. This leads us to a question: do we really need the entire network with a fixed structure for various referring expressions? Ideally, given an expression, only expression-relevant components of the REC model are required. These components should be small in number as each expression only contains very few visual and contextual clues. This paper explores the adaptation between expressions and REC models for dynamic inference. Concretely, we propose a neat yet efficient framework named Language Adaptive Dynamic Subnets (LADS), which can extract language-adaptive subnets from the REC model conditioned on the referring expressions. By using the compact subnet, the inference can be more economical and efficient. Extensive experiments on RefCOCO, RefCOCO+, RefCOCOg, and Referit show that the proposed method achieves faster inference speed and higher accuracy against state-of-the-art approaches.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
