ReMeREC：关系感知的多实体指代表达理解
ReMeREC: Relation-aware and Multi-entity Referring Expression Comprehension
AI总结：
针对多实体指代表达理解中的关系建模与数据缺失问题，本文构建 ReMeX 和 EntityText，并提出含 TMP 与 EIR 的 ReMeREC 框架，在多实体定位及关系预测上达到最优。
摘要：
指代表达理解（REC）旨在根据自然语言描述定位图像中的指定实体或区域。现有方法虽然能够处理单实体定位，却往往忽略多实体场景中复杂的实体间关系，从而限制了其准确性与可靠性。此外，缺乏具有细粒度、配对图像-文本-关系标注的高质量数据集，也阻碍了进一步进展。为应对这一挑战，我们首先构建了一个关系感知的多实体 REC 数据集 ReMeX，其中包含详细的关系和文本标注。随后，我们提出 ReMeREC，这是一种联合利用视觉与文本线索来定位多个实体并同时建模其相互关系的新颖框架。为解决语言中隐式实体边界导致的语义歧义，我们引入文本自适应多实体感知器（Text-adaptive Multi-entity Perceptron，TMP），它根据细粒度文本线索动态推断实体的数量和跨度，并生成具有区分性的表示。此外，我们的实体相互关系推理器（Entity Inter-relationship Reasoner，EIR）增强了关系推理与全局场景理解。为进一步提升对细粒度提示的语言理解能力，我们还构建了一个使用大型语言模型生成的小规模辅助数据集 EntityText。在四个基准数据集上的实验表明，ReMeREC 在多实体定位和关系预测方面取得了最先进性能，并大幅优于现有方法。
英文摘要：
Referring Expression Comprehension (REC) aims to localize specified entities or regions in an image based on natural language descriptions. While existing methods handle single-entity localization, they often ignore complex inter-entity relationships in multi-entity scenes, limiting their accuracy and reliability. Additionally, the lack of high-quality datasets with fine-grained, paired image-text-relation annotations hinders further progress. To address this challenge, we first construct a relation-aware, multi-entity REC dataset called ReMeX, which includes detailed relationship and textual annotations. We then propose ReMeREC, a novel framework that jointly leverages visual and textual cues to localize multiple entities while modeling their inter-relations. To address the semantic ambiguity caused by implicit entity boundaries in language, we introduce the Text-adaptive Multi-entity Perceptron (TMP), which dynamically infers both the quantity and span of entities from fine-grained textual cues, producing distinctive representations. Additionally, our Entity Inter-relationship Reasoner (EIR) enhances relational reasoning and global scene understanding. To further improve language comprehension for fine-grained prompts, we also construct a small-scale auxiliary dataset, EntityText, generated using large language models. Experiments on four benchmark datasets show that ReMeREC achieves state-of-the-art performance in multi-entity grounding and relation prediction, outperforming existing approaches by a large margin.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
