照亮阴影：利用概念引导的视觉-语言模型增强长尾实体 grounding
Light Up the Shadows: Enhance Long-Tailed Entity Grounding with Concept-Guided Vision-Language Models
AI总结：
针对大规模 MMKGs 中长尾实体图像稀缺且图文匹配噪声高的问题，提出概念引导的两阶段框架 COG，通过概念整合与证据融合提升长尾图文对识别准确率、灵活性和可解释性。
摘要：
多模态知识图谱（MMKGs）已被证明对各种下游任务具有价值。然而，扩展它们具有挑战性，因为构建大规模 MMKGs 往往会引入不匹配的图像（即噪声）。知识图谱中的大多数实体属于长尾，这意味着在线上可获得的相关图像很少。这种稀缺性使得很难判断找到的图像是否与实体匹配。为解决该问题，我们借鉴 Triangle of Reference Theory（指称三角理论），提出利用概念引导来增强视觉-语言模型。具体而言，我们引入 COG，这是一个包含 COncept-Guided（概念引导）视觉-语言模型的两阶段框架。该框架包括一个 Concept Integration（概念整合）模块和一个 Evidence Fusion（证据融合）模块：前者可有效识别长尾实体的图文对，后者提供可解释性并支持人工验证。为证明 COG 的有效性，我们构建了一个包含 25k 个长尾实体图文对的数据集。综合实验表明，与基线相比，COG 不仅提高了识别长尾图文对的准确率，还提供了灵活性和可解释性。
英文摘要：
Multi-Modal Knowledge Graphs (MMKGs) have proven valuable for various downstream tasks. However, scaling them up is challenging because building large-scale MMKGs often introduces mismatched images (i.e., noise). Most entities in KGs belong to the long tail, meaning there are few images of them available online. This scarcity makes it difficult to determine whether a found image matches the entity. To address this, we draw on the Triangle of Reference Theory and suggest enhancing vision-language models with concept guidance. Specifically, we introduce COG, a two-stage framework with COncept-Guided vision-language models. The framework comprises a Concept Integration module, which effectively identifies image-text pairs of long-tailed entities, and an Evidence Fusion module, which offers explainability and enables human verification. To demonstrate the effectiveness of COG, we create a dataset of 25k image-text pairs of long-tailed entities. Our comprehensive experiments show that COG not only improves the accuracy of recognizing long-tailed image-text pairs compared to baselines but also offers flexibility and explainability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
