发表机构
Carnegie Mellon University（卡内基梅隆大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对多语言多模态实体链接中稀有实体性能下降问题，提出结合知识图谱结构指标与推理检索互补的免训练框架，在MERLIN基准上显著提升稀有实体准确率。
AI 中文摘要
多模态实体链接将文本和图像中的实体提及与知识库条目进行关联。这些系统在稀有实体上性能下降，但先前的工作主要通过诸如页面浏览量等基于流行度的指标来衡量稀有性。我们利用知识图谱结构指标拓宽了这一视角，这些指标能够捕捉实体被记录和连接的程度。这些指标识别出许多流行度指标遗漏的稀有实体。在由此产生的稀有实体切片上，最先进模型的准确率下降了15.4%至39.9%，表明不同的稀有性定义暴露了不同的失败模式。为了解决这些失败，我们引入了一个简单、无需训练的框架，其中具有推理能力的视觉-语言模型迭代地搜索和推理维基百科，动态收集证据。受控实验表明，推理和检索是互补的。仅推理并不能显著提高稀有实体的准确率。无推理的检索提高了稀有实体的准确率，但可能损害整体准确率。两者的结合表现最佳。在MERLIN（一个涵盖五种语言（印地语、印度尼西亚语、日语、泰米尔语、越南语）的多语言多模态实体链接基准）上，我们最好的系统在整体上比最先进水平提高了6.9%，在稀有实体切片上提高了高达23.3%。我们发布了MERLIN-Rare，即用于针对性评估的稀有实体测试切片，以及我们的框架。
英文摘要
Multimodal entity linking grounds entity mentions in text and images to knowledge-base entries. These systems degrade on rare entities, but prior work measures rarity primarily through popularity-based metrics such as pageviews. We broaden this view using knowledge-graph structural metrics that capture how well an entity is documented and connected. These metrics identify many rare entities that popularity metrics miss. Across the resulting rare-entity slices, state-of-the-art accuracy drops by 15.4-39.9%, showing that different rarity definitions expose different failure modes. To address these failures, we introduce a simple, training-free framework in which a reasoning-capable vision-language model iteratively searches and reasons over Wikipedia, gathering evidence dynamically. Controlled experiments show that reasoning and retrieval are complementary. Reasoning alone does not significantly improve accuracy on rare entities. Retrieval without reasoning improves rare-entity accuracy but can hurt overall accuracy. Their combination performs best. On MERLIN, a multilingual multimodal entity linking benchmark over five languages (Hindi, Indonesian, Japanese, Tamil, Vietnamese), our best system improves over the state of the art by 6.9% overall and by up to 23.3% on rare-entity slices. We release MERLIN-Rare, rare-entity test slices for targeted evaluation, with our framework.
CommentsAccepted to EMNLP 2026 Main Conference
