arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.01193cs.CLcs.CV

弥合词汇歧义与视觉:关于视觉词义消歧的简要综述

Bridging Lexical Ambiguity and Vision: A Mini Review on Visual Word Sense Disambiguation

School of Computing Informatics(计算与信息学学院);Institute of Technology(技术研究所)

Shashini Nilukshi, Deshan Sumanathilaka

AI总结:

本文综述了视觉词义消歧的发展,探讨了对比模型和LLM在解决词汇歧义中的作用,并指出未来发展方向。

摘要:

本文对视觉词义消歧(VWSD)进行了简要综述,这是一种传统词义消歧(WSD)的多模态扩展。VWSD有助于解决视觉-语言任务中的词汇歧义。虽然传统WSD仅依赖文本和词汇资源,VWSD使用视觉线索在最小文本输入的情况下找到歧义词的正确含义。综述回顾了从早期多模态融合方法到使用对比模型如CLIP、基于扩散的文本到图像生成和大型语言模型(LLM)支持的新框架的发展。研究从2016年到2025年,通过基于特征、基于图和对比嵌入技术展示了VWSD的增长。它专注于提示工程、微调和适应多种语言。定量结果表明,基于CLIP的微调模型和LLM增强的VWSD系统在Mean Reciprocal Rank(MRR)上比零样本基线高出6-8%。然而,仍然存在挑战,例如上下文限制、模型对常见含义的偏见、多语言数据集的缺乏以及对更好评估框架的需求。分析突显了CLIP对齐、扩散生成和LLM推理之间的日益增长的重叠作为强大、上下文感知和多语言消歧系统的发展路径。

英文摘要:

This paper offers a mini review of Visual Word Sense Disambiguation (VWSD), which is a multimodal extension of traditional Word Sense Disambiguation (WSD). VWSD helps tackle lexical ambiguity in vision-language tasks. While conventional WSD depends only on text and lexical resources, VWSD uses visual cues to find the right meaning of ambiguous words with minimal text input. The review looks at developments from early multimodal fusion methods to new frameworks that use contrastive models like CLIP, diffusion-based text-to-image generation, and large language model (LLM) support. Studies from 2016 to 2025 are examined to show the growth of VWSD through feature-based, graph-based, and contrastive embedding techniques. It focuses on prompt engineering, fine-tuning, and adapting to multiple languages. Quantitative results show that CLIP-based fine-tuned models and LLM-enhanced VWSD systems consistently perform better than zero-shot baselines, achieving gains of up to 6-8\% in Mean Reciprocal Rank (MRR). However, challenges still exist, such as limitations in context, model bias toward common meanings, a lack of multilingual datasets, and the need for better evaluation frameworks. The analysis highlights the growing overlap of CLIP alignment, diffusion generation, and LLM reasoning as the future path for strong, context-aware, and multilingual disambiguation systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑