arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2506.05399cs.CVcs.AIcs.CL

基于注意力的跨语言图像描述Transformer模型:深入综述与评估

Attention-based transformer models for image captioning across languages: An in-depth survey and evaluation

The University of Jordan(约旦大学);Princess Sumaya University for Technology(苏曼亚公主科技大学)

Israa A. Albadarneh, Bassam H. Hammo, Omar S. Al-Kadi

AI总结:

本文系统综述了基于注意力机制的跨语言图像描述Transformer模型,分类梳理方法、数据集与评估指标,指出多语言场景下的关键挑战,并提出多模态学习与实时应用等未来方向。

摘要:

图像描述涉及从输入图像生成文本描述,弥合计算机视觉与自然语言处理之间的鸿沟。近年来,基于Transformer的模型通过利用注意力机制实现更好的场景理解,显著改善了描述生成。虽然已有多种综述探讨了基于深度学习的图像描述方法,但很少有研究全面分析跨多种语言的基于注意力的Transformer模型。本综述回顾了基于注意力的图像描述模型,将其分类为基于Transformer的方法、基于深度学习的方法和混合方法。本文探讨了基准数据集,讨论了BLEU、METEOR、CIDEr和ROUGE等评估指标,并强调了多语言描述中的挑战。此外,本文识别了当前模型的关键局限性,包括语义不一致、非英语语言的数据稀缺以及推理能力的限制。最后,我们概述了未来的研究方向,如多模态学习、在AI驱动助手、医疗保健和取证分析中的实时应用。本综述为旨在推进基于注意力的图像描述领域的研究人员提供了全面的参考。

英文摘要:

Image captioning involves generating textual descriptions from input images, bridging the gap between computer vision and natural language processing. Recent advancements in transformer-based models have significantly improved caption generation by leveraging attention mechanisms for better scene understanding. While various surveys have explored deep learning-based approaches for image captioning, few have comprehensively analyzed attention-based transformer models across multiple languages. This survey reviews attention-based image captioning models, categorizing them into transformer-based, deep learning-based, and hybrid approaches. It explores benchmark datasets, discusses evaluation metrics such as BLEU, METEOR, CIDEr, and ROUGE, and highlights challenges in multilingual captioning. Additionally, this paper identifies key limitations in current models, including semantic inconsistencies, data scarcity in non-English languages, and limitations in reasoning ability. Finally, we outline future research directions, such as multimodal learning, real-time applications in AI-powered assistants, healthcare, and forensic analysis. This survey serves as a comprehensive reference for researchers aiming to advance the field of attention-based image captioning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑