AFRICAPTION:为非洲语言图像描述建立新范式
AFRICAPTION: Establishing a New Paradigm for Image Captioning in African Languages
AI总结:
本文提出AfriCaption框架,通过基于Flickr8k的20种非洲语言对齐数据集、质量保持流水线及集成SigLIP与NLLB200的0.5B参数模型,构建首个可扩展的非洲语言图像描述资源。
摘要:
多模态AI研究绝大多数集中于高资源语言,阻碍了该领域进展的民主化。为解决这一问题,我们提出AfriCaption,一个面向20种非洲语言的多语言图像描述综合框架,其贡献有三方面:(i)基于Flickr8k构建的精选数据集,其特点是通过上下文感知的选择与翻译流程生成语义对齐的描述;(ii)动态且保留上下文的流水线,通过模型集成和自适应替换确保持续质量;(iii)AfriCaption模型,一种0.5B参数的视觉到文本架构,集成SigLIP和NLLB200,用于跨代表性不足语言生成描述。这一统一框架确保了持续的数据质量,并为代表性不足的非洲语言建立了首个可扩展的图像描述资源,为真正包容的多模态AI奠定基础。
英文摘要:
Multimodal AI research has overwhelmingly focused on high-resource languages, hindering the democratization of advancements in the field. To address this, we present AfriCaption, a comprehensive framework for multilingual image captioning in 20 African languages and our contributions are threefold: (i) a curated dataset built on Flickr8k, featuring semantically aligned captions generated via a context-aware selection and translation process; (ii) a dynamic, context-preserving pipeline that ensures ongoing quality through model ensembling and adaptive substitution; and (iii) the AfriCaption model, a 0.5B parameter vision-to-text architecture that integrates SigLIP and NLLB200 for caption generation across under-represented languages. This unified framework ensures ongoing data quality and establishes the first scalable image-captioning resource for under-represented African languages, laying the groundwork for truly inclusive multimodal AI.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。