MELLA：弥合低资源语言多模态大语言模型的语言能力与文化根基
MELLA: Bridging Linguistic Capability and Cultural Groundedness for Low-Resource Language MLLMs
AI总结：
研究针对低资源语言MLLMs文化内涵不足问题，提出MELLA数据集，采用双源策略，结合母语网络图像-替代文本对与生成翻译的图像描述进行监督，经实验表明能减轻文化幻觉，强调数据对齐对低资源语言文化基础多模态理解的重要性。
摘要：
多模态大语言模型（MLLMs）在高资源语言中表现出色，但在低资源环境中常生成语言流畅但文化内涵“单薄”的描述。我们认为这种不足不仅是语言限制，特定文化视觉知识依赖母语视觉-文本对齐，而以翻译为中心的流程很少提供。我们提出MELLA，一个跨八种低资源语言的多模态数据集，支持语言流畅性和文化根基。MELLA采用双源策略，结合母语网络图像-替代文本对进行文化基础监督，以及生成和翻译的图像描述进行语言丰富监督，明确分离多语言多模态数据中常混淆的两种学习信号。通过对多个MLLM主干进行受控诊断微调，我们表明MELLA通过帮助模型识别和阐述基于翻译的适应所忽略的特定文化实体来减轻文化幻觉。我们的发现强调数据对齐而非仅模型修改，是实现低资源语言文化基础多模态理解的途径。数据集可从此https URL获取。
英文摘要：
Multimodal Large Language Models (MLLMs) perform strongly in high-resource languages, yet often produce fluent but culturally "thin" descriptions in low-resource settings. We argue that this failure is not merely a linguistic limitation: culture-specific visual knowledge depends on native visual-textual alignments that translation-centric pipelines rarely provide. We present MELLA, a multimodal dataset across eight low-resource languages, designed to support linguistic fluency and cultural groundedness. MELLA uses a dual-source strategy that combines native web image-alt-text pairs for culture-grounded supervision with generated-and-translated image descriptions for linguistically rich supervision, explicitly separating two learning signals often conflated in multilingual multimodal data. Through controlled diagnostic fine-tuning on multiple MLLM backbones, we show that MELLA mitigates cultural hallucination by helping models recognize and articulate culturally specific entities overlooked by translation-based adaptation. Our findings highlight data alignment, rather than model modification alone, as a path toward culturally grounded multimodal understanding in low-resource languages. The dataset is available at https://opendatalab.com/applyMultilingualCorpus.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
