arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.17152cs.CL
收藏

多语言和多模态大语言模型在野:为低资源语言构建

Multilingual and Multimodal LLMs in the Wild: Building for Low-Resource Languages

Qatar Computing Research Institute(卡塔尔计算研究所);HBKU(哈马德大学);York University(约克大学)

Firoj Alam, Shammur Absar Chowdhury, Enamul Hoque Prince

AI总结:

本文探讨了在有限数据和计算资源下构建多语言多模态大语言模型的方法,涵盖了低成本数据创建、三模态对齐适配器堆栈以及文化感知评估等核心技术和资源。

摘要:

多模态大语言模型正从视觉语言扩展到三模态(见、听、读),但流水线和基准测试仍以英语为中心且计算密集。本教程概述了在有限的数据和计算预算下,跨文本、语音和视觉的多语言多模态研究领域的概述,综合了基础理论、最近的多语言模型(PALO、Maya)和语音-文本大语言模型。我们涵盖了低成本的数据创建/整理;三模态对齐的适配器堆栈;超越英语的文化感知评估以及用于微调紧凑型多语言视觉语言模型和构建语音->文本->LLM流水线的实用资源。内容将以交互式半天教程形式呈现,面向在低资源语言环境中从事多语言、多模态AI研究和实践的研究人员和从业者。

英文摘要:

Multimodal LLMs are evolving from vision-language to tri-modality that see, hear, and read, yet pipelines and benchmarks remain English-centric and compute-heavy. The tutorial offers an overview of this emerging research area for multilingual multimodality across text, speech, and vision under limited data/compute budgets, synthesizing foundations, recent multilingual models (PALO, Maya), speech-text LLMs. We cover low-cost data creation/curation; adapter stacks for tri-modal alignment; culture-aware evaluation beyond English and hands on resources for fine-tuning a compact multilingual VLM and wiring a speech->text->LLM pipeline. The content will be delivered as an interactive half-day tutorial, designed for researchers and practitioners working on multilingual, multimodal AI in low-resource language settings.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑