arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.11490cs.AIcs.CLcs.CV
收藏

人为区域适应于多模态视觉-语言模型

Anthropogenic Regional Adaptation in Multimodal Vision-Language Model

Cohere;SEACrowd;AI Singapore;Universiti Teknologi PETRONAS;Oracle;Carnegie Mellon University(卡内基梅隆大学);Monash University, Indonesia(莫纳什大学(印尼));King Mongkut’s University of Technology Thonburi(泰国孔敬大学);Nara Institute of Science and Technology(奈良科学技術大學);Stanford University(斯坦福大学);MBZUAI;National University of Singapore(新加坡国立大学);Monash University, Australia(莫纳什大学(澳大利亚));Brown University(布朗大学);University of Bath(巴斯大学);Mila - Quebec AI Institute(蒙特利尔AI研究所);Institut Teknologi Bandung(Bandung 工程技术大学);Ateneo de Manila University(马尼拉亚特内奥大学);Universitas Pelita Harapan(Pelita Harapan 大学);Seoul National University of Science and Technology(首尔科学技术大学);Thammasat University(泰国 Thammasat 大学);Independent(独立);University College London(伦敦大学学院);Nanyang Technological University(南洋理工大学);MiroMind AI;University of Indonesia(印度尼西亚大学);University of New Haven(纽黑文大学);INTI International University and Colleges(INTI 国际大学和学院);Binus University(Binus 大学);National University Philippines(菲律宾国家大学);ThoughtFull
Samuel Cahyawijaya, Peerat Limkonchotiwat, Tack Hwa Wong, Hitesh Laxmichand Patel, Amit Agarwal, Manuel Antonio Rufino, Carlos Rafael Catalan, Muhammad Reza Qor… 展开作者

Samuel Cahyawijaya, Peerat Limkonchotiwat, Tack Hwa Wong, Hitesh Laxmichand Patel, Amit Agarwal, Manuel Antonio Rufino, Carlos Rafael Catalan, Muhammad Reza Qorib, Vicky Feliren, Holy Lovenia, Aye Hninn Khine, Frederikus Hudi, David Anugraha, Alham Fikri Aji, Romrawin Chumpu, Viet-Thanh Pham, Minghan Wang, Mohamed Fazli Imam, Ruochen Zhang, Joseph Marvin Imperial, Khumaisa Nur'aini, Do Xuan Long, Musa Izzanardi Wijanarko, Joel Ruben Antony Moniz, Patrick Amadeus Irawan, Hanif Muhammad Zhafran, Isaiah Flores, Salsabila Zahirah Pranida, Jun Kevin, Jostin Jerico Rosal, Patricia Nicole Monderin, Kun Kerdthaisong, Ahmad Mustafid, My Chiffon Nguyen, Natchapon Jongwiriyanurak, Siva Worajitwannakul, Haochen Li, Adrian Xuan Wei Lim, Bin Wang, Muhammad Ravi Shulthan Habibi, Lynnette Hui Xian Ng, Mithil Bangera, Yeshil Bangera, Priyaranjan Pattnayak, Dun Li Chan, Sherissa Caren Djuniwar, Cho Chan Myei Oo, Hee Ming Shan

AI总结:

本文提出人为区域适应框架,通过地理通用化简化方法提升多模态模型在特定区域的文化相关性,实验显示在东南亚地区提升5-15%的同时保持全球性能。

摘要:

尽管视觉-语言(VL)领域在多语言和多领域整合视觉和文本信息方面取得了显著成功,但尚无专门框架用于评估以人类为中心的对齐在视觉-语言系统中的对齐。我们提出了两个贡献:首先,我们引入了人为区域适应:一种新的范式,旨在优化模型对特定区域环境的相关性,同时确保保留全球泛化能力。其次,我们提出了一种简单但有效的适应方法,称为地理通用化简化(GG-EZ),该方法利用区域数据过滤和模型融合。通过在三种VL架构(大型视觉-语言模型、文本到图像扩散模型和视觉-语言嵌入模型)以及东南亚(SEA)区域适应的案例研究中进行全面实验,我们展示了人为区域适应的重要性以及GG-EZ的有效性,显示在SEA地区文化相关性指标提升了5-15%,同时保持超过98%的全球性能,甚至偶尔超越。我们的发现确立了人为区域对齐作为多模态视觉-语言模型在不同地区应用的基础范式,并展示了一种简单但有效的基线方法,该方法在优化区域价值对齐的同时保留了全球泛化能力。

英文摘要:

While the field of vision-language (VL) has achieved remarkable success in integrating visual and textual information across multiple languages and domains, there is still no dedicated framework for assessing human-centric alignment in vision-language systems. We offer two contributions to address this gap. First, we introduce Anthropogenic Regional Adaptation: a novel paradigm that aims to optimize model relevance to specific regional contexts while ensuring the retention of global generalization capabilities. Second, we present a simple, but effective adaptation method named Geographical-generalization-made-easy (GG-EZ), which utilizes regional data filtering and model merging. Through comprehensive experiments on 3 VL architectures: large vision-language models, text-to-image diffusion models, and vision-language embedding models, and a case study in Southeast Asia (SEA) regional adaptation, we demonstrate the importance of Anthropogenic Regional Adaptation and the effectiveness of GG-EZ, showing 5-15% gains in cultural relevance metrics across SEA while maintaining over 98% of global performance and even occasionally surpassing it. Our findings establish Anthropogenic Regional Alignment as a foundational paradigm towards applicability of multimodal vision-language models in diverse regions and demonstrate a simple-yet-effective baseline method that optimizes regional value alignment while preserving global generalization.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑