MMA-ASIA: A Multilingual and Multimodal Alignment Framework for Culturally-Grounded Evaluation
Singapore University of Technology and Design(新加坡科技设计大学);Agency for Science, Technology and Research, Singapore(新加坡科技研究局);Indian Institute of Technology Delhi(印度理工学院德里分校);Alibaba DAMO Academy(阿里巴巴达摩院);Microsoft Research Asia(微软亚洲研究院);Shanghai University of Finance and Economics(上海财经大学);Inner Mongolia University(内蒙古大学);Kyoto University(京都大学);Jiangxi Normal University(江西师范大学);Korea University(韩国大学);Nanyang Technological University(南洋理工大学)
Large language models (LLMs) are now used worldwide, yet their multimodal understanding and reasoning often degrade outside Western, high-resource settings. We propose MMA-ASIA, a comprehensive framework to evaluate LLMs' cultural awareness with a focus on Asian contexts. MMA-ASIA centers on a human-curated, multilingual, and multimodally aligned multiple-choice benchmark covering 8 Asian countries and 10 languages, comprising 27,000 questions; over 79 percent require multi-step reasoning grounded in cultural context, moving beyond simple memorization. To our knowledge, this is the first dataset aligned at the input level across three modalities: text, image (visual question answering), and speech. This enables direct tests of cross-modal transfer. Building on this benchmark, we propose a five-dimensional evaluation protocol that measures: (i) cultural-awareness disparities across countries, (ii) cross-lingual consistency, (iii) cross-modal consistency, (iv) cultural knowledge generalization, and (v) grounding validity. To ensure rigorous assessment, a Cultural Awareness Grounding Validation Module detects "shortcut learning" by checking whether the requisite cultural knowledge supports correct answers. Finally, through comparative model analysis, attention tracing, and an innovative Vision-ablated Prefix Replay (VPR) method, we probe why models diverge across languages and modalities, offering actionable insights for building culturally reliable multimodal LLMs.
CultureConverse: A Multilingual Multi-turn Simulation Harness for Culturally Grounded Assistance in East and Southeast Asia
Singapore University of Technology and Design (SUTD)(新加坡科技设计大学);Agency for Science, Technology and Research (A*STAR)(新加坡科技研究局);École Polytechnique Fédérale de Lausanne (EPFL)(洛桑联邦理工学院);Microsoft Research Asia (MSRA)(微软亚洲研究院);Sungkyunkwan University (SKKU)(成均馆大学);Universiti Brunei Darussalam (UBD)(文莱大学);China University of Petroleum (East China)(中国石油大学(华东));Nanyang Technological University (NTU)(南洋理工大学);Kyoto University(京都大学);Konkuk University(建国大学);Upstage AI
Camellia: Benchmarking Cultural Biases in LLMs for Asian Languages
Georgia Institute of Technology(佐治亚理工学院);Samsung R&D Institute Philippines(三星菲律宾研发院);Sungkyunkwan University(成均馆大学);Tohoku University(东北大学);National University of Singapore(新加坡国立大学);University of Copenhagen(哥本哈根大学);University of Michigan(密歇根大学);Indian Institute of Technology Delhi(印度理工学院德里);Institute of Science Tokyo(东京科学研究院)
LaoBench: A Large-Scale Multidimensional Lao Benchmark for Large Language Models
China-ASEAN Information Harbor Co., Ltd.(中国-东盟信息港有限公司);Beijing Academy of Artificial Intelligence(北京人工智能研究院);School of Software & Microelectronics, Peking University(北京大学软件与微电子学院)
SAGE: Sustainable Agent-Guided Expert-tuning for Culturally Attuned Translation in Low-Resource Southeast Asia
Xi'an Jiaotong-Liverpool University(西安交通大学利物浦大学);University of Liverpool(利物浦大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学)
Counterfactual Cultural Cues Reduce Medical QA Accuracy in LLMs: Identifier vs Context Effects
Institute of Health Policy, Management, and Evaluation (IHPME)(健康政策、管理与评估研究所);Dalla Lana School of Public Health(达拉兰公共卫生学院);University of Toronto(多伦多大学);Faculty of Information(信息学院);Schwartz Reisman Institute(施瓦茨-雷曼研究所)
Transcending Language Boundaries: Harnessing LLMs for Low-Resource Language Translation
School of Computing, The University of Georgia(佐治亚大学计算学院);Second Language Acquisition and Teaching, University of Arizona(亚利桑那大学第二语言习得与教学系);University of Alberta(阿尔伯塔大学);Department of History, The University of Georgia(佐治亚大学历史系);National GENIUS Center(国家GENIUS中心);AI4STEM Education Center, University of Georgia(佐治亚大学AI4STEM教育中心);Department of Mathematics, Science, and Social Studies Education, University of Georgia(佐治亚大学数学、科学与社会研究教育系)
CultureVLM: Characterizing and Improving Cultural Understanding of Vision-Language Models for over 100 Countries
University of Macau(澳门大学);Georgia Institute of Technology(佐治亚理工学院);Microsoft Research(微软研究院);Squirrel AI(松鼠AI);Lehigh University(理海大学);William & Mary(威廉玛丽学院)
MemeCULT-1K: Benchmarking South Asian Cultural Context and Humor Understanding of Multimodal Models
Islamic University of Technology(伊斯兰科技大学);South East University(东南大学);Vector Institute(矢量研究所);University of British Columbia(不列颠哥伦比亚大学);York University(约克大学)
Multilingual Dyadic Interaction Corpus NoXi+J: Toward Understanding Asian-European Non-verbal Cultural Characteristics and their Influences on Engagement
University of Augsburg(奥格斯堡大学);Japan Advanced Institute of Science and Technology(日本先进科学技术学院)
When Meaning Travels: A Granular Lens on Hybrid-MoE's Role in Idiomatic Understanding for Language Models
Department of Computer Science and Engineering, Indian Institute of Technology Patna, India(印度理工学院帕纳瓦分校计算机科学与工程系);School of Information Technology, King Mongkut’s Institute of Technology Ladkrabang, Thailand(泰国拉差班国王理工大学信息科技学院)