Large vision-language models (VLMs) can assist visually impaired people by describing images from their daily lives. Current evaluation datasets may not reflect diverse cultural user backgrounds or the situational context of this use case. To address this problem, we create a survey to determine caption preferences and propose a culture-centric evaluation benchmark by filtering VizWiz, an existing dataset with images taken by people who are blind. We then evaluate several VLMs, investigating their reliability as visual assistants in a culturally diverse setting. While our results for state-of-the-art models are promising, we identify challenges such as hallucination and misalignment of automatic evaluation metrics with human judgment. We make our survey, data, code, and model outputs publicly available.
Palm: A Culturally Inclusive and Linguistically Diverse Dataset for Arabic LLMs
The University of British Columbia(不列颠哥伦比亚大学);MBZUAI(穆扎伊布人工智能研究所);Invertible AI(可逆人工智能);Birzeit University(比尔泽特大学);Prince Sultan University(沙特王储大学);UM6P;Cairo University(开罗大学);JUST(朱拉大学);Ain Shams University(艾因·夏姆斯大学);Damascus University(大马士革大学);University of Khartoum(喀土穆大学);Menoufiya University(蒙努菲亚大学);University of Nouakchott(努瓦克肖特大学);National Polytechnic School of Algiers(阿尔及利亚国家多科技术学校);Full Sail University(全帆大学);Alfaisal University(阿尔法西大学);Hamad Bin Khalifa University(哈利法大学)
Navigating the Cultural Kaleidoscope: A Hitchhiker's Guide to Sensitivity in Large Language Models
Indian Institute of Technology Kharagpur(印度理工学院克勒格布尔分校);Microsoft Corporation(微软公司);INSAIT, Sofia University "St. Kliment Ohridski"(索菲亚大学"圣·克利门特·奥赫里德斯基"分校INSAIT研究所)
Whose Morality Do They Speak? Unraveling Cultural Bias in Multilingual Language Models
Research Center Trustworthy Data Science and Security of the University Alliance Ruhr(鲁尔大学联盟可信数据科学与安全研究中心);Faculty of Informatics, Technical University Dortmund(多特蒙德工业大学信息学院)
WorldView-Bench: A Benchmark for Evaluating Global Cultural Perspectives in Large Language Models
Information Technology University Department of Computer Science(信息科技大学计算机科学系);Zayed University College of Interdisciplinary Studies(扎耶德大学跨学科研究学院);Computer Science and Engineering Department(计算机科学与工程系)
CulturePark: Boosting Cross-cultural Understanding in Large Language Models
Institute of Software, CAS(中国科学院软件研究所);Idiap Research Institute(Idiap研究所);Westlake University(西湖大学);Squirrel AI(松鼠AI);Microsoft Research(微软研究院);William & Mary(威廉玛丽学院)
Opportunities and Challenges of Large Language Models for Low-Resource Languages in Humanities Research
School of Computing, The University of Georgia(佐治亚大学计算机学院);Department of Mathematical and Statistical Sciences, University of Alberta(阿尔伯塔大学数学与统计科学系);University of California, Los Angeles(加州大学洛杉矶分校);Department of Radiology, Massachusetts General Hospital and Harvard Medical School(麻省总医院放射科和哈佛医学院)
Creating a Lens of Chinese Culture: A Multimodal Dataset for Chinese Pun Rebus Art Understanding
University of Southern California(南加州大学);Shanghai International Studies University(上海外国语大学);The Butler Museum(巴特勒博物馆);Jingdezhen Imperial Kiln Institute(景德镇御窑研究院);The Ohio State University(俄亥俄州立大学)
本文构建深植中国传统文化的多模态 Pun Rebus Art Dataset,通过元素识别、象征匹配和信息解释任务评估 VLM,揭示其文化理解短板并推动更具包容性的模型发展。
An Inclusive and Lightweight Approach to Federated Continual Learning for Cultural Heritage
Information Technologies Institute, Centre for Research and Technology Hellas(希腊研究与技术中心信息技术研究所);School of Electronic Engineering and Computer Science, Queen Mary University of London(伦敦大学玛丽女王学院电子工程与计算机科学学院)
A Culturally-diverse Multilingual Multimodal Video Benchmark & Model
Mohamed bin Zayed University of Artificial Intelligence(莫德赫·本·扎耶德人工智能大学);University of Central Florida(中央佛罗里达大学);Islamic University of Technology(伊斯兰技术大学);Air University(空军大学);ETH Zurich(苏黎世联邦理工学院);Technische Universität München(慕尼黑技术大学);National Institute of Informatics(国家信息研究所);Australian National University(澳大利亚国立大学);Linköping University(利尔贝里大学)