Break the Checkbox: Challenging Closed-Style Evaluations of Cultural Alignment in LLMs
Department of Computer Science, National Center for Text Mining, The University of Manchester（计算机科学系，文本挖掘国家中心，曼彻斯特大学）；Department of Computer Science and Engineering, Islamic University of Technology（计算机科学与工程系，伊斯兰技术大学）
A large number of studies rely on closed-style multiple-choice surveys to evaluate cultural alignment in Large Language Models (LLMs). In this work, we challenge this constrained evaluation paradigm and explore more realistic, unconstrained approaches. Using the World Values Survey (WVS) and Hofstede Cultural Dimensions as case studies, we demonstrate that LLMs exhibit stronger cultural alignment in less constrained settings, where responses are not forced. Additionally, we show that even minor changes, such as reordering survey choices, lead to inconsistent outputs, exposing the limitations of closed-style evaluations. Our findings advocate for more robust and flexible evaluation frameworks that focus on specific cultural proxies, encouraging more nuanced and accurate assessments of cultural alignment in LLMs.
Register Shifts Break LLM Safety: A Bengali Benchmark with Culturally Grounded Harms
Institute of Information Technology, University of Dhaka（达卡大学信息技术学院）；University of Maryland, Baltimore County（马里兰大学巴尔的摩县分校）；Bangladesh Air Force Shaheen College Kurmitola（孟加拉国空军库尔米托拉沙欣学院）；Ciroos Inc.（Ciroos公司）
Vernacular? I Barely Know Her: Challenges with Style Control and Stereotyping
National Institute on Drug Abuse, National Institutes of Health（美国国立卫生研究院下属国家药物滥用研究所）；School of Engineering and Applied Science, University of Pennsylvania（宾夕法尼亚大学工程与应用科学学院）
Is AI Catching Up to Human Expression? Exploring Emotion, Personality, Authorship, and Linguistic Style in English and Arabic with Six Large Language Models
College of Computer and Information Sciences, King Saud University（计算机与信息科学学院，沙特阿拉伯国王沙德大学）
The Curious Case of Curiosity across Human Cultures and LLMs
University of Michigan - Ann Arbor（密歇根大学安阿伯分校）；University of Toronto（多伦多大学）；Vector Institute（向量研究所）；MPI for Intelligent Systems, Tubingen, Germany（图宾根德国智能系统研究所）
Interpretable Cross-Lingual Alignment in Small Language Models: Probing Cultural and Pragmatic Reasoning in Japanese-English Bilingual LLMs
Sakaguchi–Inui Laboratory (Tohoku University)（东北大学坂口–乾实验室）；Natural Language Understanding Team at RIKEN AIP（理化学研究所先进智能项目中心自然语言理解团队）；Swallow Project at the Institute of Science Tokyo（东京科学大学Swallow项目）；Sakana AI；Tohoku University（东北大学）；RIKEN AIP（理化学研究所先进智能项目中心）；Institute of Science Tokyo（东京科学大学）
