As large language models (LLMs) expand into performing as agents for real-world applications beyond traditional NLP tasks, evaluating their robustness becomes increasingly important. However, existing benchmarks often overlook critical dimensions like cultural and social awareness. To address these, we introduce CASA, a benchmark designed to assess LLM agents' sensitivity to cultural and social norms across two web-based tasks: online shopping and social discussion forums. Our approach evaluates LLM agents' ability to detect and appropriately respond to norm-violating user queries and observations. Furthermore, we propose a comprehensive evaluation framework that measures awareness coverage, helpfulness in managing user queries, and the violation rate when facing misleading web content. Experiments show that current LLMs perform significantly better in non-agent than in web-based agent environments, with agents achieving less than 10% awareness coverage and over 40% violation rates. To improve performance, we explore two methods: prompting and fine-tuning, and find that combining both methods can offer complementary advantages -- fine-tuning on culture-specific datasets significantly enhances the agents' ability to generalize across different regions, while prompting boosts the agents' ability to navigate complex tasks. These findings highlight the importance of constantly benchmarking LLM agents' cultural and social awareness during the development cycle.
University of Mannheim（曼海姆大学）；University of Technology Nuremberg（纽伦堡技术大学）；National Institute of Information and Communications Technology（信息与通信技术国家研究所）
Evaluating and Improving Cultural Awareness of Reward Models for LLM Alignment
Institute of Computing and Intelligence, Harbin Institute of Technology, Shenzhen, China（计算与智能研究所，哈尔滨工业大学，深圳，中国）；Peng Cheng Laboratory, Shenzhen, China（鹏城实验室，深圳，中国）
The Role of Dataset Linguistic Structure in the Cultural Awareness of Large Language Models
University College London（伦敦大学学院）；Queen’s University Belfast（贝尔法斯特女王大学）；The University of Tokyo（东京大学）；University of Bisha（比沙大学）；King Abdulaziz University（阿卜杜勒阿齐兹国王大学）；AI Centre, University College London（伦敦大学学院人工智能中心）
Probing Cultural Awareness in LLMs: A Case Study of Cross-Culture Aesthetic Stylistics
Department of Computing, Hong Kong Polytechnic University（香港理工大学计算机系）；Institute of Digital Twin, Eastern Institute of Technology, Ningbo（宁波东部技术研究所数字孪生研究所）；Department of Language Science and Technology, Hong Kong Polytechnic University（香港理工大学语言科学与技术系）；School of Design, Hong Kong Polytechnic University（香港理工大学设计学院）；Research Institute for Quantum Technology, Hong Kong Polytechnic University（香港理工大学量子技术研究所）；Department of Communication Science, Vrije Universiteit Amsterdam（阿姆斯特丹自由大学传播科学系）
MCEval: A Dynamic Framework for Fair Multilingual Cultural Evaluation of LLMs
Zhejiang University（浙江大学）；School of Engineering, Westlake University（西溪大学工程学院）；Southern University of Science and Technology（南方科技大学）；Institute of Advanced Technology, Westlake Institute for Advanced Study（西溪研究院先进技术研究所）
MMA-ASIA: A Multilingual and Multimodal Alignment Framework for Culturally-Grounded Evaluation
Singapore University of Technology and Design（新加坡科技设计大学）；Agency for Science, Technology and Research, Singapore（新加坡科技研究局）；Indian Institute of Technology Delhi（印度理工学院德里分校）；Alibaba DAMO Academy（阿里巴巴达摩院）；Microsoft Research Asia（微软亚洲研究院）；Shanghai University of Finance and Economics（上海财经大学）；Inner Mongolia University（内蒙古大学）；Kyoto University（京都大学）；Jiangxi Normal University（江西师范大学）；Korea University（韩国大学）；Nanyang Technological University（南洋理工大学）
LLMs as Cultural Archives: Cultural Commonsense Knowledge Graph Extraction
Mohamed bin Zayed University of Artificial Intelligence（莫扎德·本·扎耶德人工智能大学）；Ubiquitous Knowledge Processing Lab (UKP Lab)（无处不在知识处理实验室）；Department of Computer Science（计算机科学系）；Hessian Center for AI (hessian.AI)（黑森人工智能中心）；Technische Universität Darmstadt（德累斯顿技术大学）
DIWALI: Diversity and Inclusivity aWare cuLture specific Items for India: Dataset and Assessment of LLMs for Cultural Text Adaptation in Indian Context
Natural Language and Information Processing Lab (NLIP) Indian Institute of Technology Hyderabad（自然语言与信息处理实验室（NLIP）印度理工学院海得拉巴）
DFKI-MLT at SemEval-2026 TASK 7: Steering Multilingual Models Towards Cultural Knowledge
German Research Center for Artificial Intelligence (DFKI GmbH)（德国人工智能研究中心（DFKI GmbH））；Saarland Informatics Campus（萨尔布吕肯信息学校区）；Barcelona Supercomputing Center (BSC-CNS)（巴塞罗那超级计算中心（BSC-CNS））
Crossing Borders Without Crossing Boundaries: How Sociolinguistic Awareness Can Optimize User Engagement with Localized Spanish AI Models Across Hispanophone Countries
