MyCulture：在低资源语言约束下探索马来西亚的多元文化
MyCulture: Exploring Malaysia's Diverse Culture under Low-Resource Language Constraints
AI总结：
针对LLMs在低资源语言中的文化偏见，本文提出马来西亚文化基准MyCulture，采用无预定义选项的开放式多选题格式全面评估LLMs，揭示了模型在文化理解上的显著差异。
摘要：
大型语言模型（LLMs）由于训练数据被英语和中文等高资源语言主导，常常表现出文化偏见。这给准确表示和评估多元文化语境带来了挑战，尤其是在低资源语言环境中。为了解决这个问题，我们引入了 MyCulture，这是一个旨在全面评估 LLMs 在马来西亚文化方面的基准，涵盖马来语呈现的六个支柱：艺术、服饰、习俗、娱乐、食物和宗教。与传统的基准不同，MyCulture 采用了一种新颖的无预定义选项的开放式多选题格式，从而减少了猜测并缓解了格式偏差。我们为这种开放式结构在提高公平性和区分力方面的有效性提供了理论论证。此外，我们通过比较模型在结构化输出与自由格式输出上的表现来分析结构偏差，并通过多语言提示变体来评估语言偏差。我们对一系列区域和国际 LLMs 的评估揭示了文化理解方面的显著差异，突显了在 LLMs 的开发和评估中迫切需要文化扎根且语言包容的基准。
英文摘要：
Large Language Models (LLMs) often exhibit cultural biases due to training data dominated by high-resource languages like English and Chinese. This poses challenges for accurately representing and evaluating diverse cultural contexts, particularly in low-resource language settings. To address this, we introduce MyCulture, a benchmark designed to comprehensively evaluate LLMs on Malaysian culture across six pillars: arts, attire, customs, entertainment, food, and religion presented in Bahasa Melayu. Unlike conventional benchmarks, MyCulture employs a novel open-ended multiple-choice question format without predefined options, thereby reducing guessing and mitigating format bias. We provide a theoretical justification for the effectiveness of this open-ended structure in improving both fairness and discriminative power. Furthermore, we analyze structural bias by comparing model performance on structured versus free-form outputs, and assess language bias through multilingual prompt variations. Our evaluation across a range of regional and international LLMs reveals significant disparities in cultural comprehension, highlighting the urgent need for culturally grounded and linguistically inclusive benchmarks in the development and assessment of LLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
