跨文化运动探索：用于评估语言模型对体育运动理解的大规模多语言、多文化基准
Let's Play Across Cultures: A Large Multilingual, Multicultural Benchmark for Assessing Language Models' Understanding of Sports
AI总结：
针对现有语言模型体育评估忽略地区本土传统体育的问题，提出覆盖60国的多语言多文化基准CultSportQA，含3.3万道多模态选择题，通过多种提示法评估不同模型，为AI传统体育理解能力评估设立新标准。
摘要：
语言模型（LMs）主要针对全球流行的体育运动接受评估，往往忽略了地区性和本土体育传统。为填补这一空白，我们提出CultSportQA（文化体育问答）基准，用于评估语言模型对60个国家、6大洲传统体育运动的理解，涵盖四大不同文化类别。该数据集包含33000道文本与图像模态的选择题（MCQs），每道题被划分为三大核心类型：历史类、规则类和场景类。我们在多种大语言模型（LLMs）、小语言模型（SLMs）和多模态大语言模型（MLMs）上，采用零样本、少样本和思维链（CoT）提示法评估模型性能。通过提供一个全面的多语言、多文化体育基准，CultSportQA为评估AI理解和推理传统体育的能力建立了新标准。
英文摘要：
Language Models (LMs) are primarily evaluated on globally popular sports, often overlooking regional and indigenous sporting traditions. To address this gap, we introduce \textbf{\textit{CultSportQA}}, a benchmark designed to assess LMs' understanding of traditional sports across 60 countries and 6 continents, encompassing four distinct cultural categories. The dataset features 33,000 multiple-choice questions (MCQs) across text and image modalities, each of which is categorized into three key types: history-based, rule-based, and scenario-based. To evaluate model performance, we employ zero-shot, few-shot, and chain-of-thought (CoT) prompting across a diverse set of Large Language Models (LLMs), Small Language Models (SLMs), and Multimodal Large Language Models (MLMs). By providing a comprehensive multilingual and multicultural sports benchmark, \textbf{\textit{CultSportQA}} establishes a new standard for assessing AI's ability to understand and reason about traditional sports.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
