谁把复活节彩蛋带到了开斋节?跨语言和地区数学应用题的文化翻译审计
Who Brought Easter Eggs to Eid? Auditing LLM-Generated Cultural Translation of Math Word Problems Across Languages and Regions
AI总结:
本研究审计了三个大型语言模型将60个英语数学应用题翻译为7种语言时的文化适应性,发现模型在62.5%的案例中一致,但仅33.5%有相同替换,且所有组合均出现熵塌缩,优先改变表面标记而保留深层结构,导致文化多样性压缩和区域误归因。
摘要:
大型语言模型越来越多地被用于大规模个性化学习中改编数学应用题,但这些改编是否跨模型一致、是否在规模上保留文化多样性、以及揭示模型认为哪些文化实体最显著,仍是未解决的问题。我们分析了Claude Opus 4、GPT-4.1和Gemini 2.5 Pro如何将60个英语数学应用题改编为孟加拉语、印地语、旁遮普语(印度)、乌尔都语、信德语(巴基斯坦)、意大利语和西西里语(意大利),这一语言集涵盖了从高资源语言(意大利语和印地语)到研究不足的语言(信德语、西西里语和旁遮普语)的完整资源谱系。我们标注了6,489个实体转换,编码模型是否保留、本地化、泛化、省略或更改名称、食物和地点等实体。模型在62.5%的案例中在转换类型上一致,在特定替换上仅33.5%一致,这意味着模型选择直接塑造了学生遇到的文化世界。所有21种语言-模型组合均出现熵塌缩,改编压缩而非扩展了文化多样性。模型优先处理表面标记(如名称、食物和货币),同时保留更深层的结构特征(如嵌入特定文化假设的年级系统)。尽管提示指定了目标国家,模型仍错误归因区域背景,例如对印度孟加拉语学生使用孟加拉国塔卡,并产生跨文化污染,例如将寻蛋活动改编为开斋节活动。某些失败在单个翻译中可见。其他失败,包括多样性塌缩、对表面标记的系统性偏好以及一致的区域误归因,仅通过语料库级分析才显现。使改编问题看起来正确的表面合理性,正是使深层失败容易被忽视的原因。
英文摘要:
Large language models are increasingly used to adapt math word problems for personalized learning at scale, but it remains an open question whether those adaptations are consistent across models, preserve cultural diversity at scale, and reveal which cultural entities models treat as most salient. We analyze how Claude Opus 4, GPT-4.1, and Gemini 2.5 Pro adapt 60 English math word problems into Bengali, Hindi, Punjabi (India), Urdu, Sindhi (Pakistan), Italian, and Sicilian (Italy), a language set spanning the full resource spectrum, from high-resource Italian and Hindi to under-studied Sindhi, Sicilian, and Punjabi. We annotate 6,489 entity transformations, coding whether models preserve, localize, generalize, omit, or change entities such as names, foods, and places. Models agree on transformation type in 62.5% of cases and on specific substitutions in only 33.5%, meaning model choice directly shapes which cultural world students encounter. All 21 language-model combinations show entropy collapse, with adaptation compressing rather than expanding cultural diversity. Models prioritize surface markers such as names, foods, and currencies while preserving deeper structural features such as grade-level systems that embed culturally specific assumptions. Despite prompts specifying target countries, models misattribute regional context by using Bangladeshi taka for Indian Bengali students and produce cross-cultural contamination, such as adapting egg hunts as Eid activities. Some failures are visible in individual translations. Others, including diversity collapse, systematic preference for surface markers, and consistent regional misattribution, emerge only through corpus-level analysis. The surface plausibility that makes adapted problems look correct is precisely what makes deeper failures easy to overlook.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。