“做我的奶酪吗？”：评估多语言大语言模型翻译中的文化细微差别
"Be My Cheese?": Assessing Cultural Nuance in Multilingual LLM Translations
AI总结：
本研究评估多语言大模型翻译习语和双关语时的文化适当性，发现即使高资源语言也常误译，提出文化适当性为关键性能指标，并强调人工润色必要性。
摘要：
这项试点研究探讨了最先进的多语言AI模型在将习语和双关语等比喻性语言从英语翻译成多种全球语言时的本地化能力。该研究在现有的大语言模型翻译研究和行业基准（这些基准强调语法准确性和词元级正确性）的基础上，聚焦于文化适当性和整体本地化质量——这些是营销和电子商务等实际应用中的关键因素。为了研究这些挑战，该项目评估了87个由大语言模型生成的、针对20种语言的24个区域方言的电子商务营销邮件翻译样本。精通每种目标语言的人工审阅者提供了关于忠实于原文语气、含义和目标受众的定量评分和定性反馈。研究结果表明，虽然领先的模型通常能生成语法正确的翻译，但具有文化细微差别的语言仍然是一个明显的改进领域，通常需要大量的人工润色。值得注意的是，即使是高资源全球语言，尽管在行业基准排行榜上名列前茅，也经常误译比喻表达和文字游戏。这项工作挑战了数据量是机器翻译质量最可靠预测指标的假设，并将文化适当性引入为多语言大语言模型性能的关键决定因素——这是现有学术和行业基准中尚未充分探索的领域。作为概念验证，这项试点研究凸显了当前多语言AI系统在实际本地化用例中的局限性。该试点研究的结果支持在更大规模上扩展研究的机会，以提供可推广的见解，并为在文化多样化的环境中部署可靠的机器翻译工作流程提供信息。
英文摘要：
This pilot study explores the localisation capabilities of state-of-the-art multilingual AI models when translating figurative language, such as idioms and puns, from English into a diverse range of global languages. It expands on existing LLM translation research and industry benchmarks, which emphasise grammatical accuracy and token-level correctness, by focusing on cultural appropriateness and overall localisation quality - critical factors for real-world applications like marketing and e-commerce. To investigate these challenges, this project evaluated a sample of 87 LLM-generated translations of e-commerce marketing emails across 24 regional dialects of 20 languages. Human reviewers fluent in each target language provided quantitative ratings and qualitative feedback on faithfulness to the original's tone, meaning, and intended audience. Findings suggest that, while leading models generally produce grammatically correct translations, culturally nuanced language remains a clear area for improvement, often requiring substantial human refinement. Notably, even high-resource global languages, despite topping industry benchmark leaderboards, frequently mistranslated figurative expressions and wordplay. This work challenges the assumption that data volume is the most reliable predictor of machine translation quality and introduces cultural appropriateness as a key determinant of multilingual LLM performance - an area currently underexplored in existing academic and industry benchmarks. As a proof of concept, this pilot highlights limitations of current multilingual AI systems for real-world localisation use cases. Results of this pilot support the opportunity for expanded research at greater scale to deliver generalisable insights and inform deployment of reliable machine translation workflows in culturally diverse contexts.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
