跨语言响应一致性在大语言模型中的研究：基于ILR的Claude多语言评估
Cross-Lingual Response Consistency in Large Language Models: An ILR-Informed Evaluation of Claude Across Six Languages
AI总结：
本文基于ILR技能描述系统，评估Claude在六种语言中的响应一致性，发现法语响应比德语更长30%，且创造性输出存在显著跨语言差异，提出ILR指导的专家评估方法补充了纯计算基准。
摘要：
本文介绍了一种基于Interagency Language Roundtable（ILR）技能描述的系统评估框架，并将其应用于Claude（Sonnet 4.6）六种语言：英语、法语、罗马尼亚语、西班牙语、意大利语和德语。我们设计了12个语义等价的提示集群，覆盖ILR复杂度等级1至3+，收集了216个响应（12个提示，6种语言，3次运行），并通过两层方法分析输出：结合自动化定量指标与专家ILR定性评估。定量分析显示，法语响应在相同提示下比德语响应长约30%，创造性与情感性集群表现出最高的跨语言表面差异。定性分析由一位具备12年ILR/OPI评估经验的六语言专家完成，识别出五种跨语言差异模式：语用歧义策略的系统差异、创造性输出的美学和文学传统差异、语言内部技术术语规范、文化校准缺口（表现为缺乏文化特定内容而采用文化中性模板）、以及情感支持响应中的语言特定机构转介行为。我们认为，应用于LLM输出的ILR指导专家判断构成了一种新颖且未被充分报告的评估方法，补充了纯计算基准，并认为Claude的跨语言输出差异具有可解释性、领域依赖性和对公平多语言AI部署的重要性。
英文摘要：
This paper introduces a systematic evaluation framework grounded in the Interagency Language Roundtable (ILR) Skill Level Descriptions and applies it to Claude (Sonnet 4.6) across six languages: English, French, Romanian, Spanish, Italian, and German. We administer a battery of 12 semantically equivalent prompt clusters spanning ILR complexity levels 1 through 3+, collect 216 responses (12 prompts, 6 languages, 3 runs), and analyze outputs through a two-layer methodology combining automated quantitative metrics with expert ILR qualitative assessment. Quantitative analysis reveals that French responses are approximately 30% longer than German responses on identical prompts, and that creative and affective clusters show the highest cross-lingual surface divergence. Qualitative analysis, conducted by a six-language professional with 12 years of ILR/OPI assessment experience, identifies five cross-lingual variation patterns: systematic differences in pragmatic disambiguation strategies, aesthetic and literary tradition divergence in creative output, language-internal technical terminology norms, cultural calibration gaps evidenced by the absence of culture-specific content in favor of culturally neutralized templates, and language-specific institutional referral behavior in emotional support responses. We argue that ILR-informed expert judgment applied to LLM outputs constitutes a novel and underreported evaluation methodology that complements purely computational benchmarks, and that cross-lingual output variation in Claude is interpretable, domain-dependent, and consequential for equitable multilingual AI deployment.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
