XIFBench：评估大型语言模型在多语言指令遵循方面的能力
XIFBench: Evaluating Large Language Models on Multilingual Instruction Following
AI总结：
本文提出XIFBench基准，通过558条多语言指令和细粒度约束评估LLMs的指令遵循能力，并引入三项方法论创新支持跨语言评估，揭示了语言资源等因素对多语言指令遵循的影响。
摘要：
大型语言模型（LLMs）在各种应用中展现了卓越的指令遵循能力。然而，其在多语言环境下的表现缺乏系统调查，现有评估在不同语言语境下缺乏细粒度约束分析。我们引入了XIFBench，这是一个基于约束的综合基准，用于评估LLMs的多语言指令遵循能力，包含558条指令，涵盖六种不同资源水平的语言，并设有五个类别（内容、风格、情境、格式和数值）的0-5个附加约束。为支持可靠且一致的跨语言评估，我们实施了三项方法论创新：文化可及性标注、约束级翻译验证，以及以英语要求作为跨语言语义锚点的基于需求的评估。对多种LLMs的广泛实验不仅量化了不同资源水平间的性能差异，还详细揭示了语言资源、约束类别、指令复杂度和文化特异性如何影响多语言指令遵循。我们的代码和数据可在https://github.com/zhenyuli801/XIFBench获取。
英文摘要：
Large Language Models (LLMs) have demonstrated remarkable instruction-following capabilities across various applications. However, their performance in multilingual settings lacks systematic investigation, with existing evaluations lacking fine-grained constraint analysis across diverse linguistic contexts. We introduce XIFBench, a comprehensive constraint-based benchmark for evaluating multilingual instruction-following abilities of LLMs, comprising 558 instructions with 0-5 additional constraints across five categories (Content, Style, Situation, Format, and Numerical) in six languages spanning different resource levels. To support reliable and consistent cross-lingual evaluation, we implement three methodological innovations: cultural accessibility annotation, constraint-level translation validation, and requirement-based evaluation using English requirements as semantic anchors across languages. Extensive experiments with various LLMs not only quantify performance disparities across resource levels but also provide detailed insights into how language resources, constraint categories, instruction complexity, and cultural specificity influence multilingual instruction-following. Our code and data are available at https://github.com/zhenyuli801/XIFBench.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
