GAIA-v2-LILT：超越翻译的代理基准多语言适应
GAIA-v2-LILT: Multilingual Adaptation of Agent Benchmark beyond Translation
AI总结：
本文提出GAIA-v2-LILT，通过功能对齐、文化对齐和难度校准改进多语言代理基准，实验表明其性能提升达32.7%，接近英语表现，但仍有较大差距，揭示了多语言性能差距主要由基准测量误差引起。
摘要：
代理基准仍以英语为中心，其多语言版本常依赖机器翻译和有限后编辑。我们主张，对于代理任务，这种最小工作流程可能通过查询-答案不一致或文化无关上下文破坏基准有效性。我们提出一种改进的工作流程，通过自动化检查和人工审查，实现英语基准到多种语言的显式功能对齐、文化对齐和难度校准。使用此流程，我们引入GAIA-v2-LILT，即覆盖五种非英语语言的重新审核多语言扩展。实验表明，此流程将代理成功率提升高达32.7%，使最接近英语表现的审核设置达到3.1%差距，但许多其他情况下仍有显著差距。这表明多语言性能差距的大部分是基准诱导的测量误差，推动在跨语言适应英语基准时进行任务层面的对齐。数据作为MAPS包的一部分在https://huggingface.co/datasets/Fujitsu-FRE/MAPS/viewer/GAIA-v2-LILT提供。我们还发布了实验中使用的代码在https://github.com/lilt/gaia-v2-lilt。
英文摘要：
Agent benchmarks remain largely English-centric, while their multilingual versions are often built with machine translation (MT) and limited post-editing. We argue that, for agentic tasks, this minimal workflow can easily break benchmark validity through query-answer misalignment or culturally off-target context. We propose a refined workflow for adapting English benchmarks into multiple languages with explicit functional alignment, cultural alignment, and difficulty calibration using both automated checks and human review. Using this workflow, we introduce GAIA-v2-LILT, a re-audited multilingual extension of GAIA covering five non-English languages. In experiments, our workflow improves agent success rates by up to 32.7% over minimally translated versions, bringing the closest audited setting to within 3.1% of English performance while substantial gaps remain in many other cases. This indicates that a substantial share of the multilingual performance gap is benchmark-induced measurement error, motivating task-level alignment when adapting English benchmarks across languages. The data is available as part of the MAPS package at https://huggingface.co/datasets/Fujitsu-FRE/MAPS/viewer/GAIA-v2-LILT. We also release the code used in our experiments at https://github.com/lilt/gaia-v2-lilt.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
