迷失于变异？评估巴斯克语和西班牙语地理变体中的 NLI 性能
Lost in Variation? Evaluating NLI Performance in Basque and Spanish Geographical Variants
AI总结：
该研究构建巴斯克语与西班牙语地理变体的人工平行 NLI 数据集，系统评估跨语言及上下文学习中的 LLM 表现，揭示语言变异本身导致性能下降。
摘要：
本文评估当前语言技术理解巴斯克语和西班牙语语言变体的能力。我们将自然语言推理（NLI）作为枢纽任务，并引入一个新颖的、人工整理的巴斯克语和西班牙语平行数据集，涵盖它们各自的变体。我们使用仅编码器和解码器式大语言模型（LLM）进行跨语言和上下文学习实验，实证分析表明，模型在处理语言变异时性能下降，尤其是在巴斯克语中。错误分析表明，这种下降并非由词汇重叠造成，而是源于语言变异本身。进一步的消融实验表明，仅编码器模型尤其难以处理西巴斯克语，这与语言学理论一致：该理论认为边缘方言（如西部方言）与标准语距离更远。所有数据和代码均公开可用。
英文摘要：
In this paper, we evaluate the capacity of current language technologies to understand Basque and Spanish language varieties. We use Natural Language Inference (NLI) as a pivot task and introduce a novel, manually-curated parallel dataset in Basque and Spanish, along with their respective variants. Our empirical analysis of crosslingual and in-context learning experiments using encoder-only and decoder-based Large Language Models (LLMs) shows a performance drop when handling linguistic variation, especially in Basque. Error analysis suggests that this decline is not due to lexical overlap, but rather to the linguistic variation itself. Further ablation experiments indicate that encoder-only models particularly struggle with Western Basque, which aligns with linguistic theory that identifies peripheral dialects (e.g., Western) as more distant from the standard. All data and code are publicly available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
