使用西班牙语作为外语的测试评估大型语言模型：通过还是失败？
Evaluating Large Language Models with Tests of Spanish as a Foreign Language: Pass or Fail?
AI总结：
本文评估了最先进的大型语言模型在西班牙语外语考试基准TELEIA上的表现，发现其虽能较好理解西班牙语，但语法能力仍远不及母语者水平。
摘要：
大型语言模型（LLMs）在回答多领域问题及执行不同自然语言理解任务的性能上已被大量评估。这些测试通常以英语进行，但大多数LLM用户并非英语母语者。因此，分析LLMs在不同层级（从段落到语素）上如何理解其他语言具有重要意义。本文评估了最先进的LLMs在TELEIA中的表现，这是一个最近发布的基准，包含与面向外国学生的西班牙语考试类似的问题，涵盖阅读理解、构词、意义与组合语义以及语法等主题。结果表明，LLMs在理解西班牙语方面表现良好，但在语法能力上仍远未达到母语者的水平。
英文摘要：
Large Language Models (LLMs) have been profusely evaluated on their ability to answer questions on many topics and their performance on different natural language understanding tasks. Those tests are usually conducted in English, but most LLM users are not native English speakers. Therefore, it is of interest to analyze how LLMs understand other languages at different levels: from paragraphs to morphems. In this paper, we evaluate the performance of state-of-the-art LLMs in TELEIA, a recently released benchmark with similar questions to those of Spanish exams for foreign students, covering topics such as reading comprehension, word formation, meaning and compositional semantics, and grammar. The results show that LLMs perform well at understanding Spanish but are still far from achieving the level of a native speaker in terms of grammatical competence.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
