Disce aut Deficere：在 INVALSI 意大利语基准上评估 LLMs 的熟练度
Disce aut Deficere: Evaluating LLMs Proficiency on the INVALSI Italian Benchmark
AI总结：
本研究引入基于 INVALSI 测试的意大利语结构化基准，评估 LLMs 在非英语环境下的语言能力，通过适配测试格式、详细评估现有模型并与人类表现对比，为学术界提供关键参考。
摘要：
大型语言模型（LLMs）的最新进展显著增强了其生成和处理人类语言的能力，突显了它们在各种应用中的潜力。在英语以外的语言中评估 LLMs 对于确保其语言多样性、文化相关性以及在多样化全球环境中的适用性至关重要，从而拓宽其可用性和有效性。我们通过引入使用 INVALSI 测试的结构化基准来应对这一挑战，INVALSI 测试是一套旨在衡量意大利各地教育能力的成熟评估体系。本研究做出三项主要贡献：首先，我们将 INVALSI 基准适配于自动化 LLM 评估，这涉及对测试格式进行严格调整以适应自动化处理，同时保留原始测试的本质。其次，我们对当前的 LLMs 进行了详细评估，为学术界提供了关键参考点。最后，我们将这些模型的性能与人类结果进行了可视化比较。此外，我们邀请研究人员提交其模型进行持续评估，以确保该基准保持时效性并成为有价值的资源。
英文摘要：
Recent advancements in Large Language Models (LLMs) have significantly enhanced their ability to generate and manipulate human language, highlighting their potential across various applications. Evaluating LLMs in languages other than English is crucial for ensuring their linguistic versatility, cultural relevance, and applicability in diverse global contexts, thus broadening their usability and effectiveness. We tackle this challenge by introducing a structured benchmark using the INVALSI tests, a set of well-established assessments designed to measure educational competencies across Italy. Our study makes three primary contributions: Firstly, we adapt the INVALSI benchmark for automated LLM evaluation, which involves rigorous adaptation of the test format to suit automated processing while retaining the essence of the original tests. Secondly, we provide a detailed assessment of current LLMs, offering a crucial reference point for the academic community. Finally, we visually compare the performance of these models against human results. Additionally, researchers are invited to submit their models for ongoing evaluation, ensuring the benchmark remains a current and valuable resource.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
