arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.12872cs.CL
收藏

CLARIN-PT-LDB: 一个开放的大型语言模型排行榜用于葡萄牙语以评估语言、文化与礼貌

CLARIN-PT-LDB: An Open LLM Leaderboard for Portuguese to assess Language, Culture and Civility

University of Lisbon NLX—Grupo de Fala e Linguagem Natural, Departmento de Informática Faculdade de Ciências, Campo Grande, 1749-016 Lisboa, Portugal(里斯本大学 NLX—语言与语言自然组,计算机学院,Campo Grande, 1749-016里斯本,葡萄牙)

João Silva, Luís Gomes, António Branco

AI总结:

本文介绍了CLARIN-PT-LDB,一个用于评估欧洲葡萄牙语大型语言模型的开放排行榜,填补了该语言模型评估的空白,并引入了新的基准测试,包括模型安全性和对葡萄牙文化的对齐性。

摘要:

本文报告了欧洲葡萄牙语(PT-PT)开放大型语言模型(LLM)排行榜及其相关基准测试的开发。该排行榜旨在填补对欧洲葡萄牙语LLM评估的空白,此前尚无专门针对该语言变体的排行榜。本文还报告了新的基准测试,包括一些此前在欧洲葡萄牙语基准测试中未涵盖的方面,如模型安全性和对葡萄牙文化的对齐性。该排行榜可在https://huggingface.co/spaces/PORTULAN/portuguese-llm-leaderboard访问。

英文摘要:

This paper reports on the development of a leaderboard of Open Large Language Models (LLM) for European Portuguese (PT-PT), and on its associated benchmarks. This leaderboard comes as a way to address a gap in the evaluation of LLM for European Portuguese, which so far had no leaderboard dedicated to this variant of the language. The paper also reports on novel benchmarks, including some that address aspects of performance that so far have not been available in benchmarks for European Portuguese, namely model safeguards and alignment to Portuguese culture. The leaderboard is available at https://huggingface.co/spaces/PORTULAN/portuguese-llm-leaderboard.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑