CLARIN-PT-LDB: 一个开放的大型语言模型排行榜用于葡萄牙语以评估语言、文化与礼貌
CLARIN-PT-LDB: An Open LLM Leaderboard for Portuguese to assess Language, Culture and Civility
AI总结:
本文介绍了CLARIN-PT-LDB,一个用于评估欧洲葡萄牙语大型语言模型的开放排行榜,填补了该语言模型评估的空白,并引入了新的基准测试,包括模型安全性和对葡萄牙文化的对齐性。
摘要:
本文报告了欧洲葡萄牙语(PT-PT)开放大型语言模型(LLM)排行榜及其相关基准测试的开发。该排行榜旨在填补对欧洲葡萄牙语LLM评估的空白,此前尚无专门针对该语言变体的排行榜。本文还报告了新的基准测试,包括一些此前在欧洲葡萄牙语基准测试中未涵盖的方面,如模型安全性和对葡萄牙文化的对齐性。该排行榜可在https://huggingface.co/spaces/PORTULAN/portuguese-llm-leaderboard访问。
英文摘要:
This paper reports on the development of a leaderboard of Open Large Language Models (LLM) for European Portuguese (PT-PT), and on its associated benchmarks. This leaderboard comes as a way to address a gap in the evaluation of LLM for European Portuguese, which so far had no leaderboard dedicated to this variant of the language. The paper also reports on novel benchmarks, including some that address aspects of performance that so far have not been available in benchmarks for European Portuguese, namely model safeguards and alignment to Portuguese culture. The leaderboard is available at https://huggingface.co/spaces/PORTULAN/portuguese-llm-leaderboard.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。