BenCzechMark：面向大型语言模型的以捷克语为中心的多任务多指标基准，采用双评分机制
BenCzechMark : A Czech-centric Multitask and Multimetric Benchmark for Large Language Models with Duel Scoring Mechanism
AI总结：
本文提出BenCzechMark，首个捷克语综合基准，含50项任务和双评分机制，并构建7B捷克语模型及语料库，提供排行榜以评估模型性能。
摘要：
我们提出了BenCzechMark（BCM），这是首个专为大型语言模型设计的综合性捷克语基准，提供多样化的任务、多种任务格式和多种评估指标。其双评分系统基于统计显著性理论，并采用受社会偏好理论启发的跨任务聚合方法。我们的基准包含50项具有挑战性的任务，配有相应的测试数据集，主要使用捷克语原生数据，其中14项为新收集的任务。这些任务涵盖8个类别，覆盖多个领域，包括历史捷克语新闻、学生或语言学习者的作文以及口语。此外，我们收集并清理了BUT-Large Czech Collection，这是最大的公开可用的干净捷克语语料库，并将其用于（i）污染分析和（ii）对首个以捷克语为中心的7B语言模型进行持续预训练，该模型采用捷克语专用分词。我们将我们的模型作为基线，与公开可用的多语言模型进行比较。最后，我们发布并维护一个排行榜，目前已有50个模型提交，新模型提交可在https://huggingface.co/spaces/CZLC/BenCzechMark进行。
英文摘要：
We present BenCzechMark (BCM), the first comprehensive Czech language benchmark designed for large language models, offering diverse tasks, multiple task formats, and multiple evaluation metrics. Its duel scoring system is grounded in statistical significance theory and uses aggregation across tasks inspired by social preference theory. Our benchmark encompasses 50 challenging tasks, with corresponding test datasets, primarily in native Czech, with 14 newly collected ones. These tasks span 8 categories and cover diverse domains, including historical Czech news, essays from pupils or language learners, and spoken word. Furthermore, we collect and clean BUT-Large Czech Collection, the largest publicly available clean Czech language corpus, and use it for (i) contamination analysis and (ii) continuous pretraining of the first Czech-centric 7B language model with Czech-specific tokenization. We use our model as a baseline for comparison with publicly available multilingual models. Lastly, we release and maintain a leaderboard with existing 50 model submissions, where new model submissions can be made at https://huggingface.co/spaces/CZLC/BenCzechMark.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
