ReadMe++：面向多领域可读性评估的多语言语言模型基准测试
ReadMe++: Benchmarking Multilingual Language Models for Multi-Domain Readability Assessment
AI总结：
本文提出了多语言多领域可读性评估基准ReadMe++，包含9757个句子的人工标注，用于全面评估大型语言模型在有监督、无监督和少样本提示设置下的表现，并发现其具有出色的领域泛化和跨语言迁移能力。
摘要：
我们提出了对大型语言模型在多语言可读性评估方面的全面评估。现有的评估资源缺乏领域和语言多样性，限制了跨领域和跨语言分析的能力。本文介绍了ReadMe++，这是一个多语言多领域数据集，包含对阿拉伯语、英语、法语、印地语和俄语的9757个句子的人工标注，数据收集自112个不同的数据源。该基准测试将鼓励开发稳健的多语言可读性评估方法的研究。使用ReadMe++，我们在有监督、无监督和少样本提示设置下对多语言和单语言语言模型进行了基准测试。ReadMe++中的领域和语言多样性使我们能够测试更有效的少样本提示，并识别出最先进的无监督方法中的不足。我们的实验还揭示了在ReadMe++上训练的模型具有出色的领域泛化和增强的跨语言迁移能力的令人兴奋的结果。我们将公开我们的数据，并发布一个python包工具，用于使用我们训练的模型进行多语言句子可读性预测：https://github.com/tareknaous/readme
英文摘要：
We present a comprehensive evaluation of large language models for multilingual readability assessment. Existing evaluation resources lack domain and language diversity, limiting the ability for cross-domain and cross-lingual analyses. This paper introduces ReadMe++, a multilingual multi-domain dataset with human annotations of 9757 sentences in Arabic, English, French, Hindi, and Russian, collected from 112 different data sources. This benchmark will encourage research on developing robust multilingual readability assessment methods. Using ReadMe++, we benchmark multilingual and monolingual language models in the supervised, unsupervised, and few-shot prompting settings. The domain and language diversity in ReadMe++ enable us to test more effective few-shot prompting, and identify shortcomings in state-of-the-art unsupervised methods. Our experiments also reveal exciting results of superior domain generalization and enhanced cross-lingual transfer capabilities by models trained on ReadMe++. We will make our data publicly available and release a python package tool for multilingual sentence readability prediction using our trained models at: https://github.com/tareknaous/readme
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
