使用开箱即用的大型语言模型为严重资源不足的语言生成高质量数据到文本
High-quality Data-to-Text Generation for Severely Under-Resourced Languages with Out-of-the-box Large Language Models
AI总结:
本研究探索预训练LLMs在爱尔兰语等严重资源不足语言的数据到文本生成任务中的表现,发现其以大幅优势达到最新技术水平,最佳系统的人工评估表现与人类相当,但BLEU分数的下降质疑了该指标对非任务特定系统的适用性。
摘要:
目前,针对严重资源不足语言的NLP方法无法期望匹及资源丰富语言NLP方法的最新水平。我们通过爱尔兰语、威尔士语、布列塔尼语和马耳他语的数据到文本生成示例,探索预训练大型语言模型(LLMs)能在多大程度上弥合这一差距。我们在这些资源不足语言和英语上,在一系列场景中测试了LLMs。我们发现,LLMs轻松地以大幅优势为这些资源不足语言设定了最新技术水平,这是通过自动和人工评估衡量的。对于我们所有的语言,人工评估显示我们最好的系统与人类表现相当,但BLEU分数与英语相比大幅下降,这让人怀疑该指标在评估非任务特定系统时的适用性。总体而言,我们的结果表明LLMs在弥合资源不足语言性能差距方面具有巨大潜力。
英文摘要:
The performance of NLP methods for severely under-resourced languages cannot currently hope to match the state of the art in NLP methods for well resourced languages. We explore the extent to which pretrained large language models (LLMs) can bridge this gap, via the example of data-to-text generation for Irish, Welsh, Breton and Maltese. We test LLMs on these under-resourced languages and English, in a range of scenarios. We find that LLMs easily set the state of the art for the under-resourced languages by substantial margins, as measured by both automatic and human evaluations. For all our languages, human evaluation shows on-a-par performance with humans for our best systems, but BLEU scores collapse compared to English, casting doubt on the metric's suitability for evaluating non-task-specific systems. Overall, our results demonstrate the great potential of LLMs to bridge the performance gap for under-resourced languages.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。