arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2401.04522cs.CL
收藏

LUNA:一个用于语言理解与自然度评估的框架

LUNA: A Framework for Language Understanding and Naturalness Assessment

HSE University(高等经济大学)

Marat Saidov, Aleksandra Bakalova, Ekaterina Taktasheva, Vladislav Mikhailov, Ekaterina Artemova

AI总结:

LUNA是一个统一框架,提供20个NLG评估指标接口,支持多种文本表示类型,便于扩展新指标,为生成文本评估提供用户友好的工具。

摘要:

对自然语言生成(NLG)模型的评估日益受到关注,促使了评估生成文本各方面的指标的发展。LUNA通过为20个NLG评估指标引入统一接口来解决这一挑战。这些指标根据其参考依赖性和所采用的文本表示类型进行分类,从基于字符串的n-gram重叠到静态嵌入和预训练语言模型的使用。LUNA简洁的设计使其易于用新指标进行扩展,只需几行代码。LUNA提供了一个用户友好的工具来评估生成文本。

英文摘要:

The evaluation of Natural Language Generation (NLG) models has gained increased attention, urging the development of metrics that evaluate various aspects of generated text. LUNA addresses this challenge by introducing a unified interface for 20 NLG evaluation metrics. These metrics are categorized based on their reference-dependence and the type of text representation they employ, from string-based n-gram overlap to the utilization of static embeddings and pre-trained language models. The straightforward design of LUNA allows for easy extension with novel metrics, requiring just a few lines of code. LUNA offers a user-friendly tool for evaluating generated texts.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑