arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2109.04817cs.CL
收藏

是否捕获STEL?基于相似度的模块化语言风格评价框架

Does It Capture STEL? A Modular, Similarity-based Linguistic Style Evaluation Framework

Utrecht University(乌得勒支大学)

Anna Wegmann, Dong Nguyen

摘要:

风格是自然语言的组成部分。然而,评估方法的风格措施是罕见的,往往是特定的任务,通常不控制的内容。我们提出了模块化,细粒度和内容控制的相似性为基础的风格评估框架(STEL),以测试任何模型的性能,可以比较两个句子的风格。我们用风格的两个一般维度(正式/非正式和简单/复杂)以及风格的两个具体特征(压缩和替代)来说明STEL。我们发现,基于BERT的方法优于简单版本的常用的风格措施,如3克,标点符号频率和基于LIWC的方法。我们邀请向STEL添加更多的任务和任务实例,并希望促进风格敏感措施的改进。

英文摘要:

Style is an integral part of natural language. However, evaluation methods for style measures are rare, often task-specific and usually do not control for content. We propose the modular, fine-grained and content-controlled similarity-based STyle EvaLuation framework (STEL) to test the performance of any model that can compare two sentences on style. We illustrate STEL with two general dimensions of style (formal/informal and simple/complex) as well as two specific characteristics of style (contrac'tion and numb3r substitution). We find that BERT-based methods outperform simple versions of commonly used style measures like 3-grams, punctuation frequency and LIWC-based approaches. We invite the addition of further tasks and task instances to STEL and hope to facilitate the improvement of style-sensitive measures.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑