arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2103.09710cs.CL
收藏

人体评价数据表1.0:用于记录NLP中人体评价实验细节的模板

The Human Evaluation Datasheet 1.0: A Template for Recording Details of Human Evaluation Experiments in NLP

Université de Lorraine(洛林大学);LORIA(洛林计算机科学与应用实验室);University of Brighton(布莱顿大学)

Anastasia Shimorina, Anya Belz

摘要:

本文介绍了人类评估数据表,一个模板,用于记录个人的自然语言处理(NLP)的评估实验的细节。人体评价数据表最初从Bender和Friedman(2018)、Mitchell等人(2019)和Gebru等人(2020)的开创性论文中获得灵感,旨在促进充分详细记录人体评价的特性,并进行充分的标准化,以支持可比性、荟萃评价和再现性测试。

英文摘要:

This paper introduces the Human Evaluation Datasheet, a template for recording the details of individual human evaluation experiments in Natural Language Processing (NLP). Originally taking inspiration from seminal papers by Bender and Friedman (2018), Mitchell et al. (2019), and Gebru et al. (2020), the Human Evaluation Datasheet is intended to facilitate the recording of properties of human evaluations in sufficient detail, and with sufficient standardisation, to support comparability, meta-evaluation, and reproducibility tests.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑