arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2105.05020cs.CL
收藏

实现NLP共同任务的透明度

Towards transparency in NLP shared tasks

Iconic Translation Machines, Ltd.(艾康尼克翻译机器有限公司);Dublin City University(都柏林城市大学)

Carla Parra Escartín, Teresa Lynn, Joss Moorkens, Jane Dunne

摘要:

本文报告了在自然语言处理(NLP)社区进行的一项调查。该调查旨在收集研究界对围绕共同任务的问题的意见,包括参与和组织。在收到的175份答复中,既有积极的意见,也有消极的意见。我们对这些反应进行了广泛的分析,并提出了报告,这使我们提出了一个共享的任务组织清单,可以支持未来的参与者和组织者。拟议的检查表是灵活的,足以适应我们的领域中的共享任务的广泛多样性,其目标不是指令性的,而是作为一种工具,鼓励共享任务组织者突出道德行为,从175个受访者认为重要的共同问题开始。它的使用不仅可以作为一种工具来反映共同任务的重要方面,而且还可以促进提高这些方面的透明度。

英文摘要:

This article reports on a survey carried out across the Natural Language Processing (NLP) community. The survey aimed to capture the opinions of the research community on issues surrounding shared tasks, with respect to both participation and organisation. Amongst the 175 responses received, both positive and negative observations were made. We carried out and report on an extensive analysis of these responses, which leads us to propose a Shared Task Organisation Checklist that could support future participants and organisers. The proposed Checklist is flexible enough to accommodate the wide diversity of shared tasks in our field and its goal is not to be prescriptive, but rather to serve as a tool that encourages shared task organisers to foreground ethical behaviour, beginning with the common issues that the 175 respondents deemed important. Its usage would not only serve as an instrument to reflect on important aspects of shared tasks, but would also promote increased transparency around them.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑