arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2108.07374cs.DBcs.CL
收藏

用于自然语言处理和生成的数据集和模型的文档化的可重用模板和指南:HuggingFace和GEM数据和模型卡的案例研究

Reusable Templates and Guides For Documenting Datasets and Models for Natural Language Processing and Generation: A Case Study of the HuggingFace and GEM Data and Model Cards

University of Washington(华盛顿大学);Hugging Face;KNUST(夸梅·恩克鲁玛科技大学);Masakhane;UT Austin(德克萨斯大学奥斯汀分校);IIIT Hyderabad(海得拉巴国际信息技术学院);Google Research(谷歌研究院)

Angelina McMillan-Major, Salomey Osei, Juan Diego Rodriguez, Pawan Sasanka Ammanamanchi, Sebastian Gehrmann, Yacine Jernite

摘要:

为数据集和模型开发文档指南和易于使用的模板是一项具有挑战性的任务,特别是考虑到参与构建自然语言处理(NLP)工具的人员的背景,技能和动机的多样性。尽管如此,在整个NLP领域采用标准文档实践,促进了对NLP数据集和模型的更易访问和更详细的描述,同时支持研究人员和开发人员反思他们的工作。为了帮助文档的标准化,我们提出了两个案例研究,旨在开发可重用的文档模板-HuggingFace数据卡,NLP数据集的通用卡,GEM基准数据和模型卡,重点是自然语言生成。我们描述了我们开发这些模板的过程,包括识别相关的利益相关者群体,定义一套指导原则,使用现有模板作为我们的基础,以及基于反馈的迭代修订。

英文摘要:

Developing documentation guidelines and easy-to-use templates for datasets and models is a challenging task, especially given the variety of backgrounds, skills, and incentives of the people involved in the building of natural language processing (NLP) tools. Nevertheless, the adoption of standard documentation practices across the field of NLP promotes more accessible and detailed descriptions of NLP datasets and models, while supporting researchers and developers in reflecting on their work. To help with the standardization of documentation, we present two case studies of efforts that aim to develop reusable documentation templates -- the HuggingFace data card, a general purpose card for datasets in NLP, and the GEM benchmark data and model cards with a focus on natural language generation. We describe our process for developing these templates, including the identification of relevant stakeholder groups, the definition of a set of guiding principles, the use of existing templates as our foundation, and iterative revisions based on feedback.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑