arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2111.04424cs.CYcs.AI
收藏

弃用数据集的框架:文档、标识和通信的标准化

A Framework for Deprecating Datasets: Standardizing Documentation, Identification, and Communication

Hugging Face;University of Southern California(南加州大学);New York University(纽约大学);Microsoft Research(微软研究院)

Alexandra Sasha Luccioni, Frances Corry, Hamsini Sridharan, Mike Ananny, Jason Schultz, Kate Crawford

摘要:

数据集是训练机器学习(ML)模型的核心。ML社区最近对模型开发生存周期中的数据管理和文档实践进行了重大改进。然而,弃用或删除数据集的行为在很大程度上被忽视了,目前还没有标准化的方法来构建数据集生存周期的这一阶段。在本文中,我们研究了ML中数据集弃用的实践,确定了几个尽管已被弃用但仍继续流通的数据集案例,并描述了这种延续所带来的不同技术,法律,道德和组织问题。然后,我们提出了一个数据集弃用框架,其中包括考虑风险,减轻影响,上诉机制,时间轴,弃用后协议和发布检查,可以由ML社区进行调整和实施。最后,我们建议创建一个集中的,可持续的存储库系统,用于归档数据集,跟踪数据集的修改或弃用,并促进可以集成到研究和出版过程中的护理和管理实践。

英文摘要:

Datasets are central to training machine learning (ML) models. The ML community has recently made significant improvements to data stewardship and documentation practices across the model development life cycle. However, the act of deprecating, or deleting, datasets has been largely overlooked, and there are currently no standardized approaches for structuring this stage of the dataset life cycle. In this paper, we study the practice of dataset deprecation in ML, identify several cases of datasets that continued to circulate despite having been deprecated, and describe the different technical, legal, ethical, and organizational issues raised by such continuations. We then propose a Dataset Deprecation Framework that includes considerations of risk, mitigation of impact, appeal mechanisms, timeline, post-deprecation protocols, and publication checks that can be adapted and implemented by the ML community. Finally, we propose creating a centralized, sustainable repository system for archiving datasets, tracking dataset modifications or deprecations, and facilitating practices of care and stewardship that can be integrated into research and publication processes.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑