arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2211.05764cs.LGcs.AIcs.CYcs.SEstat.ML
收藏

DC-Check:指导可靠机器学习系统开发的以数据为中心的 AI 检查清单

DC-Check: A Data-Centric AI checklist to guide the development of reliable machine learning systems

University of Cambridge(剑桥大学);UCLA(加利福尼亚大学洛杉矶分校);Alan Turing Institute(阿兰·图灵研究所)

Nabeel Seedat, Fergus Imrie, Mihaela van der Schaar

AI总结:

本文提出 DC-Check 检查清单框架,覆盖数据、训练、测试和部署阶段,帮助从业者与研究者系统考量以数据为中心的 AI 问题,并提供可持续更新的配套资源。

摘要:

尽管机器学习(ML)已经取得了多项显著突破,但大部分关注点都集中在模型开发上。然而,要在真实世界环境中真正释放机器学习的潜力,必须在整个 ML 流水线中考虑更多方面。以数据为中心的 AI 正在成为一种统一范式,可促成此类可靠的端到端流水线。但该领域仍处于初期阶段,尚无标准化框架来指导从业者完成必要的以数据为中心的考量,或沟通以数据为中心驱动的 ML 系统设计。为弥补这一空白,我们提出 DC-Check,这是一种可操作的检查清单式框架,用于在 ML 流水线的不同阶段引出以数据为中心的考量:数据、训练、测试和部署。这种以数据为中心的开发视角旨在促进系统开发前的审慎思考与透明度。此外,我们重点指出了具体的以数据为中心的 AI 挑战和研究机会。DC-Check 面向从业者和研究人员,以指导日常开发。因此,为便于接触和使用 DC-Check 及相关资源,我们提供了 DC-Check 配套网站(https://www.vanderschaar-lab.com/dc-check/)。随着方法和工具随时间演进,该网站也将作为持续更新的资源。

英文摘要:

While there have been a number of remarkable breakthroughs in machine learning (ML), much of the focus has been placed on model development. However, to truly realize the potential of machine learning in real-world settings, additional aspects must be considered across the ML pipeline. Data-centric AI is emerging as a unifying paradigm that could enable such reliable end-to-end pipelines. However, this remains a nascent area with no standardized framework to guide practitioners to the necessary data-centric considerations or to communicate the design of data-centric driven ML systems. To address this gap, we propose DC-Check, an actionable checklist-style framework to elicit data-centric considerations at different stages of the ML pipeline: Data, Training, Testing, and Deployment. This data-centric lens on development aims to promote thoughtfulness and transparency prior to system development. Additionally, we highlight specific data-centric AI challenges and research opportunities. DC-Check is aimed at both practitioners and researchers to guide day-to-day development. As such, to easily engage with and use DC-Check and associated resources, we provide a DC-Check companion website (https://www.vanderschaar-lab.com/dc-check/). The website will also serve as an updated resource as methods and tooling evolve over time.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑