验证数据科学
Veridical Data Science
摘要:
基于统计学、机器学习和科学探究的原理,我们提出了用于验证数据科学的可预测性、可计算性和稳定性(PCS)框架。我们的框架由工作流程和文档组成,旨在在整个数据科学生命周期中提供负责任、可靠、可重复和透明的结果。PCS工作流程使用可预测性作为现实检查,并考虑数据收集/存储和算法设计中计算的重要性。它通过数据科学生命周期的总体稳定性原则增强了可预测性和可计算性。稳定性扩展了统计不确定性考虑因素,以评估人类判断调用如何通过数据和模型/算法扰动影响数据结果。此外,我们开发的推理程序,建立在PCS,即PCS扰动区间和PCS假设检验,调查的稳定性数据结果相对于问题的制定,数据清洗,建模决策和解释。我们说明PCS推理,通过神经科学和基因组学项目,我们自己和他人,并将其与现有的方法在高维,稀疏线性模型模拟。在广泛的错误指定的模拟模型,PCS推理表现出良好的性能,在ROC曲线。最后,我们提出了基于R Markdown或Markyter Notebook的PCS文档,其中包含公开可用的可复制代码和叙述,以支持整个分析过程中的人类选择。在Zenodo上提供的基因组学案例研究中演示了PCS工作流程和文档。
英文摘要:
Building and expanding on principles of statistics, machine learning, and scientific inquiry, we propose the predictability, computability, and stability (PCS) framework for veridical data science. Our framework, comprised of both a workflow and documentation, aims to provide responsible, reliable, reproducible, and transparent results across the entire data science life cycle. The PCS workflow uses predictability as a reality check and considers the importance of computation in data collection/storage and algorithm design. It augments predictability and computability with an overarching stability principle for the data science life cycle. Stability expands on statistical uncertainty considerations to assess how human judgment calls impact data results through data and model/algorithm perturbations. Moreover, we develop inference procedures that build on PCS, namely PCS perturbation intervals and PCS hypothesis testing, to investigate the stability of data results relative to problem formulation, data cleaning, modeling decisions, and interpretations. We illustrate PCS inference through neuroscience and genomics projects of our own and others and compare it to existing methods in high dimensional, sparse linear model simulations. Over a wide range of misspecified simulation models, PCS inference demonstrates favorable performance in terms of ROC curves. Finally, we propose PCS documentation based on R Markdown or Jupyter Notebook, with publicly available, reproducible codes and narratives to back up human choices made throughout an analysis. The PCS workflow and documentation are demonstrated in a genomics case study available on Zenodo.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。