增强数据科学:走向数据科学的产业化和民主化
Augmented Data Science: Towards Industrialization and Democratization of Data Science
摘要:
将原始数据转化为见解和知识需要数据科学家付出大量努力。尽管机器学习(ML)和人工智能(AI)取得了惊人的进步,但数据科学家仍然将大部分精力花在理解和准备ML/AI的原始数据上。这项工作通常是手动的和临时的,需要一定程度的领域知识。当数据多样性(包括形式和上下文)增加时,工作的复杂性会急剧增加。在本文中,我们介绍了我们的解决方案,增强数据科学(ADS),以解决这个“人类瓶颈”,从不同的数据集创造价值。ADS是一种数据驱动的方法,它依赖于统计和ML,以与领域无关的方式从任何数据集中提取见解,以促进数据科学过程。ADS的主要特点是用自动化取代基本的数据探索和处理步骤,以及用自动生成的见解增强数据科学家的判断。我们展示了端到端解决方案的构建模块,并提供了一个案例研究来验证其功能。
英文摘要:
Conversion of raw data into insights and knowledge requires substantial amounts of effort from data scientists. Despite breathtaking advances in Machine Learning (ML) and Artificial Intelligence (AI), data scientists still spend the majority of their effort in understanding and then preparing the raw data for ML/AI. The effort is often manual and ad hoc, and requires some level of domain knowledge. The complexity of the effort increases dramatically when data diversity, both in form and context, increases. In this paper, we introduce our solution, Augmented Data Science (ADS), towards addressing this "human bottleneck" in creating value from diverse datasets. ADS is a data-driven approach and relies on statistics and ML to extract insights from any data set in a domain-agnostic way to facilitate the data science process. Key features of ADS are the replacement of rudimentary data exploration and processing steps with automation and the augmentation of data scientist judgment with automatically-generated insights. We present building blocks of our end-to-end solution and provide a case study to exemplify its capabilities.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。