arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2001.06846cs.DBcs.LG
收藏

SQLFlow:连接SQL和机器学习的桥梁

SQLFlow: A Bridge between SQL and Machine Learning

Ant Financial USA(蚂蚁金服美国);Ant Financial(蚂蚁金服);DiDi(滴滴)

Yi Wang, Yang Yang, Weiguo Zhu, Yi Wu, Xu Yan, Yongfeng Liu, Yu Wang, Liang Xie, Ziyao Gao, Wenjing Zhu, Xiang Chen, Wei Yan, Mingjie Tang, Yuan Tang

摘要:

工业AI系统大多是端到端的机器学习(ML)工作流程。典型的推荐或商业智能系统包括许多在线微服务和离线作业。我们描述了SQLFlow在SQL中有效地开发这样的工作流。SQL使开发人员能够编写简短的程序,专注于目的(什么)而忽略过程(如何)。以前的数据库系统扩展了它们的SQL方言以支持ML。SQLFlow(https:sqlflow.org/sqlflow)采用另一种策略作为各种数据库系统的桥梁,包括MySQL,Apache Hive和Alibaba MaxCompute,以及TensorFlow,XGBoost和scikit-learn等ML引擎。我们仔细地扩展了SQL语法,使扩展能够与各种SQL方言一起工作。我们通过发明一个协作解析算法来实现扩展。SQLFlow对各种各样的ML技术都是高效和有表现力的--监督和无监督学习;深度网络和树模型;除了训练和预测之外的视觉模型解释;除了ML之外的数据处理和特征提取。SQLFlow将SQL程序编译成Kubernetes原生工作流,以实现容错执行和云上部署。目前的行业用户包括蚂蚁金服、滴滴和阿里巴巴集团。

英文摘要:

Industrial AI systems are mostly end-to-end machine learning (ML) workflows. A typical recommendation or business intelligence system includes many online micro-services and offline jobs. We describe SQLFlow for developing such workflows efficiently in SQL. SQL enables developers to write short programs focusing on the purpose (what) and ignoring the procedure (how). Previous database systems extended their SQL dialect to support ML. SQLFlow (https://sqlflow.org/sqlflow ) takes another strategy to work as a bridge over various database systems, including MySQL, Apache Hive, and Alibaba MaxCompute, and ML engines like TensorFlow, XGBoost, and scikit-learn. We extended SQL syntax carefully to make the extension working with various SQL dialects. We implement the extension by inventing a collaborative parsing algorithm. SQLFlow is efficient and expressive to a wide variety of ML techniques -- supervised and unsupervised learning; deep networks and tree models; visual model explanation in addition to training and prediction; data processing and feature extraction in addition to ML. SQLFlow compiles a SQL program into a Kubernetes-native workflow for fault-tolerable execution and on-cloud deployment. Current industrial users include Ant Financial, DiDi, and Alibaba Group.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑