arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2012.02469cs.LGcs.DB
收藏

RPT:关系预训练的Transformer几乎是您实现数据准备民主化所需的全部

RPT: Relational Pre-trained Transformer Is Almost All You Need towards Democratizing Data Preparation

QCRI, HBKU, Qatar(卡塔尔计算研究所,哈马德·本·哈利法大学);Renmin University, China(中国人民大学);Tsinghua University, China(清华大学);CSAIL, MIT, USA(麻省理工学院计算机科学与人工智能实验室)

Nan Tang, Ju Fan, Fangyi Li, Jianhong Tu, Xiaoyong Du, Guoliang Li, Sam Madden, Mourad Ouzzani

摘要:

人工智能能否帮助自动化人类容易但计算机难以完成的数据准备任务,这些任务会给数据科学家,从业者和人群工作者带来负担?我们通过介绍RPT来回答这个问题,RPT是一个用于元组到X模型的去噪自动编码器(X可以是元组,令牌,标签,JSON等)。RPT通过破坏输入元组然后学习模型以重建原始元组来预先训练元组到元组模型。它采用了一种基于transformer的神经翻译架构,由一个双向编码器(类似于BERT)和一个从左到右的自回归解码器(类似于GPT)组成,从而使BERT和GPT都得到了推广。预先训练的RPT已经可以支持几个常见的数据准备任务,如数据清理,自动完成和模式匹配。更好的是,RPT可以在广泛的数据准备任务上进行微调,例如值规范化,数据转换,数据注释等。为了补充RPT,我们还讨论了几种有吸引力的技术,例如用于实体解析的协作训练和少样本学习,以及用于信息抽取的少样本学习和NLP问答。此外,我们确定了一系列的研究机会,以推进数据准备领域。

英文摘要:

Can AI help automate human-easy but computer-hard data preparation tasks that burden data scientists, practitioners, and crowd workers? We answer this question by presenting RPT, a denoising auto-encoder for tuple-to-X models (X could be tuple, token, label, JSON, and so on). RPT is pre-trained for a tuple-to-tuple model by corrupting the input tuple and then learning a model to reconstruct the original tuple. It adopts a Transformer-based neural translation architecture that consists of a bidirectional encoder (similar to BERT) and a left-to-right autoregressive decoder (similar to GPT), leading to a generalization of both BERT and GPT. The pre-trained RPT can already support several common data preparation tasks such as data cleaning, auto-completion and schema matching. Better still, RPT can be fine-tuned on a wide range of data preparation tasks, such as value normalization, data transformation, data annotation, etc. To complement RPT, we also discuss several appealing techniques such as collaborative training and few-shot learning for entity resolution, and few-shot learning and NLP question-answering for information extraction. In addition, we identify a series of research opportunities to advance the field of data preparation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑