预先训练的语言模型作为汽车投诉分析的知识库
Pre-trained language models as knowledge bases for Automotive Complaint Analysis
摘要:
最近,研究表明,像BERT这样的大型预训练语言模型(Devlin等人,2018)能够存储在其预训练语料库中捕获的常识事实知识(Petroni等人,2019年)。在我们的工作中,我们进一步评估了这种能力,从行业的应用程序创建一组探头,专门设计用于揭示技术质量问题,作为描述的事件,在汽车行业的非结构化客户反馈。在使用填充掩码任务探测预训练模型的开箱即用版本后,我们通过对缺陷调查办公室(ODI)投诉数据集的持续预训练,动态地为其提供更多知识。在我们的实验中,与Petroni等人(2019)所做的事实知识本身的查询相比,模型表现出对特定领域主题的查询性能。对于大多数被评估的架构,正确的令牌被预测为具有高于60\%的$Precision@1$($P@1$),而对于$P@5$和$P@10$,甚至分别达到远高于80\%和高达90\%的值。这些结果表明,使用语言模型作为知识库的客户反馈的结构化分析的潜力。
英文摘要:
Recently it has been shown that large pre-trained language models like BERT (Devlin et al., 2018) are able to store commonsense factual knowledge captured in its pre-training corpus (Petroni et al., 2019). In our work we further evaluate this ability with respect to an application from industry creating a set of probes specifically designed to reveal technical quality issues captured as described incidents out of unstructured customer feedback in the automotive industry. After probing the out-of-the-box versions of the pre-trained models with fill-in-the-mask tasks we dynamically provide it with more knowledge via continual pre-training on the Office of Defects Investigation (ODI) Complaints data set. In our experiments the models exhibit performance regarding queries on domain-specific topics compared to when queried on factual knowledge itself, as Petroni et al. (2019) have done. For most of the evaluated architectures the correct token is predicted with a $Precision@1$ ($P@1$) of above 60\%, while for $P@5$ and $P@10$ even values of well above 80\% and up to 90\% respectively are reached. These results show the potential of using language models as a knowledge base for structured analysis of customer feedback.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。