在虚拟助手中利用用户参与信号进行实体标记
Leveraging User Engagement Signals For Entity Labeling in a Virtual Assistant
摘要:
Siri、Cortana和Alexa等个人助理AI系统已被广泛用作通过自然语言命令完成任务的手段。然而,这些系统中的组件通常依赖于有监督的机器学习算法,该算法需要大量手工注释的训练数据,这是昂贵且耗时的。在本文中,我们描述了一个框架,利用用户参与信号(表现出积极或消极的内容响应的用户行为)自动创建粒度实体标签的训练数据增强。采用多任务学习和使用外部知识库进行验证等策略来合并参与注释数据并提高模型在序列标记任务上的准确性。我们的研究结果表明,从用户参与信号自动标记的数据中学习,在生产深度学习系统中实现了显着的准确性增益,当测量序列标记任务以及系统端到端产生的面向用户的结果时。我们相信这是第一次使用用户参与信号来帮助大规模地为序列标记任务生成训练数据,并且可以应用于实际设置中,以在几乎没有人类注释数据可用时加快新功能的部署。
英文摘要:
Personal assistant AI systems such as Siri, Cortana, and Alexa have become widely used as a means to accomplish tasks through natural language commands. However, components in these systems generally rely on supervised machine learning algorithms that require large amounts of hand-annotated training data, which is expensive and time consuming to collect. The ability to incorporate unsupervised, weakly supervised, or distantly supervised data holds significant promise in overcoming this bottleneck. In this paper, we describe a framework that leverages user engagement signals (user behaviors that demonstrate a positive or negative response to content) to automatically create granular entity labels for training data augmentation. Strategies such as multi-task learning and validation using an external knowledge base are employed to incorporate the engagement annotated data and to boost the model's accuracy on a sequence labeling task. Our results show that learning from data automatically labeled by user engagement signals achieves significant accuracy gains in a production deep learning system, when measured on both the sequence labeling task as well as on user facing results produced by the system end-to-end. We believe this is the first use of user engagement signals to help generate training data for a sequence labeling task on a large scale, and can be applied in practical settings to speed up new feature deployment when little human annotated data is available.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。