arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1911.02557cs.LGcs.AI
收藏

大规模会话型人工智能智能体中基于反馈的自学习

Feedback-Based Self-Learning in Large-Scale Conversational AI Agents

Amazon Alexa(亚马逊Alexa)

Pragaash Ponnusamy, Alireza Roshan Ghias, Chenlei Guo, Ruhi Sarikaya

摘要:

如今,大多数大型对话式人工智能智能体(例如Alexa、Siri或Google Assistant)都是使用手动注释的数据来训练系统的不同组件。通常,这些组件中的ML模型的准确性通过手动转录和注释数据来提高。随着这些系统的范围增加以覆盖更多的场景和领域,用于提高这些组件的准确性的手动注释变得过于昂贵和耗时。在本文中,我们提出了一个系统,利用用户-系统交互反馈信号自动学习,而无需任何手动注释。这里的用户倾向于修改前一个查询,希望修复前一轮中的错误以获得正确的结果。这些重新表述,这往往是由ASR,NLU,ER或应用程序中的错误引起的有缺陷的经验。在某些情况下,用户可能没有正确地制定他们的请求(例如提供歌曲的部分标题),但在更广泛的用户和会话池中收集揭示了潜在的重复模式。我们提出的自学习系统自动检测错误,生成重新制定和部署修复的运行时系统,以纠正不同类型的错误发生在系统的不同组件。特别是,我们建议利用吸收马尔可夫链模型作为一种新的尝试,挖掘这些模式的协同过滤机制。我们表明,我们的方法是高度可扩展的,并且能够通过汇集数百万客户的匿名数据来学习减少Alexa用户错误的重新表述。建议的自学习系统实现了11.8的赢/输比,并有效地减少了超过30%的缺陷率在我们的生产A/B测试的话语级改写。据我们所知,这是第一个生产中的自学习大规模对话AI系统。

英文摘要:

Today, most large-scale conversational AI agents (e.g. Alexa, Siri, or Google Assistant) are built using manually annotated data to train the different components of the system. Typically, the accuracy of the ML models in these components are improved by manually transcribing and annotating data. As the scope of these systems increase to cover more scenarios and domains, manual annotation to improve the accuracy of these components becomes prohibitively costly and time consuming. In this paper, we propose a system that leverages user-system interaction feedback signals to automate learning without any manual annotation. Users here tend to modify a previous query in hopes of fixing an error in the previous turn to get the right results. These reformulations, which are often preceded by defective experiences caused by errors in ASR, NLU, ER or the application. In some cases, users may not properly formulate their requests (e.g. providing partial title of a song), but gleaning across a wider pool of users and sessions reveals the underlying recurrent patterns. Our proposed self-learning system automatically detects the errors, generate reformulations and deploys fixes to the runtime system to correct different types of errors occurring in different components of the system. In particular, we propose leveraging an absorbing Markov Chain model as a collaborative filtering mechanism in a novel attempt to mine these patterns. We show that our approach is highly scalable, and able to learn reformulations that reduce Alexa-user errors by pooling anonymized data across millions of customers. The proposed self-learning system achieves a win/loss ratio of 11.8 and effectively reduces the defect rate by more than 30% on utterance level reformulations in our production A/B tests. To the best of our knowledge, this is the first self-learning large-scale conversational AI system in production.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑