arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2301.06544cs.CL
收藏

区分有意义和无意义:虚拟助理的范围外检测

Distinguish Sense from Nonsense: Out-of-Scope Detection for Virtual Assistants

IBM Watson(IBM沃森);Apple Inc.(苹果公司)

Cheng Qian, Haode Qi, Gengyu Wang, Ladislav Kunc, Saloni Potdar

摘要:

对话式AI解决方案中的超范围(OOS)检测使聊天机器人能够在无法理解最终用户查询时优雅地处理对话。当聊天机器人不具备处理与其训练的现有主题具有语义重叠的主题时,准确地将查询标记为域外特别困难。我们提出了一种简单而有效的OOS检测方法,该方法在虚拟助手的实际部署中优于标准OOS检测方法。我们讨论了云平台解决方案的各种设计和部署考虑因素,以训练虚拟助理并大规模部署它们。此外,我们提出了一个数据集的集合,复制现实世界的场景,并显示在各种设置中使用离线和在线评估指标的综合结果。

英文摘要:

Out of Scope (OOS) detection in Conversational AI solutions enables a chatbot to handle a conversation gracefully when it is unable to make sense of the end-user query. Accurately tagging a query as out-of-domain is particularly hard in scenarios when the chatbot is not equipped to handle a topic which has semantic overlap with an existing topic it is trained on. We propose a simple yet effective OOS detection method that outperforms standard OOS detection methods in a real-world deployment of virtual assistants. We discuss the various design and deployment considerations for a cloud platform solution to train virtual assistants and deploy them at scale. Additionally, we propose a collection of datasets that replicates real-world scenarios and show comprehensive results in various settings using both offline and online evaluation metrics.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑