arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2010.08684cs.CLcs.AI
收藏

示例驱动的带有观察者的意图预测

Example-Driven Intent Prediction with Observers

Carnegie Mellon University(卡内基梅隆大学);Amazon Alexa AI(亚马逊Alexa AI)

Shikib Mehri, Mihail Eric

摘要:

对话系统研究的一个关键挑战是有效地和高效地适应新的领域。适应的可扩展范例需要开发在Few-Shot设置中表现良好的可推广模型。在本文中,我们专注于意图分类问题,其目的是确定用户意图给定的话语寻址到对话系统。我们提出了两种方法来提高话语分类模型的泛化能力:(1)观察者和(2)示例驱动的训练。先前的工作表明,BERT类模型倾向于将大量的注意力归因于[CLS]令牌,我们假设这会导致稀释表示。观察者是不受关注的令牌,并且是[CLS]令牌的替代品,作为话语的语义表示。示例驱动的训练通过与示例进行比较来学习对话语进行分类,从而使用底层编码器作为句子相似性模型。这些方法是相辅相成的;通过观察者改进表示可以让示例驱动模型更好地衡量句子相似性。当组合时,所提出的方法在全数据和Few-Shot(每个意图10个示例)设置中在三个意图预测数据集(\textsc{banking77}、\textsc{clinc150}、\textsc{hwu64})上获得最先进的结果。此外,我们证明了所提出的方法可以在没有任何额外训练的情况下转移到新的意图和跨数据集。

英文摘要:

A key challenge of dialog systems research is to effectively and efficiently adapt to new domains. A scalable paradigm for adaptation necessitates the development of generalizable models that perform well in few-shot settings. In this paper, we focus on the intent classification problem which aims to identify user intents given utterances addressed to the dialog system. We propose two approaches for improving the generalizability of utterance classification models: (1) observers and (2) example-driven training. Prior work has shown that BERT-like models tend to attribute a significant amount of attention to the [CLS] token, which we hypothesize results in diluted representations. Observers are tokens that are not attended to, and are an alternative to the [CLS] token as a semantic representation of utterances. Example-driven training learns to classify utterances by comparing to examples, thereby using the underlying encoder as a sentence similarity model. These methods are complementary; improving the representation through observers allows the example-driven model to better measure sentence similarities. When combined, the proposed methods attain state-of-the-art results on three intent prediction datasets (\textsc{banking77}, \textsc{clinc150}, \textsc{hwu64}) in both the full data and few-shot (10 examples per intent) settings. Furthermore, we demonstrate that the proposed approach can transfer to new intents and across datasets without any additional training.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑