arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2009.07968cs.CL
收藏

基于精确ThingTalk表示的Few-Shot对话语义分析器

A Few-Shot Semantic Parser for Wizard-of-Oz Dialogues with the Precise ThingTalk Representation

Stanford University(斯坦福大学)

Giovanni Campagna, Sina J. Semnani, Ryan Kearns, Lucas Jun Koba Sato, Silei Xu, Monica S. Lam

摘要:

以前的尝试,以建立有效的语义解析器的巫师奥兹(WOZ)的对话遭受困难,在获得高质量的,手动标注的训练集。仅基于对话合成的方法是不够的,因为从基于状态机的模型生成的对话是真实对话的差的近似。此外,以前提出的对话状态表示是模糊的,缺乏建立一个有效的代理所需的精度。本文提出了一种新的对话表示和一个样本有效的方法,可以预测精确的对话状态在WOZ会话。我们扩展了ThingTalk表示,以捕获代理正确响应所需的所有信息。我们的训练策略是样本高效的:我们结合了(1)稀疏采样整个对话空间的少量数据和(2)覆盖由简洁的基于状态的对话模型生成的对话子集空间的合成数据。扩展的ThingTalk语言的完整性证明了一个完全可操作的代理,这也是用于训练数据合成。我们证明了我们的方法在MultiWOZ 3.0上的有效性,MultiWOZ 3.0是ThingTalk中MultiWOZ 2.1数据集的重新注释。ThingTalk可以代表98%的测试回合,而模拟器可以模拟85%的验证集。我们训练上下文语义分析器使用我们的策略,并获得79%的回合逐回合精确匹配的准确率重新标注的测试集。

英文摘要:

Previous attempts to build effective semantic parsers for Wizard-of-Oz (WOZ) conversations suffer from the difficulty in acquiring a high-quality, manually annotated training set. Approaches based only on dialogue synthesis are insufficient, as dialogues generated from state-machine based models are poor approximations of real-life conversations. Furthermore, previously proposed dialogue state representations are ambiguous and lack the precision necessary for building an effective agent. This paper proposes a new dialogue representation and a sample-efficient methodology that can predict precise dialogue states in WOZ conversations. We extended the ThingTalk representation to capture all information an agent needs to respond properly. Our training strategy is sample-efficient: we combine (1) fewshot data sparsely sampling the full dialogue space and (2) synthesized data covering a subset space of dialogues generated by a succinct state-based dialogue model. The completeness of the extended ThingTalk language is demonstrated with a fully operational agent, which is also used in training data synthesis. We demonstrate the effectiveness of our methodology on MultiWOZ 3.0, a reannotation of the MultiWOZ 2.1 dataset in ThingTalk. ThingTalk can represent 98% of the test turns, while the simulator can emulate 85% of the validation set. We train a contextual semantic parser using our strategy, and obtain 79% turn-by-turn exact match accuracy on the reannotated test set.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑