arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1805.07231cs.CL
收藏

基于会话级Tokenization的对话行为识别研究

A Study on Dialog Act Recognition using Character-Level Tokenization

INESC-ID Lisboa(里斯本系统与计算机工程研究所);Instituto Superior Técnico, Universidade de Lisboa(里斯本大学高等技术学院);Instituto Universitário de Lisboa (ISCTE-IUL)(里斯本大学学院(ISCTE-IUL))

Eugénio Ribeiro, Ricardo Ribeiro, David Martins de Matos

摘要:

对话行为识别是对话系统中的一个重要环节,它揭示了话语背后的意图。大多数任务的方法都使用单词级标记。相反,本文探讨了字符级标记化的使用。这是相关的,因为在子词水平上存在与词的功能相关的信息,因此,它们的意图。我们还探讨了在每个标记周围使用不同的上下文窗口,这些窗口能够捕获重要的元素,例如词缀。此外,我们评估标点符号和大写的重要性。我们在Switchboard Dialog Act语料库和DIHANA语料库上进行了实验。在这两种情况下,实验不仅表明,字符级标记化导致更好的性能比典型的词级方法,而且这两种方法都能够捕捉互补的信息。因此,通过在两个级别上组合标记化来实现最佳结果。

英文摘要:

Dialog act recognition is an important step for dialog systems since it reveals the intention behind the uttered words. Most approaches on the task use word-level tokenization. In contrast, this paper explores the use of character-level tokenization. This is relevant since there is information at the sub-word level that is related to the function of the words and, thus, their intention. We also explore the use of different context windows around each token, which are able to capture important elements, such as affixes. Furthermore, we assess the importance of punctuation and capitalization. We performed experiments on both the Switchboard Dialog Act Corpus and the DIHANA Corpus. In both cases, the experiments not only show that character-level tokenization leads to better performance than the typical word-level approaches, but also that both approaches are able to capture complementary information. Thus, the best results are achieved by combining tokenization at both levels.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑