arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2010.11490cs.CL
收藏

神经网络中使用word2vec表示对对话行为识别的影响

On the Effects of Using word2vec Representations in Neural Networks for Dialogue Act Recognition

LORIA-UMR7503(洛里亚联合实验室(UMR7503));University of West Bohemia(西波西米亚大学);NTIS - New Technologies for the Information Society(信息社会新技术研究所)

Christophe Cerisara, Pavel Kral, Ladislav Lenc

摘要:

对话行为识别是大量自然语言处理流水线的重要组成部分。在这一领域已经进行了许多研究工作,但相对较少的研究深入神经网络和词嵌入。这是令人惊讶的,因为这两种技术在大多数其他语言相关领域都被证明是非常好的。在这项工作中,我们提出了一种新的深度神经网络,该网络探索循环模型来捕获句子中的单词序列,并进一步研究预训练的词嵌入的影响。我们验证了三种语言:英语,法语和捷克语的模型。所提出的方法的性能在这些语言中是一致的,并且可以与英语中的最先进的结果相媲美。更重要的是,我们证实了深度神经网络确实优于最大熵分类器,这是预期的。然而,更令人惊讶的是,我们还发现,无论训练语料库的大小如何,标准的word 2 vec em-beddings似乎都没有为这项任务和所提出的模型带来有价值的信息。因此,我们进一步分析所产生的嵌入,并得出结论,一个可能的解释可能是有关的词汇语义信息的类型之间的不匹配所捕获的word 2 vec嵌入,和一种词之间的关系,这是最有用的对话行为识别任务。

英文摘要:

Dialogue act recognition is an important component of a large number of natural language processing pipelines. Many research works have been carried out in this area, but relatively few investigate deep neural networks and word embeddings. This is surprising, given that both of these techniques have proven exceptionally good in most other language-related domains. We propose in this work a new deep neural network that explores recurrent models to capture word sequences within sentences, and further study the impact of pretrained word embeddings. We validate this model on three languages: English, French and Czech. The performance of the proposed approach is consistent across these languages and it is comparable to the state-of-the-art results in English. More importantly, we confirm that deep neural networks indeed outperform a Maximum Entropy classifier, which was expected. However , and this is more surprising, we also found that standard word2vec em-beddings do not seem to bring valuable information for this task and the proposed model, whatever the size of the training corpus is. We thus further analyse the resulting embeddings and conclude that a possible explanation may be related to the mismatch between the type of lexical-semantic information captured by the word2vec embeddings, and the kind of relations between words that is the most useful for the dialogue act recognition task.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑