arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1909.08166cs.CL
收藏

用于文本分类的递归图形神经网络

Recursive Graphical Neural Networks for Text Classification

Peking University(北京大学);Tencent(腾讯);Beijing Institute of Big Data Research(北京大数据研究院)

Wei Li, Shuheng Li, Shuming Ma, Yancheng He, Deli Chen, Xu Sun

摘要:

自然语言复杂的语法结构很难用基于序列的模型进行显式建模。图是一种自然的结构来描述令牌之间的复杂关系。图神经网络(GNN)的最新进展为图结构数据建模提供了强有力的工具,但简单的图模型如图卷积网络(GCN)存在过平滑问题,即当堆叠多层时,所有节点将收敛到相同的值。在本文中,我们提出了一种新的递归图形神经网络模型(ReGNN)来表示以图形形式组织的文本。在我们提出的模型中,LSTM用于动态决定聚合邻居信息的哪一部分应该传输到上层,从而缓解过度平滑问题。此外,为了促进局部和全局信息之间的交换,设计了全局图级节点。我们进行实验,单和多标签文本分类任务。实验结果表明,我们的ReGNN模型在大多数数据集上都明显优于强基线,并大大消除了过平滑问题。

英文摘要:

The complicated syntax structure of natural language is hard to be explicitly modeled by sequence-based models. Graph is a natural structure to describe the complicated relation between tokens. The recent advance in Graph Neural Networks (GNN) provides a powerful tool to model graph structure data, but simple graph models such as Graph Convolutional Networks (GCN) suffer from over-smoothing problem, that is, when stacking multiple layers, all nodes will converge to the same value. In this paper, we propose a novel Recursive Graphical Neural Networks model (ReGNN) to represent text organized in the form of graph. In our proposed model, LSTM is used to dynamically decide which part of the aggregated neighbor information should be transmitted to upper layers thus alleviating the over-smoothing problem. Furthermore, to encourage the exchange between the local and global information, a global graph-level node is designed. We conduct experiments on both single and multiple label text classification tasks. Experiment results show that our ReGNN model surpasses the strong baselines significantly in most of the datasets and greatly alleviates the over-smoothing problem.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑