具有层次递归神经网络的分块结构涌现
The Emergence of Chunking Structures with Hierarchical RNN
AI总结：
本文提出一种无监督分块方法，利用层次递归神经网络建模词与分块的层次结构，通过预训练和微调提升分块性能，揭示分块结构在训练过程中的短暂性。
摘要：
在自然语言处理（NLP）中，预测语言结构，如解析和分块，大多依赖于人工标注的句法结构。本文介绍了一种无监督的分块方法，这是一种句法任务，涉及非层次化地将单词分组。我们提出了一种层次递归神经网络（HRNN），旨在建模词到分块和分块到句子的组成。我们的方法包括一个两阶段的训练过程：先用无监督解析器进行预训练，然后在下游NLP任务上进行微调。在多个数据集上的实验显示，预训练和微调阶段的无监督分块性能有显著提升。有趣的是，我们发现分块结构的出现在神经模型的下游任务训练过程中是短暂的。本研究推动了无监督句法结构发现的发展，并为语言理论进一步研究开辟了新途径。
英文摘要：
In Natural Language Processing (NLP), predicting linguistic structures, such as parsing and chunking, has mostly relied on manual annotations of syntactic structures. This paper introduces an unsupervised approach to chunking, a syntactic task that involves grouping words in a non-hierarchical manner. We present a Hierarchical Recurrent Neural Network (HRNN) designed to model word-to-chunk and chunk-to-sentence compositions. Our approach involves a two-stage training process: pretraining with an unsupervised parser and finetuning on downstream NLP tasks. Experiments on multiple datasets reveal a notable improvement of unsupervised chunking performance in both pretraining and finetuning stages. Interestingly, we observe that the emergence of the chunking structure is transient during the neural model's downstream-task training. This study contributes to the advancement of unsupervised syntactic structure discovery and opens avenues for further research in linguistic theory.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
