arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2109.01135cs.CLcs.LG
收藏

基于潜在神经文法的序列到序列学习

Sequence-to-Sequence Learning with Latent Neural Grammars

MIT CSAIL(麻省理工学院计算机科学与人工智能实验室)

Yoon Kim

摘要:

使用神经网络的序列到序列学习已经成为序列预测任务的事实标准。这种方法通常使用一个强大的神经网络对下一个单词的局部分布进行建模,该神经网络可以根据任意上下文进行调节。虽然灵活且性能良好,但这些模型通常需要大型数据集进行训练,并且可能在旨在测试组合泛化的基准测试中失败。这项工作探讨了一种替代的,层次化的方法,序列到序列的学习与准同步语法,在目标树中的每个节点是由源树中的一个节点转换。源树和目标树都被视为潜在的,并在训练过程中诱导。我们开发了一个神经参数化的语法,使参数共享的组合空间的推导规则,而不需要手动功能工程。我们将这种潜在的神经语法应用于各个领域-诊断语言导航任务,旨在测试成分概括(SCAN),风格转移和小规模机器翻译-并发现与标准基线相比,它的表现相当不错。

英文摘要:

Sequence-to-sequence learning with neural networks has become the de facto standard for sequence prediction tasks. This approach typically models the local distribution over the next word with a powerful neural network that can condition on arbitrary context. While flexible and performant, these models often require large datasets for training and can fail spectacularly on benchmarks designed to test for compositional generalization. This work explores an alternative, hierarchical approach to sequence-to-sequence learning with quasi-synchronous grammars, where each node in the target tree is transduced by a node in the source tree. Both the source and target trees are treated as latent and induced during training. We develop a neural parameterization of the grammar which enables parameter sharing over the combinatorial space of derivation rules without the need for manual feature engineering. We apply this latent neural grammar to various domains -- a diagnostic language navigation task designed to test for compositional generalization (SCAN), style transfer, and small-scale machine translation -- and find that it performs respectably compared to standard baselines.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑