arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1705.09231cs.AIcs.PL
收藏

用于程序生成的神经属性机

Neural Attribute Machines for Program Generation

University of Wisconsin(威斯康星大学);Rice University(莱斯大学);GrammaTech, Inc.(GrammaTech公司)

Matthew Amodio, Swarat Chaudhuri, Thomas W. Reps

摘要:

递归神经网络在生成具有复杂结构的序列方面取得了显着的成功,这要归功于包括更丰富的输入嵌入和消失梯度的治愈在内的进步。尽管只接受了来自已知语法的序列的训练,他们仍然很难学习语法的规则和约束。神经属性机(NAM)配备有表示底层语法的逻辑机,其用于通过(i)增加输入序列和(ii)优化定制损失函数来向神经机教导约束。与传统的RNN不同,NAM暴露于语法以及来自语法语言的样本。在生成过程中,NAM对底层语法约束的违反比仅在语法语言样本上训练的RNN少得多。

英文摘要:

Recurrent neural networks have achieved remarkable success at generating sequences with complex structures, thanks to advances that include richer embeddings of input and cures for vanishing gradients. Trained only on sequences from a known grammar, though, they can still struggle to learn rules and constraints of the grammar. Neural Attribute Machines (NAMs) are equipped with a logical machine that represents the underlying grammar, which is used to teach the constraints to the neural machine by (i) augmenting the input sequence, and (ii) optimizing a custom loss function. Unlike traditional RNNs, NAMs are exposed to the grammar, as well as samples from the language of the grammar. During generation, NAMs make significantly fewer violations of the constraints of the underlying grammar than RNNs trained only on samples from the language of the grammar.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑