arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1506.03134stat.MLcs.CGcs.LGcs.NE
收藏

指针网络

Pointer Networks

Google Brain(谷歌大脑);UC Berkeley(加州大学伯克利分校)

Oriol Vinyals, Meire Fortunato, Navdeep Jaitly

摘要:

我们引入了一种新的神经架构来学习输出序列的条件概率,其中输出序列的元素是与输入序列中的位置相对应的离散令牌。这些问题不能通过现有的方法(如序列到序列和神经图灵机)来解决,因为输出的每一步中目标类的数量取决于输入的长度,而输入的长度是可变的。问题,如排序可变大小的序列,和各种组合优化问题属于这一类。我们的模型使用最近提出的神经注意力机制解决了可变大小输出字典的问题。它与以前的注意力尝试的不同之处在于,它不是在每个解码器步骤使用注意力将编码器的隐藏单元混合到上下文向量,而是使用注意力作为指针来选择输入序列的成员作为输出。我们称这种架构为指针网(Ptr-Net)。我们展示了Ptr-Nets可以用来学习三个具有挑战性的几何问题的近似解决方案-找到平面凸包,计算Delaunay三角剖分和平面旅行推销员问题-仅使用训练示例。Ptr-Nets不仅改善了具有输入注意力的序列到序列,而且还允许我们推广到可变大小的输出字典。我们证明了学习模型的泛化能力超出了它们训练的最大长度。我们希望我们在这些任务上的结果将鼓励对离散问题的神经学习进行更广泛的探索。

英文摘要:

We introduce a new neural architecture to learn the conditional probability of an output sequence with elements that are discrete tokens corresponding to positions in an input sequence. Such problems cannot be trivially addressed by existent approaches such as sequence-to-sequence and Neural Turing Machines, because the number of target classes in each step of the output depends on the length of the input, which is variable. Problems such as sorting variable sized sequences, and various combinatorial optimization problems belong to this class. Our model solves the problem of variable size output dictionaries using a recently proposed mechanism of neural attention. It differs from the previous attention attempts in that, instead of using attention to blend hidden units of an encoder to a context vector at each decoder step, it uses attention as a pointer to select a member of the input sequence as the output. We call this architecture a Pointer Net (Ptr-Net). We show Ptr-Nets can be used to learn approximate solutions to three challenging geometric problems -- finding planar convex hulls, computing Delaunay triangulations, and the planar Travelling Salesman Problem -- using training examples alone. Ptr-Nets not only improve over sequence-to-sequence with input attention, but also allow us to generalize to variable size output dictionaries. We show that the learnt models generalize beyond the maximum lengths they were trained on. We hope our results on these tasks will encourage a broader exploration of neural learning for discrete problems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑