arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2312.13614cs.LGcs.CL
收藏

神经有限状态转换器的结构感知路径推断

Structure-Aware Path Inference for Neural Finite State Transducers

Johns Hopkins University(约翰斯·霍普金斯大学);Google(谷歌)

Weiting Tan, Chu-cheng Lin, Jason Eisner

AI总结:

本文研究神经有限状态转换器中潜在对齐路径的推断问题,提出三种自回归模型进行摊销推断,其中结构感知模型虽新颖但在多数任务上不如简单方法。

摘要:

神经有限状态转换器(NFSTs)构成了一类富有表现力的神经符号序列转换模型。NFST将每个字符串对建模为有限状态转换器中一条潜在路径的生成结果。由于它们是深度生成模型,NFST的训练和推断都需要推断网络来近似这些潜在变量上的后验分布。在本文中,我们聚焦于由此产生的挑战:补全解释给定输入和输出字符串对的潜在对齐路径(例如,在训练期间)。我们训练了三个自回归近似模型用于路径的摊销推断,这些模型随后可用作重要性采样的提议分布。所有三个模型都执行前瞻。我们最复杂(且新颖)的模型利用FST结构来考虑未来路径的图;不幸的是,我们发现它输给了更简单的方法——除了在我们设计用来迷惑更简单方法的人工任务上。

英文摘要:

Neural finite-state transducers (NFSTs) form an expressive family of neurosymbolic sequence transduction models. An NFST models each string pair as having been generated by a latent path in a finite-state transducer. As they are deep generative models, both training and inference of NFSTs require inference networks that approximate posterior distributions over such latent variables. In this paper, we focus on the resulting challenge of imputing the latent alignment path that explains a given pair of input and output strings (e.g., during training). We train three autoregressive approximate models for amortized inference of the path, which can then be used as proposal distributions for importance sampling. All three models perform lookahead. Our most sophisticated (and novel) model leverages the FST structure to consider the graph of future paths; unfortunately, we find that it loses out to the simpler approaches -- except on an artificial task that we concocted to confuse the simpler approaches.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑