arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2106.07306cs.LGcs.CL
收藏

将线性链条件随机场约束到正则语言

Constraining Linear-chain CRFs to Regular Languages

University of Stuttgart(斯图加特大学)

Sean Papay, Roman Klinger, Sebastian Padó

AI总结:

本文提出正则约束CRF(RegCCRF),通过将输出空间限定为正则语言来引入非局部约束,证明约束训练优于约束解码,并在语义角色标注上取得领先结果。

摘要:

结构化预测中的一个主要挑战是如何表示输出结构内部的相互依赖关系。当输出被结构化为序列时,线性链条件随机场(CRFs)是一类被广泛使用的模型,能够学习输出中的局部依赖。然而,CRF的马尔可夫假设使其无法表示具有非局部依赖的分布,并且标准CRF无法遵守数据中的非局部约束(例如输出标签上的全局元数约束)。我们提出了一种CRF的推广形式,通过将可能输出结构的空间指定为正则语言$\mathcal{L}$,可以强制执行包括非局部约束在内的一大类约束。由此得到的正则约束CRF(RegCCRF)具有与标准CRF相同的形式性质,但对所有不在$\mathcal{L}$中的标签序列赋予零概率。值得注意的是,RegCCRF可以在训练过程中纳入其约束,而相关模型仅在解码阶段强制执行约束。我们证明了约束训练永远不会比约束解码差,并通过实验表明在实践中它可以显著更好。此外,我们通过将RegCCRF纳入用于语义角色标注的深度神经模型,展示了下游任务上的实际收益,在标准数据集上超越了最先进的结果。

英文摘要:

A major challenge in structured prediction is to represent the interdependencies within output structures. When outputs are structured as sequences, linear-chain conditional random fields (CRFs) are a widely used model class which can learn \textit{local} dependencies in the output. However, the CRF's Markov assumption makes it impossible for CRFs to represent distributions with \textit{nonlocal} dependencies, and standard CRFs are unable to respect nonlocal constraints of the data (such as global arity constraints on output labels). We present a generalization of CRFs that can enforce a broad class of constraints, including nonlocal ones, by specifying the space of possible output structures as a regular language $\mathcal{L}$. The resulting regular-constrained CRF (RegCCRF) has the same formal properties as a standard CRF, but assigns zero probability to all label sequences not in $\mathcal{L}$. Notably, RegCCRFs can incorporate their constraints during training, while related models only enforce constraints during decoding. We prove that constrained training is never worse than constrained decoding, and show empirically that it can be substantially better in practice. Additionally, we demonstrate a practical benefit on downstream tasks by incorporating a RegCCRF into a deep neural model for semantic role labeling, exceeding state-of-the-art results on a standard dataset.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑