arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2005.00250cs.CL
收藏

低资源多任务序列标记--再论动态条件随机场

Low Resource Multi-Task Sequence Tagging -- Revisiting Dynamic Conditional Random Fields

University of Bristol(布里斯托尔大学);Technische Universität Darmstadt(达姆施塔特工业大学)

Jonas Pfeiffer, Edwin Simpson, Iryna Gurevych

摘要:

我们比较了不同的模型低资源多任务序列标记,利用不同任务的标签序列之间的依赖关系。我们的分析针对的是每个示例都有多个任务标签的数据集。当前的方法要么为每个任务使用单独的模型,要么使用标准的多任务学习来学习共享的特征表示。然而,这些方法忽略了标签序列之间的相关性,这可以在具有小训练数据集的设置中提供重要信息。为了分析哪些场景可以从不同任务中的标签之间的依赖关系建模中获益,我们重新审视了动态条件随机场(CRF),并将其与深度神经网络相结合。我们比较了单任务,多任务和动态CRF设置三个不同的数据集在句子和文档水平在英语和德语低资源的情况下。我们表明,将来自预训练词性标记器的银色标签作为辅助任务可以提高下游任务的性能。我们发现,特别是在低资源的情况下,任务预测之间的相互依赖关系的显式建模优于单任务以及标准的多任务模型。

英文摘要:

We compare different models for low resource multi-task sequence tagging that leverage dependencies between label sequences for different tasks. Our analysis is aimed at datasets where each example has labels for multiple tasks. Current approaches use either a separate model for each task or standard multi-task learning to learn shared feature representations. However, these approaches ignore correlations between label sequences, which can provide important information in settings with small training datasets. To analyze which scenarios can profit from modeling dependencies between labels in different tasks, we revisit dynamic conditional random fields (CRFs) and combine them with deep neural networks. We compare single-task, multi-task and dynamic CRF setups for three diverse datasets at both sentence and document levels in English and German low resource scenarios. We show that including silver labels from pretrained part-of-speech taggers as auxiliary tasks can improve performance on downstream tasks. We find that especially in low-resource scenarios, the explicit modeling of inter-dependencies between task predictions outperforms single-task as well as standard multi-task models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑