arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2210.02898cs.CLcs.AI
收藏

学习自然语言定义的非纠缠表示

Learning Disentangled Representations for Natural Language Definitions

University of Manchester(曼彻斯特大学);Idiap Research Institute(伊迪亚普研究院)

Danilo S. Carvalho, Giangiacomo Mercatali, Yingji Zhang, Andre Freitas

摘要:

在自然语言处理中,解开神经模型的编码是提高可解释性、语义控制和下游任务性能的基本方面。目前,大多数解纠缠方法都是无监督的,或者依赖于具有已知生成因子的合成数据集。我们认为,在文本数据中的经常性的句法和语义重叠可以用来提供模型的结构偏见和生成因素。我们利用语义结构中存在的一个代表性的和语义密集的类别的句子类型,定义句,训练变分自编码器学习解开表示。我们的实验结果表明,所提出的模型优于无监督基线的定性和定量基准的解纠缠,它也提高了在定义建模的下游任务的结果。

英文摘要:

Disentangling the encodings of neural models is a fundamental aspect for improving interpretability, semantic control and downstream task performance in Natural Language Processing. Currently, most disentanglement methods are unsupervised or rely on synthetic datasets with known generative factors. We argue that recurrent syntactic and semantic regularities in textual data can be used to provide the models with both structural biases and generative factors. We leverage the semantic structures present in a representative and semantically dense category of sentence types, definitional sentences, for training a Variational Autoencoder to learn disentangled representations. Our experimental results show that the proposed model outperforms unsupervised baselines on several qualitative and quantitative benchmarks for disentanglement, and it also improves the results in the downstream task of definition modeling.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑