arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2208.04749cs.CL
收藏

作文语义学的表示学习在哪里?

Where's the Learning in Representation Learning for Compositional Semantics and the Case of Thematic Fit

Columbia University(哥伦比亚大学);University of Gothenburg(哥德堡大学);University of Washington(华盛顿大学)

Mughilan Muthupari, Samrat Halder, Asad Sayeed, Yuval Marton

摘要:

观察到对于某些NLP任务,如语义角色预测或主题拟合估计,随机嵌入的表现与预训练的嵌入一样好,我们探索了哪些设置允许这一点,并检查了大部分学习的编码:词嵌入,语义角色嵌入或"网络“。根据任务及其与培训目标的关系,我们找到了微妙的答案。我们在多任务学习中研究这些表示学习方面,其中角色预测和角色填充是监督任务,而几个主题拟合任务不在模型的直接监督范围内。我们观察到一些任务的质量分数和训练数据大小之间的非单调关系。为了更好地理解这一观察结果,我们使用这些任务的更简单的每个动词版本来分析这些结果。

英文摘要:

Observing that for certain NLP tasks, such as semantic role prediction or thematic fit estimation, random embeddings perform as well as pretrained embeddings, we explore what settings allow for this and examine where most of the learning is encoded: the word embeddings, the semantic role embeddings, or ``the network''. We find nuanced answers, depending on the task and its relation to the training objective. We examine these representation learning aspects in multi-task learning, where role prediction and role-filling are supervised tasks, while several thematic fit tasks are outside the models' direct supervision. We observe a non-monotonous relation between some tasks' quality score and the training data size. In order to better understand this observation, we analyze these results using easier, per-verb versions of these tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑