arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2404.17877cs.CL
收藏

PromptCL:通过提示模板与对比学习改进事件表示

PromptCL: Improving Event Representation via Prompt Template and Contrastive Learning

Dalian University of Technology(大连理工大学)

Yubo Feng, Lishuang Li, Yi Xiang, Xueyang Qin

AI总结:

针对事件文本较短导致与预训练分布不一致的问题,提出PromptCL框架,利用提示模板扩展输入并结合对比学习,同时引入SPO语序与EventMLM训练模型理解事件组件关系,显著提升了事件表示性能与泛化能力。

摘要:

文本中的事件表示在各种NLP任务中扮演着重要角色。最近的研究表明,对比学习能够提升预训练语言模型(PLMs)的事件理解能力,并增强事件表示学习的性能。然而,基于对比学习和PLMs的事件表示学习的有效性受到事件文本较短长度的限制。事件文本的长度与PLMs预训练时使用的文本长度存在显著差异。因此,预训练与事件表示学习之间在文本长度分布上存在不一致性,这可能会削弱基于PLMs的事件表示学习过程。在本研究中,我们提出了PromptCL,这是一个用于事件表示学习的新颖框架,能够有效地激发PLMs全面捕捉短事件文本语义的能力。PromptCL利用借鉴自提示学习的提示模板,在对比学习期间扩展输入文本。这通过提供事件组件的结构化大纲,有助于增强事件表示学习。此外,我们提出了主-谓-宾(SPO)语序和面向事件的掩码语言建模(EventMLM),以训练PLMs理解事件组件之间的关系。我们的实验结果表明,PromptCL在事件相关任务上优于最先进的基线方法。此外,我们进行了详尽的分析,并证明使用提示能够提升事件表示的泛化能力。我们的代码将在https://github.com/YuboFeng2023/PromptCL上提供。

英文摘要:

The representation of events in text plays a significant role in various NLP tasks. Recent research demonstrates that contrastive learning has the ability to improve event comprehension capabilities of Pre-trained Language Models (PLMs) and enhance the performance of event representation learning. However, the efficacy of event representation learning based on contrastive learning and PLMs is limited by the short length of event texts. The length of event texts differs significantly from the text length used in the pre-training of PLMs. As a result, there is inconsistency in the distribution of text length between pre-training and event representation learning, which may undermine the learning process of event representation based on PLMs. In this study, we present PromptCL, a novel framework for event representation learning that effectively elicits the capabilities of PLMs to comprehensively capture the semantics of short event texts. PromptCL utilizes a Prompt template borrowed from prompt learning to expand the input text during Contrastive Learning. This helps in enhancing the event representation learning by providing a structured outline of the event components. Moreover, we propose Subject-Predicate-Object (SPO) word order and Event-oriented Masked Language Modeling (EventMLM) to train PLMs to understand the relationships between event components. Our experimental results demonstrate that PromptCL outperforms state-of-the-art baselines on event related tasks. Additionally, we conduct a thorough analysis and demonstrate that using a prompt results in improved generalization capabilities for event representations. Our code will be available at https://github.com/YuboFeng2023/PromptCL.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑