arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2501.12857cs.LG
收藏

HierPromptLM:一种基于纯PLM的异构文本富网络表示学习框架

HierPromptLM: A Pure PLM-based Framework for Representation Learning on Heterogeneous Text-rich Networks

Nanyang Technological University(南洋理工大学);HEC Paris(巴黎高等商学院)

Qiuyu Zhu, Liang Zhang, Qianxiong Xu, Cheng Long

AI总结:

针对异构文本富网络表示学习,提出纯PLM框架HierPromptLM,通过层次化提示模块和两个定制预训练任务统一建模文本与结构信息,在节点分类和链接预测上分别提升6.08%和10.84%。

摘要:

异构文本富网络(HTRNs)由多种类型的节点和边组成,每个节点关联文本信息,对其的表示学习对于各种现实应用至关重要。鉴于预训练语言模型(PLMs)在处理文本数据方面的成功,近期工作致力于将PLMs集成到HTRN表示学习中。这些方法通常分别处理文本和结构信息,同时使用PLMs和异构图表神经网络(HGNNs)。然而,这种分离未能捕捉HTRNs中这两类信息之间的关键交互。此外,它需要额外的对齐步骤,而由于PLMs和HGNNs生成的不同嵌入空间之间存在根本差异,这一步骤颇具挑战。为解决此问题,我们提出了HierPromptLM,一种新颖的纯PLM框架,无需分别处理即可无缝建模文本数据和图结构。首先,我们开发了一个层次化提示模块,利用提示学习在统一文本空间内的节点和边层级上集成文本数据和异构图结构。在此基础上,我们进一步引入了两个创新的针对HTRN的预训练任务,通过强调HTRNs中文本与结构信息固有的异质性和交互,对PLMs进行微调以用于表示学习。在两个真实世界HTRN数据集上的大量实验表明,HierPromptLM优于最先进的方法,在节点分类和链接预测上分别实现了高达6.08%和10.84%的显著提升。

英文摘要:

Representation learning on heterogeneous text-rich networks (HTRNs), which consist of multiple types of nodes and edges with each node associated with textual information, is essential for various real-world applications. Given the success of pretrained language models (PLMs) in processing text data, recent efforts have focused on integrating PLMs into HTRN representation learning. These methods typically handle textual and structural information separately, using both PLMs and heterogeneous graph neural networks (HGNNs). However, this separation fails to capture the critical interactions between these two types of information within HTRNs. Additionally, it necessitates an extra alignment step, which is challenging due to the fundamental differences between distinct embedding spaces generated by PLMs and HGNNs. To deal with it, we propose HierPromptLM, a novel pure PLM-based framework that seamlessly models both text data and graph structures without the need for separate processing. Firstly, we develop a Hierarchical Prompt module that employs prompt learning to integrate text data and heterogeneous graph structures at both the node and edge levels, within a unified textual space. Building upon this foundation, we further introduce two innovative HTRN-tailored pretraining tasks to fine-tune PLMs for representation learning by emphasizing the inherent heterogeneity and interactions between textual and structural information within HTRNs. Extensive experiments on two real-world HTRN datasets demonstrate HierPromptLM outperforms state-of-the-art methods, achieving significant improvements of up to 6.08% for node classification and 10.84% for link prediction.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑