arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.04381cs.IR
收藏

利用预训练语言模型理解与建模职位市场

Understanding and Modeling Job Marketplace with Pretrained Language Models

Yaochen Zhu, Liang Wu, Binchi Zhang, Song Wang, Qi Guo, Liangjie Hong, Luke Simon, Jundong Li

AI总结:

本文提出职位市场基础模型PLM4Job,将预训练语言模型与职位市场异构图结合,通过异构自我图提示和邻近感知注意力对齐策略建模成员、职位文本特征及异构关系,并在LinkedIn实验中验证其有效性。

摘要:

职位市场是一个由成员(求职者)、公司和职位之间的交互构成的异构图。理解和建模职位市场能够同时惠及求职者和雇主,最终为社会的整体福祉做出贡献。然而,现有的基于图神经网络(GNN)的方法对相关文本特征和异构关系的理解较为浅层。为解决上述挑战,我们提出了PLM4Job,一个职位市场基础模型,该模型将预训练语言模型(PLM)与职位市场图紧密结合,旨在充分利用预训练知识和推理能力,同时建模成员/职位的文本特征以及各种成员-职位关系。在预训练阶段,我们提出了一种基于异构自我图的提示策略,围绕目标成员/职位节点的拓扑结构来建模和聚合成员/职位文本特征,其中引入了实体类型嵌入和图位置嵌入,以建模不同的实体及其异构关系。同时,设计了一种邻近感知的注意力对齐策略,动态调整PLM对提示中自我图节点token的注意力,使注意力能够更好地与职位市场语义对齐。在LinkedIn上进行的大量实验证明了PLM4Job的有效性。

英文摘要:

Job marketplace is a heterogeneous graph composed of interactions among members (job-seekers), companies, and jobs. Understanding and modeling job marketplace can benefit both job seekers and employers, ultimately contributing to the greater good of the society. However, existing graph neural network (GNN)-based methods have shallow understandings of the associated textual features and heterogeneous relations. To address the above challenges, we propose PLM4Job, a job marketplace foundation model that tightly couples pretrained language models (PLM) with job market graph, aiming to fully utilize the pretrained knowledge and reasoning ability to model member/job textual features as well as various member-job relations simultaneously. In the pretraining phase, we propose a heterogeneous ego-graph-based prompting strategy to model and aggregate member/job textual features based on the topological structure around the target member/job node, where entity type embeddings and graph positional embeddings are introduced accordingly to model different entities and their heterogeneous relations. Meanwhile, a proximity-aware attention alignment strategy is designed to dynamically adjust the attention of the PLM on ego-graph node tokens in the prompt, such that the attention can be better aligned with job marketplace semantics. Extensive experiments at LinkedIn demonstrate the effectiveness of PLM4Job.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑