利用预训练语言模型理解与建模职位市场
Understanding and Modeling Job Marketplace with Pretrained Language Models
AI总结:
本文提出职位市场基础模型PLM4Job,将预训练语言模型与职位市场异构图结合,通过异构自我图提示和邻近感知注意力对齐策略建模成员、职位文本特征及异构关系,并在LinkedIn实验中验证其有效性。
摘要:
职位市场是一个由成员(求职者)、公司和职位之间的交互构成的异构图。理解和建模职位市场能够同时惠及求职者和雇主,最终为社会的整体福祉做出贡献。然而,现有的基于图神经网络(GNN)的方法对相关文本特征和异构关系的理解较为浅层。为解决上述挑战,我们提出了PLM4Job,一个职位市场基础模型,该模型将预训练语言模型(PLM)与职位市场图紧密结合,旨在充分利用预训练知识和推理能力,同时建模成员/职位的文本特征以及各种成员-职位关系。在预训练阶段,我们提出了一种基于异构自我图的提示策略,围绕目标成员/职位节点的拓扑结构来建模和聚合成员/职位文本特征,其中引入了实体类型嵌入和图位置嵌入,以建模不同的实体及其异构关系。同时,设计了一种邻近感知的注意力对齐策略,动态调整PLM对提示中自我图节点token的注意力,使注意力能够更好地与职位市场语义对齐。在LinkedIn上进行的大量实验证明了PLM4Job的有效性。
英文摘要:
Job marketplace is a heterogeneous graph composed of interactions among members (job-seekers), companies, and jobs. Understanding and modeling job marketplace can benefit both job seekers and employers, ultimately contributing to the greater good of the society. However, existing graph neural network (GNN)-based methods have shallow understandings of the associated textual features and heterogeneous relations. To address the above challenges, we propose PLM4Job, a job marketplace foundation model that tightly couples pretrained language models (PLM) with job market graph, aiming to fully utilize the pretrained knowledge and reasoning ability to model member/job textual features as well as various member-job relations simultaneously. In the pretraining phase, we propose a heterogeneous ego-graph-based prompting strategy to model and aggregate member/job textual features based on the topological structure around the target member/job node, where entity type embeddings and graph positional embeddings are introduced accordingly to model different entities and their heterogeneous relations. Meanwhile, a proximity-aware attention alignment strategy is designed to dynamically adjust the attention of the PLM on ego-graph node tokens in the prompt, such that the attention can be better aligned with job marketplace semantics. Extensive experiments at LinkedIn demonstrate the effectiveness of PLM4Job.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。