arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2307.09665cs.LGcs.AIcs.SI
收藏

基于动态图Transformer的研究社区技术专长与能力演化预测

Anticipating Technical Expertise and Capability Evolution in Research Communities using Dynamic Graph Transformers

Pacific Northwest National Laboratory(太平洋西北国家实验室)

Sameera Horawalavithana, Ellyn Ayton, Anastasiya Usenko, Robin Cosbey, Svitlana Volkova

AI总结:

本研究提出基于动态异构图表示的技术专长演化预测问题,构建结合离散与连续时间的动态图Transformer模型,在AI和核不扩散领域的多任务预测上优于静态图基线30%-80%,提升了新兴协作模式下的归纳任务性能。

摘要:

预测全球范围内的技术专长与能力演化趋势对国家和全球安全至关重要,在核不扩散(NN)这类安全关键领域以及人工智能(AI)这类快速兴起的领域中尤其如此。本研究拓展了传统统计关系学习方法(如协作网络中的链接预测),提出了利用动态异构图表示预测技术专长与能力演化的问题。我们开发了新型能力,可在两个不同研究领域中,从不同粒度(如科学家层面和机构层面)预测协作模式、作者署名行为以及技术能力演化。我们实现了一种动态图Transformer(DGT)神经架构,该架构从两方面推进了现有最先进的图神经网络模型:一是预测异质(而非同质)节点与边,二是同时结合离散时间和连续时间输入。实验表明,我们的DGT模型在预测协作、合作关系以及专长模式时,在AI领域的平均倒数排名值分别为0.26、0.73和0.53,在NN领域分别为0.48、0.93和0.22。在AI和NN领域中,DGT模型的性能比表现最优的静态图基线模型高出30%-80%。研究结果显示,对于存在新兴协作模式的领域(如AI),当测试数据中出现此前未见过的节点时,DGT模型能提升归纳任务的性能。具体而言,在AI领域中,模型能准确预测哪些资深科学家会与青年科学家开展协作,反之亦然。

英文摘要:

The ability to anticipate technical expertise and capability evolution trends globally is essential for national and global security, especially in safety-critical domains like nuclear nonproliferation (NN) and rapidly emerging fields like artificial intelligence (AI). In this work, we extend traditional statistical relational learning approaches (e.g., link prediction in collaboration networks) and formulate a problem of anticipating technical expertise and capability evolution using dynamic heterogeneous graph representations. We develop novel capabilities to forecast collaboration patterns, authorship behavior, and technical capability evolution at different granularities (e.g., scientist and institution levels) in two distinct research fields. We implement a dynamic graph transformer (DGT) neural architecture, which pushes the state-of-the-art graph neural network models by (a) forecasting heterogeneous (rather than homogeneous) nodes and edges, and (b) relying on both discrete -- and continuous -- time inputs. We demonstrate that our DGT models predict collaboration, partnership, and expertise patterns with 0.26, 0.73, and 0.53 mean reciprocal rank values for AI and 0.48, 0.93, and 0.22 for NN domains. DGT model performance exceeds the best-performing static graph baseline models by 30-80% across AI and NN domains. Our findings demonstrate that DGT models boost inductive task performance, when previously unseen nodes appear in the test data, for the domains with emerging collaboration patterns (e.g., AI). Specifically, models accurately predict which established scientists will collaborate with early career scientists and vice-versa in the AI domain.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑