arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.03934cs.CL
收藏

从文字到价值:使用LLM进行新生文章影响预测

From Words to Worth: Newborn Article Impact Prediction with LLM

Nankai University(南开大学);NKIARI(南开大学深圳研究院(注:NKIARI为南开大学深圳研究院英文缩写,若为独立机构则保留原名直译));Nanjing University(南京大学)

Penghai Zhao, Qinghua Xing, Kairan Dou, Jinyu Tian, Ying Tai, Jian Yang, Ming-Ming Cheng, Xiang Li

摘要:

随着学术领域的扩大,有效识别有影响力的新发表文章的挑战变得越来越重要。本文介绍了一种很有前途的方法,利用LLM的能力来预测新生文章的未来影响,仅基于标题和摘要。超越了严重依赖外部信息的传统方法,所提出的方法采用LLM从大量的标题摘要对中识别出高度影响力的论文的共享语义特征。这些语义特征被进一步用于预测所提出的指标TNCSI_SP,该指标在值、字段和时间上具有良好的归一化特性。为了促进LLM的参数高效微调,我们还精心策划了一个包含12,000多个条目的数据集,每个条目都标有标题,摘要及其相应的TNCSI_SP值。定量结果表明,MAE为0.216,NDCG@20为0.901,与几种有前途的方法相比,所提出的方法在预测新生儿文章的影响方面达到了最先进的性能。最后,我们提出了一个真实世界的应用程序的例子来预测新生期刊文章的影响,以证明其值得注意的实用价值。总的来说,我们的研究结果挑战了现有的范式,并提出了一个转向更注重内容的学术影响力预测,为文章影响力预测提供了新的见解。

英文摘要:

As the academic landscape expands, the challenge of efficiently identifying impactful newly published articles grows increasingly vital. This paper introduces a promising approach, leveraging the capabilities of LLMs to predict the future impact of newborn articles solely based on titles and abstracts. Moving beyond traditional methods heavily reliant on external information, the proposed method employs LLM to discern the shared semantic features of highly impactful papers from a large collection of title-abstract pairs. These semantic features are further utilized to predict the proposed indicator, TNCSI_SP, which incorporates favorable normalization properties across value, field, and time. To facilitate parameter-efficient fine-tuning of the LLM, we have also meticulously curated a dataset containing over 12,000 entries, each annotated with titles, abstracts, and their corresponding TNCSI_SP values. The quantitative results, with an MAE of 0.216 and an NDCG@20 of 0.901, demonstrate that the proposed approach achieves state-of-the-art performance in predicting the impact of newborn articles when compared to several promising methods. Finally, we present a real-world application example for predicting the impact of newborn journal articles to demonstrate its noteworthy practical value. Overall, our findings challenge existing paradigms and propose a shift towards a more content-focused prediction of academic impact, offering new insights for article impact prediction.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑