CausalCite:论文引用的因果性表述
CausalCite: A Causal Formulation of Paper Citations
AI总结:
提出CausalCite,通过因果推断方法TextMatch评估论文对后续论文的因果影响,以更准确衡量论文重要性,并在多项标准上验证其有效性。
摘要:
论文的引用次数通常被用作评估论文在科学界重要性的常用指标。然而,引用指标因未能准确反映论文的真实影响而受到广泛批评。因此,我们提出CausalCite,一种通过评估论文对其后续论文的因果影响来衡量论文重要性的新方法。CausalCite基于一种新颖的因果推断方法TextMatch,该方法将传统的匹配框架适应于高维文本嵌入。TextMatch使用大型语言模型(LLMs)的文本嵌入对每篇论文进行编码,通过余弦相似度提取相似样本,并根据相似度值将相似论文的加权平均值综合为一个反事实样本。我们在多种标准上展示了CausalCite的有效性,例如与科学专家在先前包含1K篇论文的数据集上报告的论文影响高度相关,对过往论文的(时间检验)奖项,以及其在AI各子领域的稳定性。我们还提供了一系列发现,可作为未来研究人员使用我们指标以更好理解论文质量的建议方式。我们的代码可在https://github.com/causalNLP/causal-cite获取。
英文摘要:
Citation count of a paper is a commonly used proxy for evaluating the significance of a paper in the scientific community. Yet citation measures are widely criticized for failing to accurately reflect the true impact of a paper. Thus, we propose CausalCite, a new way to measure the significance of a paper by assessing the causal impact of the paper on its follow-up papers. CausalCite is based on a novel causal inference method, TextMatch, which adapts the traditional matching framework to high-dimensional text embeddings. TextMatch encodes each paper using text embeddings from large language models (LLMs), extracts similar samples by cosine similarity, and synthesizes a counterfactual sample as the weighted average of similar papers according to their similarity values. We demonstrate the effectiveness of CausalCite on various criteria, such as high correlation with paper impact as reported by scientific experts on a previous dataset of 1K papers, (test-of-time) awards for past papers, and its stability across various subfields of AI. We also provide a set of findings that can serve as suggested ways for future researchers to use our metric for a better understanding of the quality of a paper. Our code is available at https://github.com/causalNLP/causal-cite.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。