ClaimFlow: 追踪自然语言处理中科学主张的演变
ClaimFlow: Tracing the Evolution of Scientific Claims in NLP
AI总结:
提出ClaimFlow框架,通过标注1617篇论文中的5689个主张和4871个跨论文关系,定义主张关系分类任务,并分析NLP领域主张的演变模式。
摘要:
科学论文提出$\textit{主张}$,后续工作支持、扩展或有时反驳这些主张。然而,现有的引文和主张分析方法仅捕捉到这一对话的片段。在这项工作中,我们在单个科学主张的层面上使这些交互变得明确。我们引入$\texttt{ClaimFlow}$,一种以主张为中心的NLP文献视图,基于$1{,}617$篇ACL文集论文$(1979-2025)$构建,这些论文被手动标注了$5{,}689$个主张和$4{,}871$个跨论文主张关系,指示引用论文是$\texttt{支持}$、$\texttt{扩展}$、$\texttt{限定}$、$\texttt{反驳}$还是将引用的主张作为$\texttt{背景}$引用。基于$\texttt{ClaimFlow}$,我们定义了一个新任务——$\textit{主张关系分类}$——要求模型从文本和引文上下文中推断对引用的主张的科学立场。评估神经网络模型和大语言模型在该任务上的表现,我们报告了$0.81$宏F1的基线性能,表明该任务是可处理的,同时仍有改进空间。然后,我们将该框架扩展到约$13$k篇NLP论文,以研究跨越数十年NLP研究的主张演变。我们显示$63.5\%$的主张从未被重复使用;只有$11.1\%$曾受到挑战。广泛传播的主张更常通过限定和扩展$\textit{重塑}$,而非被支持或反驳。总体而言,$\texttt{ClaimFlow}$提供了一个审视NLP中思想如何转变和成熟的视角。
英文摘要:
Scientific papers advance $\textit{claims}$ that later work supports, extends, or sometimes refutes. Yet existing methods for citation and claim analysis capture only fragments of this dialogue. In this work, we make these interactions explicit at the level of individual scientific claims. We introduce $\texttt{ClaimFlow}$, a claim-centric view of the NLP literature, built from $1{,}617$ ACL Anthology papers $(1979 - 2025)$ that are manually annotated with $5{,}689$ claims and $4{,}871$ cross-paper claim relations, indicating whether a citing paper $\texttt{supports}$, $\texttt{extends}$, $\texttt{qualifies}$, $\texttt{refutes}$, or references a cited claim as $\texttt{background}$. Building on $\texttt{ClaimFlow}$, we define a new task -- $\textit{Claim Relation Classification}$ -- which requires models to infer the scientific stance toward a cited claim from the text and citation context. Evaluating neural models and large language models on this task, we report baseline performance of $0.81$ macro-F1, suggesting that the task is tractable while leaving room for improvement. We then scale this framework to $\sim$$13k$ NLP papers to study claim evolution across decades of NLP research. We show that $63.5\%$ claims are never reused; only $11.1\%$ are ever challenged. Widely propagated claims are more often $\textit{reshaped}$ through qualification and extension than supported or refuted. Overall, $\texttt{ClaimFlow}$ offers a lens for examining how ideas shift and mature within NLP.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。