大型语言模型是否减少研究新颖性?来自信息系统期刊的证据
Research Novelty in Information Systems Journals After ChatGPT: Differences Across Institutional Language Contexts
AI总结:
本文通过分析2020-2025年间44个信息系统期刊的13847篇文章,发现非英语主导国家的作者在ChatGPT发布后研究新颖性显著下降,贡献了关于LLM对学术多样性影响的见解。
摘要:
大型语言模型如ChatGPT增加了学术产出,但其是否带来真正的学术进步尚未验证。本文通过测量2020-2025年间44个信息系统期刊发表的13847篇文章的语义新颖性,发现非英语主导国家的作者在ChatGPT发布后相对新颖性下降了0.18个标准差,相当于新颖性分布下降7个百分点。该发现通过替代新颖性规格、治疗突破日期和子样本检验均稳健,且在治疗突破前通过了安慰剂测试。作者通过构念层次理论解释,认为LLM作为接近工具,使研究者从抽象探索思维转向具体惯例执行。本文为关于LLM驱动的生产力提升是否以牺牲学术多样性为代价的争论提供了贡献。
英文摘要:
Large language models are increasingly used in scholarly work, yet it remains unclear whether their productivity gains are accompanied by changes in research novelty. We examine how relative abstract-level semantic novelty in Information Systems journals changed after ChatGPT became widely available and whether this change differed across institutional language contexts. We analyze 13,847 articles published from 2020 to 2025 in 44 A* and A Information Systems journals. Using SPECTER2 representations of titles and abstracts, we measure each article's semantic distance from its nearest recent predecessors and estimate a comparative pre/post model. Articles whose first authors were affiliated with institutions in non-English-dominant countries show a 0.176 standard deviation larger post-2022 decline in relative semantic novelty than articles from English-dominant affiliations, equivalent to about 7 percentile points. The pattern is similar across several alternative specifications, although the balanced-author estimate is less precise. We interpret this finding through a tension in generative AI-supported knowledge work. GenAI can widen access to prior knowledge and support new combinations, but it can also make established frames easier to reproduce. Because individual LLM use is not observed, the result identifies a heterogeneous post-2022 shift rather than an effect of LLM adoption. The study extends research on LLMs and scholarly productivity by shifting attention from publication counts to the semantic positioning of published articles and by showing that post-2022 change differs across institutional contexts.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。