arXiv 2502.111932025-02-16cs.CL

大型语言模型在学术写作与同行评审中的渗透

Large Language Models Penetration in Scholarly Writing and Peer Review

The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；University of Electronic Science and Technology of China（电子科技大学）；Wuhan University of Technology（武汉理工大学）；University of Copenhagen（哥本哈根大学）；Shenzhen Research Institute of Big Data（深圳大数据研究院）

Li Zhou, Ruijie Zhang, Xunlian Dai, Daniel Hershcovich, Haizhou Li

备注：: Transparency in NLP, LLM-generated text evaluation and detection, LLM Penetration, Scholarly Credibility and Accountability

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对LLMs广泛使用引发的学术可信度担忧，该研究从多视角多维度评估其在学术工作流中的渗透情况，提出含ScholarLens数据集与LLMetrica评估工具的框架，实验验证工具有效性并揭示LLMs影响力增长，强调需规范其使用以维护学术可信度。

摘要：

尽管大型语言模型（LLMs）的广泛使用带来了便利，但也引发了对学术研究和学术流程可信度的担忧。为更好地理解这些动态，我们从多个视角和维度评估了LLMs在学术工作流中的渗透情况，提供了其影响力日益增长的有力证据。我们提出了一个包含两个组件的框架：ScholarLens（一个精选数据集，包含学术写作和同行评审中人类与LLM生成的内容，用于多视角评估）和LLMetrica（一个工具，通过基于规则的指标和基于模型的检测器进行多维度评估，以衡量LLM的渗透程度）。我们的实验证明了LLMetrica的有效性，揭示了LLMs在学术流程中日益重要的作用。这些发现强调了LLM使用中透明度、问责制和伦理实践的必要性，以维护学术可信度。

英文摘要：

While the widespread use of Large Language Models (LLMs) brings convenience, it also raises concerns about the credibility of academic research and scholarly processes. To better understand these dynamics, we evaluate the penetration of LLMs across academic workflows from multiple perspectives and dimensions, providing compelling evidence of their growing influence. We propose a framework with two components: \texttt{ScholarLens}, a curated dataset of human- and LLM-generated content across scholarly writing and peer review for multi-perspective evaluation, and \texttt{LLMetrica}, a tool for assessing LLM penetration using rule-based metrics and model-based detectors for multi-dimensional evaluation. Our experiments demonstrate the effectiveness of \texttt{LLMetrica}, revealing the increasing role of LLMs in scholarly processes. These findings emphasize the need for transparency, accountability, and ethical practices in LLM usage to maintain academic credibility.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。