大型语言模型（LLM）作为文献综述工具的兴起：一项LLM自动化系统综述
The emergence of Large Language Models (LLM) as a tool in literature reviews: an LLM automated systematic review
AI总结：
本研究通过LLM辅助系统综述分析LLM在科学综述各阶段的自动化应用，发现GPT类模型占主导且数据提取表现突出，实际采用LLM的综述仍有限。
摘要：
目的：本研究旨在总结大型语言模型（LLM）在创建科学综述过程中的使用情况。我们考察综述中可实现自动化的各个阶段，并评估该领域当前最先进的研究项目。材料与方法：人工评审人员于2024年6月在PubMed、Scopus、Dimensions和Google Scholar数据库中进行检索。筛选和提取过程在Covidence中借助使用OpenAI gpt-4o模型的LLM附加组件完成。ChatGPT用于清理提取的数据并生成本文稿图表的代码；ChatGPT和Scite.ai用于起草文稿的所有组成部分，但方法和讨论部分除外。结果：共检索到3,788篇文章，172项研究被认为符合最终综述纳入条件。ChatGPT和基于GPT的LLM成为综述自动化中最占主导地位的架构（n=126，73.2%）。研究发现了大量综述自动化项目，但只有数量有限的论文（n=26，15.1%）是在创建过程中使用LLM的实际综述。大多数引用集中于综述特定阶段的自动化，例如文献检索（n=60，34.9%）和数据提取（n=54，31.4%）。在比较基于GPT和基于BERT的模型的汇总性能时，前者在数据提取方面表现更好，平均精确率为83.0%（SD=10.4），召回率为86.0%（SD=9.8），而在标题和摘要筛选阶段的准确性略低（平均准确率=77.3%，SD=13.0）。讨论/结论：我们这项LLM辅助的系统综述揭示了大量与使用LLM进行综述自动化相关的研究项目。结果看起来很有前景，我们预计LLM将在不久的将来改变科学综述的开展方式。
英文摘要：
Objective: This study aims to summarize the usage of Large Language Models (LLMs) in the process of creating a scientific review. We look at the range of stages in a review that can be automated and assess the current state-of-the-art research projects in the field. Materials and Methods: The search was conducted in June 2024 in PubMed, Scopus, Dimensions, and Google Scholar databases by human reviewers. Screening and extraction process took place in Covidence with the help of LLM add-on which uses OpenAI gpt-4o model. ChatGPT was used to clean extracted data and generate code for figures in this manuscript, ChatGPT and Scite.ai were used in drafting all components of the manuscript, except the methods and discussion sections. Results: 3,788 articles were retrieved, and 172 studies were deemed eligible for the final review. ChatGPT and GPT-based LLM emerged as the most dominant architecture for review automation (n=126, 73.2%). A significant number of review automation projects were found, but only a limited number of papers (n=26, 15.1%) were actual reviews that used LLM during their creation. Most citations focused on automation of a particular stage of review, such as Searching for publications (n=60, 34.9%), and Data extraction (n=54, 31.4%). When comparing pooled performance of GPT-based and BERT-based models, the former were better in data extraction with mean precision 83.0% (SD=10.4), and recall 86.0% (SD=9.8), while being slightly less accurate in title and abstract screening stage (Maccuracy=77.3%, SD=13.0). Discussion/Conclusion: Our LLM-assisted systematic review revealed a significant number of research projects related to review automation using LLMs. The results looked promising, and we anticipate that LLMs will change in the near future the way the scientific reviews are conducted.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
