拨开冗余信息迷雾:大语言模型在系统文献综述高效筛选中的潜力
Cutting Through the Clutter: The Potential of LLMs for Efficient Filtration in Systematic Literature Reviews
AI总结:
针对系统文献综述筛选效率低、耗人力的问题,研究团队开发开源工具LLMSurver,借助大语言模型实现交互式文献筛选,可将8300余篇文献的筛选时间从数周缩至数分钟,召回率超98.8%,优于人工误差水平。
摘要:
系统文献综述(SLRs)是学术研究中的关键环节,但由于论文发表量庞大、基于关键词的筛选效率低下,该过程十分耗费人力。为简化这一流程,我们评估了大语言模型(LLMs)在提升文献库筛选效率与准确率的同时减少人工工作量的效果。我们开发的开源工具LLMSurver(大语言模型文献综述工具)提供可视化界面,支持借助LLMs开展文献筛选、评估结果并交互式优化查询条件。我们在近期一项综述构建工作中,用该方法筛选了超过8300篇文章,将结果与人工筛选结果对比以评估其实际表现。结果显示,当前的LLM模型可将筛选时间从数周缩短至数分钟;其共识机制可保证召回率超过98.8%,优于典型的人工误差阈值,同时提升了筛选准确率。本研究推动了文献综述方法的发展,凸显了负责任的人机协作在学术研究中的潜力。
英文摘要:
Systematic literature reviews (SLRs) are essential but labor-intensive due to high publication volumes and inefficient keyword-based filtering. To streamline this process, we evaluate Large Language Models (LLMs) for enhancing efficiency and accuracy in corpus filtration while minimizing manual effort. Our open-source tool LLMSurver presents a visual interface to utilize LLMs for literature filtration, evaluate the results, and refine queries in an interactive way. We assess the real-world performance of our approach in filtering over 8.3k articles during a recent survey construction, comparing results with human efforts. The findings show that recent LLM models can reduce filtering time from weeks to minutes. A consensus scheme ensures recall rates >98.8%, surpassing typical human error thresholds and improving selection accuracy. This work advances literature review methodologies and highlights the potential of responsible human-AI collaboration in academic research.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。