Systematic reviews are crucial for evidence-based medicine as they comprehensively analyse published research findings on specific questions. Conducting such reviews is often resource- and time-intensive, especially in the screening phase, where abstracts of publications are assessed for inclusion in a review. This study investigates the effectiveness of using zero-shot large language models~(LLMs) for automatic screening. We evaluate the effectiveness of eight different LLMs and investigate a calibration technique that uses a predefined recall threshold to determine whether a publication should be included in a systematic review. Our comprehensive evaluation using five standard test collections shows that instruction fine-tuning plays an important role in screening, that calibration renders LLMs practical for achieving a targeted recall, and that combining both with an ensemble of zero-shot models saves significant screening time compared to state-of-the-art approaches.
Leveraging LLMs for Title and Abstract Screening for Systematic Review: A Cost-Effective Dynamic Few-Shot Learning Approach
Duke University（杜克大学）；National University of Singapore（新加坡国立大学）；Nanyang Technological University（南洋理工大学）；Duke University School of Medicine（杜克大学医学院）；Duke Clinical Research Institute（杜克临床研究所）
A Status Quo Investigation of Large Language Models towards Cost-Effective CFD Automation with OpenFOAMGPT: ChatGPT vs. Qwen vs. Deepseek
International Research Institute for Multidisciplinary Science, Beihang University, 100191 Beijing, China（多学科科学国际研究院，北航）；Cluster of Excellence SimTech, University of Stuttgart, Stuttgart, Germany（卓越仿真技术集群，斯图加特大学）；Faculty of Environment, Science and Economy, University of Exeter, Exeter EX4 4QF, United Kingdom（环境、科学与经济学院，埃克塞特大学）；Institute of Fluid Mechanics, Beihang University, 100191 Beijing, China（流体力学研究所，北航）；Institute of Thermodynamics（热力学研究所）
The "10th Juror": Open-Set Standpoint Screening for Bureaucratic Bias Detection
Sydney Smart Technology College, Northeastern University（东北大学悉尼智能科技学院）；Taiyuan University of Technology（太原理工大学）；School of Resources, Environment and Materials, Guangxi University（广西大学资源环境与材料学院）
AI Hiring with LLMs: A Context-Aware and Explainable Multi-Agent Framework for Resume Screening
Imperial College London（伦敦帝国理工学院）；The Chinese University of Hong Kong（香港中文大学）；The University of Hong Kong（香港大学）；Wedon Education Technologies（韦登教育科技）；Brest Business School（布雷斯特商学院）
Information Extraction from Electricity Invoices with General-Purpose Large Language Models
Centro de Tecnologías de la Imagen (CTIM)（图像技术中心）；Instituto Universitario de Cibernética, Empresas y Sociedad (IUCES)（大学网络研究所、企业与社会）；University of Las Palmas de Gran Canaria（大加那利岛大学）
Adaptive Test-Time Scaling for Zero-Shot Respiratory Audio Classification
Eindhoven University of Technology The Netherlands（埃因霍温理工大学荷兰）；Erasmus University Medical Center The Netherlands（埃因霍温医学院荷兰）；Kyutai France（Kyutai法国）；Eindhoven Artificial Intelligence Systems Institute The Netherlands（埃因霍温人工智能系统研究所荷兰）
Evaluating Reasoning LLMs for Suicide Screening with the Columbia-Suicide Severity Rating Scale
Ira A. Fulton Schools of Engineering Arizona State University（亚利桑那州立大学工程学院）；Heinz College of Information Systems and Public Policy Carnegie Mellon University（卡内基梅隆大学信息系统与公共政策学院）；School of Education and Counseling Psychology Santa Clara University（圣克拉拉大学教育与咨询心理学学院）
