LUMEN：用于自动化系统评价和荟萃分析的成本透明多智能体流水线
LUMEN: Cost-Transparent Multi-Agent Pipeline for Automated Systematic Review and Meta-Analysis
AI总结：
提出开源多智能体流水线LUMEN，自动化系统评价与荟萃分析的六个阶段，通过11个专用LLM代理和模型路由实现，首次报告端到端成本（19-29美元）并揭示阶段依赖的架构反转。
摘要：
系统评价和荟萃分析（SR/MA）仍然是证据合成的金标准，但完成一项通常需要67周和大量专家努力。最近的大型语言模型（LLM）系统在单个SR阶段表现出色——筛选（otto-SR：96.7%灵敏度）、提取（Gartlehner等人：91.0%准确率）和搜索（TrialMind：0.83召回率）——但尚无研究报告运行端到端流水线的实际成本、成本在各阶段的分布情况，或架构选择如何影响成本-质量权衡。我们提出LUMEN，一个开源多智能体流水线，使用11个专用LLM代理和精心设计的模型路由，自动化六个SR/MA阶段。我们在七个数据集上评估LUMEN：五个自行进行的领域综述（精神病学、心理学、外科、疫苗学、心脏病学）和两个SYNERGY筛选基准。在13个与真实结果可比的结果中，LUMEN与已发表荟萃分析的方向一致性达到100%，对于同质研究设计，效应量差异在1%以内。主要贡献是对此类流水线的首次经验性成本和操作特征描述：完整综述成本为19至29美元（中位数22.65美元），其中标题-摘要筛选和数据提取共同占据主导支出。三臂提取消融实验揭示了阶段依赖的架构反转：多智能体设计在筛选中效果不佳，但对提取至关重要，与单模型替代方案相比，产生5.7倍更可合并的分析，同时消除了临床危险的方向错误。两个数据集的筛选基准表明，模型排名具有领域依赖性，且在不同综述主题间不可迁移。所有代码和成本日志均已公开。
英文摘要：
Systematic reviews and meta-analyses (SR/MA) remain the gold standard for evidence synthesis, yet completing one typically requires 67 weeks and substantial expert effort. Recent large language model (LLM) systems have demonstrated strong performance on individual SR phases - screening (otto-SR: 96.7% sensitivity), extraction (Gartlehner et al.: 91.0% accuracy), and search (TrialMind: 0.83 recall) - but no study has reported what it actually costs to run an end-to-end pipeline, how cost distributes across phases, or how architectural choices affect the cost-quality trade-off. We present LUMEN, an open-source multi-agent pipeline that automates six SR/MA phases using 11 specialized LLM agents with deliberate model routing. We evaluate LUMEN on seven datasets: five self-conducted domain reviews (psychiatry, psychology, surgery, vaccinology, cardiology) and two SYNERGY screening benchmarks. Across 13 ground-truth-comparable outcomes, LUMEN achieves 100% directional agreement with published meta-analyses, with effect sizes within 1% for homogeneous study designs. The primary contribution is the first empirical cost and operational characterization of such a pipeline: a complete review costs 19 to 29 USD (median 22.65 USD), with title-abstract screening and data extraction together dominating expenditure. A three-arm extraction ablation reveals a phase-dependent architecture reversal: multi-agent design hurts screening but is essential for extraction, producing 5.7x more poolable analyses than single-model alternatives while eliminating clinically dangerous direction errors. A two-dataset screening benchmark demonstrates that model ranking is domain-dependent and not transferable across review topics. All code and cost logs are publicly available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
