智能体能否像人类一样评判系统综述？基于LLM的多智能体系统评估SLR
Can Agents Judge Systematic Reviews Like Humans? Evaluating SLRs with LLM-based Multi-Agent System
AI总结：
本文提出基于LLM的多智能体系统评估系统文献综述质量，自动化协议验证、方法学评估和主题相关性检查，在五篇SLR上与专家PRISMA评分达到84%一致，展示了NLP驱动系统简化跨学科综述流程的潜力。
摘要：
系统文献综述（SLRs）是循证研究的基础，但在各学科中仍然劳动密集且容易产生不一致。我们提出一个基于LLM的SLR评估副驾驶，构建于多智能体系统（MAS）架构之上，以协助研究人员评估系统文献综述的整体质量。该系统利用学术数据库自动化协议验证、方法学评估和主题相关性检查。与传统的单智能体方法不同，我们的设计整合了一种与PRISMA指南对齐的专用智能体方法，以支持更结构化和可解释的评估。我们对来自不同领域的五篇已发表SLR进行了初步研究，将系统输出与专家标注的PRISMA分数进行比较，观察到84%的一致性。虽然早期结果令人鼓舞，但这项工作代表了迈向可扩展且准确的NLP驱动系统用于跨学科工作流程的第一步，并揭示了其进行严格的、领域无关的知识聚合以简化综述过程的能力。
英文摘要：
Systematic Literature Reviews (SLRs) are foundational to evidence-based research but remain labor-intensive and prone to inconsistency across disciplines. We present an LLM-based SLR evaluation copilot built on a Multi-Agent System (MAS) architecture to assist researchers in assessing the overall quality of the systematic literature reviews. The system automates protocol validation, methodological assessment, and topic relevance checks using a scholarly database. Unlike conventional single-agent methods, our design integrates a specialized agentic approach aligned with PRISMA guidelines to support more structured and interpretable evaluations. We conducted an initial study on five published SLRs from diverse domains, comparing system outputs to expert-annotated PRISMA scores, and observed 84% agreement. While early results are promising, this work represents a first step toward scalable and accurate NLP-driven systems for interdisciplinary workflows and reveals their capacity for rigorous, domain-agnostic knowledge aggregation to streamline the review process.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
