在消费级硬件上实现GraphRAG：对本地LLMs在医疗EHR模式检索中的基准测试
GraphRAG on Consumer Hardware: Benchmarking Local LLMs for Healthcare EHR Schema Retrieval
AI总结：
本文研究了在消费级硬件上使用本地LLMs进行医疗EHR模式检索的GraphRAG方法，评估了四种不同模型在索引效率、知识图构建、查询延迟、回答质量和幻觉方面的表现，发现模型参数大小和检索模式对结果有显著影响。
摘要：
基于图的检索增强生成（GraphRAG）扩展了检索增强生成，以支持对复杂语料库的结构化推理，但其在资源受限、隐私敏感的部署中的可靠性仍不清楚。在医疗领域，电子健康记录（EHR）数据复杂且严格监管，依赖云基于大语言模型（LLMs）会带来成本、延迟和合规性的挑战。本文系统评估了GraphRAG在EHR模式检索中的应用，使用本地部署的开源LLMs。我们实现了Microsoft GraphRAG管道在真实的EHR模式文档上，并基准测试了四种模型，包括Llama 3.1（8B）、Mistral（7B）、Qwen 2.5（7B）和Phi-4-mini（3.8B），这些模型通过Ollama在单个消费级GPU（8 GB VRAM）上部署。我们评估了索引效率、知识图构建、查询延迟、回答质量和幻觉在全局和局部检索模式下的表现。我们的结果揭示了显著差异：Llama 3.1生成最丰富的知识图（1,172个实体），Qwen 2.5达到最佳回答质量（3.3/5），Phi-4-mini因结构化输出错误无法完成流程，而Mistral表现出退化重复行为。我们进一步表明，GraphRAG具有实际容量阈值，其中模型参数低于约7B的模型无法可靠地生成有效的结构化输出并无法完成流程。此外，索引和回答质量在不同模型之间是脱耦的，局部检索在延迟和事实基础方面均优于全局总结，且幻觉减少。这些发现表明，GraphRAG可以在消费级硬件上实现，同时强调了模型选择和检索设计在受监管环境中的重要性。
英文摘要：
Graph-based Retrieval Augmented Generation (GraphRAG) extends retrieval-augmented generation to support structured reasoning over complex corpora, but its reliability under resource-constrained, privacy-sensitive deployments remains unclear. In healthcare, where Electronic Health Record (EHR) data is complex and strictly regulated, reliance on cloud-based large language models (LLMs) introduces challenges in cost, latency, and compliance. In this work, we present a systematic evaluation of GraphRAG for EHR schema retrieval using locally deployed open-source LLMs. We implement the Microsoft GraphRAG pipeline on real-world EHR schema documentation and benchmark four models, including Llama 3.1 (8B), Mistral (7B), Qwen 2.5 (7B), and Phi-4-mini (3.8B), each deployed via Ollama on a single consumer GPU (8 GB VRAM). We evaluate indexing efficiency, knowledge graph construction, query latency, answer quality, and hallucination under both global and local retrieval modes. Our results reveal substantial differences: Llama 3.1 produces the richest knowledge graph (1,172 entities), Qwen 2.5 achieves the best answer quality (3.3/5), Phi-4-mini fails to complete the pipeline due to structured-output errors, and Mistral exhibits degenerate repetition behavior. We further show that GraphRAG exhibits a practical capacity threshold, where models below approximately 7B parameters fail to reliably produce valid structured outputs and cannot complete the pipeline. In addition, indexing and answer quality are decoupled across models, and local retrieval consistently outperforms global summarization in both latency and factual grounding, with reduced hallucination. These findings demonstrate that GraphRAG is feasible on consumer hardware while highlighting the importance of model selection and retrieval design for robust deployment in regulated settings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
