医疗保健互操作性的闭环验证修复：临床大语言模型中模式合规性的多模型研究
Closed-Loop Validation-Repair for Healthcare Interoperability: A Multi-Model Study of Schema Compliance in Clinical LLMs
浏览论文内容
中文总结 AI 辅助
研究医疗大语言模型集成到电子健康记录系统面临的模式不合规问题，通过在多场景下部署三个开源模型并评估，发现模式不合规具有一致性，验证 - 修复框架可有效提高合规率，为医疗互操作性提供保障。
中文摘要 AI 辅助
医疗保健互操作性要求人工智能系统生成符合标准化模式的结构化输出，如用于诊断编码的ICD - 10、用于程序计费的CPT和用于数据交换的HL7 FHIR。虽然大语言模型展示了临床推理能力，但它们集成到电子健康记录系统面临模式不合规的关键障碍。我们通过在跨越十个医学专业的320个临床场景中进行本地部署，评估了三个开源模型Qwen2.5 7B、Llama 3.1 8B和Gemma2 9B，产生了960个模型 - 场景对，并在配对基线和验证 - 修复条件下进行评估。首先，模式不合规在三个模型家族中是一致的，基线合规率在85.9%至91.6%之间。其次，验证器检测到的96%的故障是表示级格式违规。第三，验证 - 修复框架总体合规率达到99.0%，大多数错误在一两次迭代内解决。这些结果支持闭环验证修复作为医疗保健互操作性的有效系统级保障。
英文摘要
Healthcare interoperability requires AI systems to produce structured outputs conforming to standardized schemas including ICD-10 for diagnostic coding, CPT for procedure billing, and HL7 FHIR for data exchange. While large language models demonstrate clinical reasoning capabilities, their integration into electronic health record systems faces a critical barrier: schema noncompliance. We evaluate three open-source models, Qwen2.5 7B, Llama 3.1 8B, and Gemma2 9B, via local deployment across 320 clinical scenarios spanning ten medical specialties, yielding 960 model-scenario pairs assessed under paired baseline and validation-repair conditions. First, schema noncompliance is consistent across the three model families, with baseline compliance rates ranging from 85.9 to 91.6 percent despite varying architectures and training data, suggesting shared gaps in medical training corpora rather than model-specific limitations. Second, 96 percent of validator-detected failures are representation-level format violations such as alternative medical abbreviations and code prefixes, indicating models follow clinical writing conventions but lack awareness of healthcare IT standards. Third, the validation-repair framework achieves 99.0 percent overall compliance, ranging from 98.4 to 99.4 percent across models, with most errors resolving within one or two iterations. Exact McNemar p-values below 0.001 and absolute improvements of 7.8 to 12.5 percentage points across model sizes confirm statistical significance. These results support closed-loop validation-repair as an effective system-level safeguard for healthcare interoperability, improving schema-level readiness for downstream clinical system integration.
发表机构
- University of Montana（蒙大拿大学）
机构由 AI 辅助整理，请以论文原文为准。
