发表机构
Duke University; Case Western Reserve University（杜克大学; 凯斯西储大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究针对EHR算法验证的高成本问题，提出基于设计的验证框架，结合SRS审查与可选自适应分配，可降低部分验证偏差，提升罕见结局算法的验证精度。
AI 中文摘要
电子健康记录（EHR）表型算法支持临床预测、观察性研究和质量改进，但用于金标准验证的手动病历审查成本高昂。针对罕见结局的算法中，算法阴性患者的数量往往远多于算法阳性患者。我们提出一种基于设计的验证框架，该框架完全验证算法阳性患者，并对算法阴性患者进行简单随机样本（SRS）审查。假阴性（FN）和真阴性总数的Horvitz-Thompson估计量可用于计算灵敏度、特异度、阴性预测值、阳性预测值和准确率。我们推导了有限总体校正的方差估计量和置信区间（CI），并针对预先指定的CI半宽度进行样本量规划。可选的第二阶段扩展使用试点数据评估风险引导的奈曼分配是否可提高精度。在模拟中，与朴素验证样本分析相比，该基于设计的方法大幅降低了偏差，且达到接近名义水平的CI覆盖率。当假阴性集中在风险层时，自适应分配降低了均方根误差和CI宽度，但在风险信号较弱时几乎没有益处。我们使用包含3336名患者的乳腺癌复发算法说明该框架，终点特异性的算法阴性审查预算分别为125、150和175份病历。该框架结合了基于SRS的验证和精度规划，以及可选的试点引导自适应分配。
英文摘要
Electronic health record (EHR) phenotyping algorithms support clinical prediction, observational research, and quality improvement, but manual chart review for gold-standard validation is costly. For algorithms targeting rare outcomes, algorithm-negative patients often greatly outnumber algorithm-positive patients. We propose a design-based validation framework that fully verifies algorithm-positive patients and reviews a simple random sample (SRS) of algorithm-negative patients. Horvitz-Thompson estimators of false-negative (FN) and true-negative totals yield estimates of sensitivity, specificity, negative predictive value, positive predictive value, and accuracy. We derive finite-population-corrected variance estimators and confidence intervals (CIs), with sample-size planning for a prespecified CI half-width. An optional Phase 2 extension uses pilot data to assess whether risk-guided Neyman allocation may improve precision. In simulations, the design-based approach substantially reduced bias relative to naive verified-sample analyses and achieved near-nominal CI coverage. Adaptive allocation reduced root mean squared error and CI width when FNs were concentrated across risk strata, but offered little benefit when the risk signal was weak. We illustrate the framework using a breast cancer recurrence algorithm in 3,336 patients, with endpoint-specific algorithm-negative review budgets of 125, 150, and 175 charts. The framework combines SRS-based validation and precision planning with optional pilot-guided adaptive allocation.
Comments33 pages, 3 figures, submitted to Statistics in Medicine
