RECITE:结合文本证据修订合格标准
RECITE: Revising Eligibility Criteria Incorporating Textual Evidence
AI总结:
针对临床试验合格标准过严导致入组失败的问题,提出RECITE框架,构建相关数据集并测试15种LLM,发现Gemma 2 9B、GPT-4o表现良好,还识别出44篇论文-试验匹配项,证实临床知识可转化为试验设计建议。
摘要:
过于严格的合格标准导致临床试验广泛的入组失败,然而已发表的关于如何放宽特定标准的建议与大规模试验设计脱节。我们提出RECITE(结合文本证据修订合格标准),这是一个用于自动发现、提取和实施基于证据的合格标准修订的框架。RECITE利用来自该http URL的方案修订文档,构建了包含5735项试验的11913个精选实例的数据集,每个实例都将原始标准与文献证据及目标修订配对。我们在截断至上下文窗口和检索增强生成条件下,对15种大语言模型(LLM)配置进行基准测试,在经10位领域专家验证的LLM作为评判者协议下,对3116个实例样本进行测试(评判者与人类的二次加权Cohen κqw=0.694±0.096)。最佳开放权重模型Gemma 2 9B在0-4量表上达到与κqw对齐的序数评分3.0±0.9(归一化值0.742),最佳封闭权重模型GPT-4o达到3.0±0.8(归一化值0.745)。随后,我们使用智能体文献发现系统,在5篇文献中识别出44篇论文-试验匹配项,这些文献提供了试验设计和合格标准(EC)修改的证据。我们的发现表明,现有临床知识可被转化为可操作的试验设计建议。
英文摘要:
Overly restrictive eligibility criteria contribute to widespread enrollment failure in clinical trials, yet published recommendations for how specific criteria should be broadened remain disconnected from trial design at scale. We introduce RECITE (Revising Eligibility Criteria Incorporating Textual Evidence), a framework for automated discovery, extraction, and implementation of evidence-based eligibility criteria revisions. RECITE uses protocol amendment documents from ClinicalTrials.gov to construct a dataset of 11,913 curated instances across 5,735 trials, each pairing original criteria with documentary evidence and target revisions. We benchmark fifteen LLM configurations under truncated-to-context-window and retrieval-augmented generation conditions, on a 3,116-instance sample under an LLM-as-judge protocol validated against ten domain experts (judge vs. human quadratic-weighted Cohen's $κ_{qw} = 0.694 \pm 0.096$). The best open-weight model, Gemma 2 9B, achieves $κ_{qw}$-aligned ordinal score $3.0 \pm 0.9$ on a 0--4 scale (normalized 0.742) and the best closed-weight model, GPT-4o, reaches $3.0 \pm 0.8$ (normalized 0.745). Using an agentic literature discovery system, we then identified 44 paper--trial matches across five publications providing trial-design and EC-modification evidence. Our findings suggest that existing clinical knowledge can be operationalized into actionable trial design recommendations.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。