arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.14780cs.AI
收藏

ACR:一个用于自动队列检索的基准

ACR: A Benchmark for Automatic Cohort Retrieval

Mendel AI(孟德尔人工智能公司);Columbia University(哥伦比亚大学);University of Pennsylvania(宾夕法尼亚大学)

Dung Ngoc Thai, Victor Ardulov, Jose Ulises Mena, Simran Tiwari, Gleb Erofeev, Ramy Eskander, Karim Tarabishy, Ravi B Parikh, Wael Salloum

AI总结:

论文提出自动队列检索(ACR)任务与配套基准,评估LLM及商业领域神经符号方法,以支持跨电子病历的纵向患者队列发现。

摘要:

识别患者队列是众多医疗保健任务的基础,包括临床试验招募和回顾性研究。医疗机构当前的队列检索方法依赖于对结构化数据的自动查询并结合人工整理,这既耗时、劳动密集,又往往产生低质量结果。大型语言模型(LLM)和信息检索(IR)的最新进展为革新这些系统提供了有前景的途径。主要挑战包括管理大量纳入标准,以及处理非结构化电子病历(EMR)的纵向特性,同时确保解决方案在真实世界应用中仍具有成本效益。本文介绍了一项新任务——自动队列检索(ACR),并评估了LLM以及商业化、特定领域神经符号方法的性能。我们提供了一个基准任务、一个查询数据集、一个EMR数据集和一个评估框架。研究结果强调了高效、高质量的ACR系统的必要性,这类系统应能够跨大型患者数据库进行纵向推理。

英文摘要:

Identifying patient cohorts is fundamental to numerous healthcare tasks, including clinical trial recruitment and retrospective studies. Current cohort retrieval methods in healthcare organizations rely on automated queries of structured data combined with manual curation, which are time-consuming, labor-intensive, and often yield low-quality results. Recent advancements in large language models (LLMs) and information retrieval (IR) offer promising avenues to revolutionize these systems. Major challenges include managing extensive eligibility criteria and handling the longitudinal nature of unstructured Electronic Medical Records (EMRs) while ensuring that the solution remains cost-effective for real-world application. This paper introduces a new task, Automatic Cohort Retrieval (ACR), and evaluates the performance of LLMs and commercial, domain-specific neuro-symbolic approaches. We provide a benchmark task, a query dataset, an EMR dataset, and an evaluation framework. Our findings underscore the necessity for efficient, high-quality ACR systems capable of longitudinal reasoning across extensive patient databases.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑