arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2508.07308cs.CLcs.AIcs.IRcs.LG

HealthBranches:通过决策路径合成基于临床的问答数据集

HealthBranches: Synthesizing Clinically-Grounded Question Answering Datasets via Decision Pathways

University of Calabria(卡拉布里亚大学);University of Catanzaro(卡坦扎罗大学);University of Eastern Piedmont(东皮埃蒙特大学);University of Pisa(比萨大学);University of Cambridge(剑桥大学)

Cristian Cosentino, Annamaria Defilippo, Marco Dossena, Christopher Irwin, Sara Joubbi, Pietro Liò

AI总结:

HealthBranches是一个新型医疗问答基准数据集,通过半自动化流水线将医学决策路径转化为包含4,063个案例的现实问答,支持多步推理评估,旨在推动高风险领域更可靠的LLMs发展。

摘要:

HealthBranches是一个用于医疗问答(Q&A)的新型基准数据集,专门设计用于评估大型语言模型(LLMs)中的复杂推理能力。该数据集通过半自动化流水线生成,将医学来源中的显式决策路径转化为带有相关问答的现实患者病例。涵盖17个医疗主题下的4,063个案例研究,每个数据点均基于临床验证的推理链。HealthBranches支持开放式和多项选择题格式,并独特地包含了每个问答的完整推理路径。其结构化设计能够稳健评估LLMs的多步推理能力,包括在结构化检索增强生成(RAG)上下文中的表现。HealthBranches为在高风险领域开发更值得信赖、可解释和临床可靠的LLMs奠定了基础,同时也作为教育目的的宝贵资源。

英文摘要:

HealthBranches is a novel benchmark dataset for medical Question-Answering (Q&A), specifically designed to evaluate complex reasoning in Large Language Models (LLMs). This dataset is generated through a semi-automated pipeline that transforms explicit decision pathways from medical source into realistic patient cases with associated questions and answers. Covering 4,063 case studies across 17 healthcare topics, each data point is based on clinically validated reasoning chains. HealthBranches supports both open-ended and multiple-choice question formats and uniquely includes the full reasoning path for each Q&A. Its structured design enables robust evaluation of LLMs' multi-step inference capabilities, including their performance in structured Retrieval-Augmented Generation (RAG) contexts. HealthBranches establishes a foundation for the development of more trustworthy, interpretable, and clinically reliable LLMs in high-stakes domains while also serving as a valuable resource for educational purposes.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑