arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2401.01596cs.AIcs.CL
收藏

MedSumm:一种面向印英混合临床查询摘要生成的多模态方法

MedSumm: A Multimodal Approach to Summarizing Code-Mixed Hindi-English Clinical Queries

Indian Institute of Technology Patna(印度理工学院巴特那分校);Stanford University(斯坦福大学);Amazon GenAI(亚马逊生成式人工智能部门);Indira Gandhi Institute of Medical Sciences(英迪拉·甘地医学科学研究所)

Akash Ghosh, Arkadeep Acharya, Prince Jha, Aniket Gaudgaul, Rajdeep Majumdar, Sriparna Saha, Aman Chadha, Raghav Jain, Setu Sinha, Shivani Agarwal

AI总结:

本文提出多模态医疗混合查询摘要任务,构建了印英混合的MMCQS数据集,并利用LLMs和VLMs提出MedSumm框架,通过整合视觉信息生成更准确的医学摘要,改善医疗决策。

摘要:

在医疗保健领域,总结患者提出的医疗问题对于改善医患互动和医疗决策至关重要。尽管医疗数据的复杂性和数量都在增长,但目前该领域的研究主要集中在基于文本的方法上,忽视了视觉线索的整合。此外,先前在医疗问题摘要领域的工作仅限于英语。本工作引入了在低资源环境下处理混合输入的多模态医疗问题摘要任务。为填补这一空白,我们提出了多模态医疗混合问题摘要数据集(MMCQS),该数据集结合了印英混合的医疗查询与视觉辅助信息。这种整合丰富了患者医疗状况的表征,提供了更全面的视角。我们还提出了一个名为MedSumm的框架,利用大语言模型(LLMs)和视觉语言模型(VLMs)来完成此任务。通过使用我们的MMCQS数据集,我们展示了整合图像视觉信息在改进医学详细摘要生成方面的价值。这种多模态策略不仅改善了医疗决策,还促进了对患者查询的更深入理解,为未来在个性化和响应式医疗护理方面的探索铺平了道路。我们的数据集、代码和预训练模型将公开发布。

英文摘要:

In the healthcare domain, summarizing medical questions posed by patients is critical for improving doctor-patient interactions and medical decision-making. Although medical data has grown in complexity and quantity, the current body of research in this domain has primarily concentrated on text-based methods, overlooking the integration of visual cues. Also prior works in the area of medical question summarisation have been limited to the English language. This work introduces the task of multimodal medical question summarization for codemixed input in a low-resource setting. To address this gap, we introduce the Multimodal Medical Codemixed Question Summarization MMCQS dataset, which combines Hindi-English codemixed medical queries with visual aids. This integration enriches the representation of a patient's medical condition, providing a more comprehensive perspective. We also propose a framework named MedSumm that leverages the power of LLMs and VLMs for this task. By utilizing our MMCQS dataset, we demonstrate the value of integrating visual information from images to improve the creation of medically detailed summaries. This multimodal strategy not only improves healthcare decision-making but also promotes a deeper comprehension of patient queries, paving the way for future exploration in personalized and responsive medical care. Our dataset, code, and pre-trained models will be made publicly available.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑