HAIBU-ReMUD：连接通用与特定领域的推理型多模态超声数据集及模型
HAIBU-ReMUD: Reasoning Multimodal Ultrasound Dataset and Model Bridging to General Specific Domains
AI总结：
本文提出图文推理监督微调数据生成流程，构建含45,000余条QA/VQA数据的医学超声ReMUD数据集，并训练ReMUD-7B，性能优于通用MLLMs。
摘要：
多模态大语言模型（MLLMs）在通用领域展现出巨大潜力，但由于缺乏图像-文本数据或视频-文本数据等特定领域数据，其在某些特定领域表现不佳。在一些特定领域，存在大量分散的图文数据，但缺少标准化整理。在医学超声领域，有超声诊断书籍、超声临床指南、超声诊断报告等资料。然而，这些超声资料通常以PDF、图像等形式保存，无法直接用于MLLMs训练。本文提出一种新颖的图文推理监督微调数据生成流程，用于从特定领域材料中创建领域四元组（图像、问题、思考轨迹和答案）。研究建立了医学超声领域数据集ReMUD，包含超过45,000条推理型与非推理型监督微调问答（QA）和视觉问答（VQA）数据。在Qwen2.5-VL-7B-Instruct上微调得到的ReMUD-7B模型，在医学超声领域优于通用领域MLLMs。为促进研究，ReMUD数据集、数据生成代码库和ReMUD-7B参数将在https://github.com/ShiDaizi/ReMUD发布，以解决特定领域MLLMs的数据短缺问题。
英文摘要：
Multimodal large language models (MLLMs) have shown great potential in general domains but perform poorly in some specific domains due to a lack of domain-specific data, such as image-text data or vedio-text data. In some specific domains, there is abundant graphic and textual data scattered around, but lacks standardized arrangement. In the field of medical ultrasound, there are ultrasonic diagnostic books, ultrasonic clinical guidelines, ultrasonic diagnostic reports, and so on. However, these ultrasonic materials are often saved in the forms of PDF, images, etc., and cannot be directly used for the training of MLLMs. This paper proposes a novel image-text reasoning supervised fine-tuning data generation pipeline to create specific domain quadruplets (image, question, thinking trace, and answer) from domain-specific materials. A medical ultrasound domain dataset ReMUD is established, containing over 45,000 reasoning and non-reasoning supervised fine-tuning Question Answering (QA) and Visual Question Answering (VQA) data. The ReMUD-7B model, fine-tuned on Qwen2.5-VL-7B-Instruct, outperforms general-domain MLLMs in medical ultrasound field. To facilitate research, the ReMUD dataset, data generation codebase, and ReMUD-7B parameters will be released at https://github.com/ShiDaizi/ReMUD, addressing the data shortage issue in specific domain MLLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
