arXiv 2508.193192025-08-26eess.IVcs.AIcs.CV

MedVQA-TREE：一种用于肌少症预测的多模态推理与检索框架

MedVQA-TREE: A Multimodal Reasoning and Retrieval Framework for Sarcopenia Prediction

Department of Electrical and Computer Engineering, Isfahan University of Technology（电气与计算机工程系，伊斯法罕技术大学）；SDU Health Informatics and Technology, The Maersk Mc-Kinney Moller Institute, University of Southern Denmark（南部丹麦大学健康信息学与技术，马士基麦金尼莫勒研究所）；Geriatric Research Unit, Department of Clinical Research, University of Southern Denmark（老年医学研究单元，临床研究系，南部丹麦大学）

Pardis Moradbeiki, Nasser Ghadiri, Sayed Jalal Zahabi, Uffe Kock Wiil, Kristoffer Kittelmann Brockhattingen, Ali Ebrahimi

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出MedVQA-TREE多模态框架，通过分层视觉理解、门控特征融合和UMLS引导的多跳检索，在肌少症超声诊断中实现高达99%的准确率，超越现有方法逾10%。

摘要：

由于影像线索细微、标注数据有限以及大多数模型缺乏临床背景，通过超声进行准确的肌少症诊断仍然具有挑战性。我们提出了MedVQA-TREE，这是一个多模态框架，整合了分层图像解释模块、门控特征级融合机制以及新颖的多跳多查询检索策略。视觉模块包括解剖分类、区域分割和基于图的空间推理，以捕获粗粒度、中粒度和细粒度结构。门控融合机制选择性地将视觉特征与文本查询相整合，同时通过UMLS引导的流程访问PubMed和肌少症专用外部知识库来检索临床知识。MedVQA-TREE在两个公开的MedVQA数据集（VQA-RAD和PathVQA）以及一个自建的肌少症超声数据集上进行了训练和评估。该模型实现了高达99%的诊断准确率，并比以往最先进的方法高出超过10%。这些结果凸显了将结构化视觉理解与引导式知识检索相结合对于有效的AI辅助肌少症诊断的益处。

英文摘要：

Accurate sarcopenia diagnosis via ultrasound remains challenging due to subtle imaging cues, limited labeled data, and the absence of clinical context in most models. We propose MedVQA-TREE, a multimodal framework that integrates a hierarchical image interpretation module, a gated feature-level fusion mechanism, and a novel multi-hop, multi-query retrieval strategy. The vision module includes anatomical classification, region segmentation, and graph-based spatial reasoning to capture coarse, mid-level, and fine-grained structures. A gated fusion mechanism selectively integrates visual features with textual queries, while clinical knowledge is retrieved through a UMLS-guided pipeline accessing PubMed and a sarcopenia-specific external knowledge base. MedVQA-TREE was trained and evaluated on two public MedVQA datasets (VQA-RAD and PathVQA) and a custom sarcopenia ultrasound dataset. The model achieved up to 99% diagnostic accuracy and outperformed previous state-of-the-art methods by over 10%. These results underscore the benefit of combining structured visual understanding with guided knowledge retrieval for effective AI-assisted diagnosis in sarcopenia.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。