Medical image segmentation is more clinically valuable when it supports diagnosis rather than merely producing lesion masks. However, diagnostically relevant lesion cues are often subtle and localized, while existing models may be distracted by background tissues, acoustic artifacts, and irrelevant visual correlations. To address this problem, we propose Rad-VLSM, a two-stage cross-modal framework for semantics-assisted lesion focusing, robust segmentation, and visually grounded diagnosis. In the first stage, a BLIP-2-based vision-language alignment module identifies lesion-related candidate regions under semantic guidance and converts them into box prompts. In the second stage, these prompts are fed into a SAM-based multitask network, where a multi-candidate region aggregation strategy improves prompt stability and guides lesion segmentation. The predicted masks are then used as spatial priors for diagnosis, and a visual-radiomics fusion head integrates lesion-aware visual features with selected radiomics descriptors. By using semantic information for localization rather than direct prediction, Rad-VLSM reduces text-to-diagnosis dependence and grounds diagnosis in lesion-level evidence. Experiments on a private clinical breast ultrasound dataset and public benchmarks show that Rad-VLSM achieves strong segmentation and diagnostic performance with favorable generalization.
VLSM-Ensemble: Ensembling CLIP-based Vision-Language Models for Enhanced Medical Image Segmentation
Insight Research Ireland Centre for Data Analytics, DCU, Dublin, Ireland(爱尔兰洞察研究爱尔兰数据分析中心,都柏林大学,都柏林);Research Ireland Centre for Research Training in Machine Learning, DCU, Dublin, Ireland(爱尔兰研究爱尔兰机器学习研究培训中心,都柏林大学,都柏林);School of Computing, Dublin City University, Dublin, Ireland(计算学院,都柏林城市大学,都柏林)
DiffVP: Differential Visual Semantic Prompting for LLM-Based CT Report Generation
Department of Electronic Engineering;Information Science, School of Information Science;Technology, University of Science;Technology of China (USTC), Hefei, Anhui 230026, China School of Biomedical Engineering, Division of Life Sciences;Medicine, University of Science;Technology of China (USTC), Hefei, Anhui 230026, China Center for Medical Imaging, Robotics, Analytic Computing \& Learning (MIRACLE), Suzhou Institute for Advanced Research, University of Science;Technology of China (USTC), Suzhou, Jiangsu 215123, China Jiangsu Provincial Key Laboratory of Multimodal Digital Twin Technology, University of Science;Technology of China (USTC), Suzhou, Jiangsu 215123, China State Key Laboratory of Precision;Intelligent Chemistry, University of Science
Governance-Ready Small Language Models for Medical Imaging: Prompting, Abstention, and PACS Integration
Department of Data Science, University of Southern California(数据科学系,南加州大学);Department of Electrical and Computer Engineering, Carnegie Mellon University(电气与计算机工程系,卡内基梅隆大学);Department of Computer Science and Engineering, Santa Clara University(计算机科学与工程系,圣克拉拉大学);Department of Applied Mathematics, University of Washington(应用数学系,华盛顿大学);School of Computer Science, Georgia Institute of Technology(计算机科学学院,佐治亚理工学院)
Medical Image Synthesis via Fine-Grained Image-Text Alignment and Anatomy-Pathology Prompting
City University of Hong Kong(香港城市大学);The Chinese University of Hong Kong(香港中文大学);Massachusetts General Hospital(麻省总医院);Harvard Medical School(哈佛医学院);Lehigh University(里海大学)
针对医学图像数据稀缺及现有方法难以准确呈现解剖与病理细节的问题,提出结合解剖-病理提示和视觉码本细粒度图文对齐的合成模型,并在胸部 X 射线数据上验证其语义准确性。
Beyond Diagnosis: Evaluating Multimodal LLMs for Pathology Localization in Chest Radiographs
University of California(加州大学);Dana-Farber Cancer Institute(达纳-法伯癌症研究所);Massachusetts General Hospital(麻省总医院);St. Jude Children’s Research Hospital(圣犹大儿童研究医院);Brigham and Women’s Hospital & Harvard Medical School(布里法伦医院及哈佛医学院)
MedSyn2: Flexible Control of 3D CT Generation via Text and Semantically-Defined Segmentation Prompts
Boston University School of Engineering(波士顿大学工程学院);Stanford University(斯坦福大学);University of Pittsburgh Medical Center(匹兹堡大学医学中心);Boston University School of Medicine(波士顿大学医学院)
Hierarchical Modeling for Medical Visual Question Answering with Cross-Attention Fusion
School of Artificial Intelligence, Tiangong University(天津工业大学人工智能学院);Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);Linying Medical Technology (Shenzhen) Co., Ltd.(临影医疗科技(深圳)有限公司)
Enhancing Medical Visual Grounding via Knowledge-guided Spatial Prompts
School of Computer Science and Engineering, Nanjing University of Science and Technology(南京理工大学计算机科学与工程学院);School of Computer Science and Engineering, Southeast University(东南大学计算机科学与工程学院);Yong Yoo Lin School of Medicine, National University of Singapore(新加坡国立大学杨潞龄医学院);Institute of High-Performance Computing, Agency for Science, Technology and Research(新加坡科技研究局高性能计算研究所)
RadSAM: Segmenting 3D radiological images with a 2D promptable model
Raidium;Université de Paris Cité(巴黎大学);AP-HP(法国国家卫生与医学研究院);Hôpital Européen Georges Pompidou(欧仁·蓬皮杜医院);Department of Vascular and Oncological Interventional Radiology(血管和肿瘤介入放射科);HEKA INRIA;INSERM PARCC U 970(法国国家医学研究院 PARCC U 970)