通知：联邦节假日调休：哥伦布日 / 原住民日（Columbus Day / Indigenous Peoples' Day），今日（13日）无更新。通知：联邦节假日调休：哥伦布日 / 原住民日（Columbus Day / Indigenous Peoples' Day），今日（13日）无更新。
arXiv 2505.08838eess.IVcs.AIcs.CV
面向标准化文本的基于多模态大语言模型的超声报告生成
Ultrasound Report Generation with Multimodal Large Language Models for Standardized Texts
Shenzhen Institutes of Advanced Technology（深圳先进技术研究院）；University of Macau（澳门大学）；Chinese PLA General Hospital（中国人民解放军总医院）；Macau University of Science and Technology（澳门科学技术大学）
Peixuan Ge, Tongkun Su, Faqin Lv, Baoliang Zhao, Peng Zhang, Chi Hong Wong, Liang Yao, Yu Sun, Zenan Wang, Pak Kin Wong, Ying Hu
Ultrasound (US) report generation is a challenging task due to the variability of US images, operator dependence, and the need for standardized text. Unlike X-ray and CT, US imaging lacks consistent datasets, making automation difficult. In this study, we propose a unified framework for multi-organ and multilingual US report generation, integrating fragment-based multilingual training and leveraging the standardized nature of US reports. By aligning modular text fragments with diverse imaging data and curating a bilingual English-Chinese dataset, the method achieves consistent and clinically accurate text generation across organ sites and languages. Fine-tuning with selective unfreezing of the vision transformer (ViT) further improves text-image alignment. Compared to the previous state-of-the-art KMVE method, our approach achieves relative gains of about 2\% in BLEU scores, approximately 3\% in ROUGE-L, and about 15\% in CIDEr, while significantly reducing errors such as missing or incorrect content. By unifying multi-organ and multi-language report generation into a single, scalable framework, this work demonstrates strong potential for real-world clinical workflows.
Can Multimodal Decision Models Classify Breast Ultrasound Images? Benchmarking CLEF and CLEF-Flash Across Prompts and Datasets
Research and Development Center, VNSO Technology JSC（VNSO科技股份研发中心）；Department of Information Technology, Ho Chi Minh University of Education（胡志明市教育大学信息技术系）；Faculty of Information Science and Engineering, University of Information Technology（胡志明市信息技术大学信息科学与工程学部）；Vietnam National University, Ho Chi Minh City（胡志明市越南国立大学）
Breast Ultrasound Report Generation using LangChain
Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院）；Chung-Ang University Hospital（中央大学医院）；Chung-Ang University College of Medicine（中央大学医学院）
Semantic Scene Graph for Ultrasound Image Explanation and Scanning Guidance
Computer Aided Medical Procedures (CAMP)（计算机辅助医疗程序）；TU Munich, Germany（慕尼黑工业大学，德国）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；The Chinese University of Hong Kong, Hong Kong, China（香港中文大学，中国香港）
LLaVA-Ultra: Large Chinese Language and Vision Assistant for Ultrasound
ZJU-UIUC Institute, Zhejiang University（浙江大学ZJU-UIUC学院）；University of Washington（华盛顿大学）；Zhejiang University School of Medicine Sir Run Run Shaw Hospital（浙江大学医学院附属邵逸夫医院）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）
MedVQA-TREE: A Multimodal Reasoning and Retrieval Framework for Sarcopenia Prediction
Department of Electrical and Computer Engineering, Isfahan University of Technology（电气与计算机工程系，伊斯法罕技术大学）；SDU Health Informatics and Technology, The Maersk Mc-Kinney Moller Institute, University of Southern Denmark（南部丹麦大学健康信息学与技术，马士基麦金尼莫勒研究所）；Geriatric Research Unit, Department of Clinical Research, University of Southern Denmark（老年医学研究单元，临床研究系，南部丹麦大学）
Closing the Gap: Data-Centric Fine-Tuning of Vision Language Models for the Standardized Exam Questions
organization= Department of Computer Engineering, Middle East Technical University (METU) , city= Ankara , country= Türkiye；organization= METU-DTX Digital Transformation \& Innovation Centre, METU , city= Ankara , country= Türkiye
US-VLA: An Ultrasound Vision-Language-Action Model for Embodied Abdomina
Faculty of Computer Science and Technology, Ocean University of China（中国海洋大学计算机科学与技术学院）；School of Information Science and Engineering, Shandong University（山东大学信息科学与工程学院）；the Qilu Second Hospital of Shandong University（山东大学第二齐鲁医院）；Innovation School of Artificial Intelligence, Hefei University of Technology（合肥工业大学人工智能创新学院）
OADAT: Experimental and Synthetic Clinical Optoacoustic Data for Standardized Image Processing
Swiss Data Science Center（瑞士数据科学中心）；ETH Zurich（苏黎世联邦理工学院）；EPFL（洛桑联邦理工学院）；Institute for Machine Learning, Department of Computer Science, ETH Zurich（苏黎世联邦理工学院计算机科学系机器学习研究所）
We propose a three-dimensional (3D) multimodal medical imaging system that combines freehand ultrasound and structured light 3D reconstruction in a single coordinate system without requiring registration. To the best of our knowledge, these techniques have not been combined before as a multimodal imaging technique. The system complements the internal 3D information acquired with ultrasound, with the external surface measured with the structure light technique. Moreover, the ultrasound probe's optical tracking for pose estimation was implemented based on a convolutional neural network. Experime…
Emotion-LLaMAv2 and MMEVerse: A New Framework and Benchmark for Multimodal Emotion Understanding
Shenzhen Technology University（深圳技术大学）；Shenzhen University of Information Technology（深圳信息科技学院）；University of Washington（华盛顿大学）；Foundation Model Team, Meituan（美团基础模型团队）；National University of Singapore（新加坡国立大学）；Tongji University（同济大学）
OpenPros: A Large-Scale Dataset for Limited View Prostate Ultrasound Computed Tomography
The University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；Johns Hopkins University（约翰霍普金斯大学）；The Pennsylvania State University（宾夕法尼亚州立大学）；QT Imaging, Inc.（QT影像公司）；National Institutes of Health（美国国立卫生研究院）；Iowa State University（爱荷华州立大学）；Google DeepMind（谷歌DeepMind）
Learning multimodal representations involves integrating information from multiple heterogeneous sources of data. It is a challenging yet crucial area with numerous real-world applications in multimedia, affective computing, robotics, finance, human-computer interaction, and healthcare. Unfortunately, multimodal research has seen limited resources to study (1) generalization across domains and modalities, (2) complexity during training and inference, and (3) robustness to noisy and missing modalities. In order to accelerate progress towards understudied modalities and tasks while ensuring real…
Cesarean Scar Defect Segmentation in Transvaginal Ultrasound Images: a Dataset and Benchmark
Department of Obstetrics and Gynecology, International Peace Maternity and Child Health Hospital affiliated to Shanghai Jiao Tong University School of Medicine（妇产科部门，上海交通大学医学院国际和平妇产儿童医院）；School of Computer Science, University of Nottingham Ningbo China（Nottingham Ningbo中国大学计算机学院）；School of Computer Science, University of Nottingham（Nottingham大学计算机学院）；Department of Computer Science and Engineering, University of California, San Diego（加州大学圣地亚哥分校计算机科学与工程系）；Department of Ultrasound, International Peace Maternity and Child Health Hospital affiliated to Shanghai Jiao Tong University School of Medicine（超声科，上海交通大学医学院国际和平妇产儿童医院）；School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University（上海交通大学电子信息与电气工程学院）；Department of Cardiology, Gold Coast University Hospital（心内科，Gold Coast大学医院）
ViTaL: A Multimodality Dataset and Benchmark for Multi-pathological Ovarian Tumor Recognition
Department of Electronics and Information Engineering, Beihang University（电子信息工程系，北京航空航天大学）；Department of Gynecology and Obstetrics, Beijing Shijitan Hospital, Capital Medical University（妇产科，北京人民医院，首都医科大学）；Department of Computer Science, University of Liverpool（计算机科学系，利物浦大学）
Unified Review and Benchmark of Deep Segmentation Architectures for Cardiac Ultrasound on CAMUS
Department of Computer Science；Artificial Intelligence, Dongguk University, Seoul 04620, Republic of Korea；Department of Semiconductor System Engineering, Sejong University, Seoul, 05006, Republic of Korea；Department of Operations Research；Statistics, Faculty of Organizational Sciences, University of Belgrade, Belgrade, Serbia；Department of Industrial Engineering \& Management, Yuan Ze University, Taoyuan City 320315, Taiwan；Department of Applied Mathematical Science, College of Science；Technology, Korea University, Sejong 30019, Republic of Korea
In this study, we combine a standardized acquisition protocol with image analysis algorithms to investigate if it is possible to automatically detect maternal risk factors without a trained sonographer. The standardized acquisition protocol can be taught to any health care worker within two hours. This protocol was acquired from 280 pregnant women at St.\ Luke's Catholic Hospital, Wolisso, Ethiopia. A VGG-like network was used to perform a frame classification for each frame within the acquired ultrasound data. This frame classification was used to automatically determine the number of fetuses…
Placenta Accreta Spectrum Detection using Multimodal Deep Learning
Department of Computer Science, Faculty of Computing and Information Technology, King Abdulaziz University（计算机科学系，计算与信息科技学院，国王阿卜杜勒阿齐兹大学）；Department of Radiology, King Abdulaziz University Hospital（放射科，国王阿卜杜勒阿齐兹大学医院）
UrbanWell: Benchmarking Multimodal Large Language Models for Spatio-Temporal Urban Wellbeing Analytics
University of Helsinki（赫尔辛基大学）；Zhongguancun Academy（中关村学院）；University of Oxford（牛津大学）；Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
CHIMERA Challenge Task 2 and 3: Response Subtypes Classification and Progression Survival Prediction in Bladder Cancer Patients using Multimodal Datasets
Radboud University Medical Center（拉德堡德大学医学中心）；Erasmus University Medical Center（伊拉斯姆斯大学医学中心）；Oncode Institute（Oncode研究所）；Canisius Wilhelmina Hospital（卡尼修斯·威廉米娜医院）；Tampere University（坦佩雷大学）；Tampere University Hospital（坦佩雷大学医院）
Despite renewed awareness of the importance of articulation, it remains a challenge for instructors to handle the pronunciation needs of language learners. There are relatively scarce pedagogical tools for pronunciation teaching and learning. Unlike inefficient, traditional pronunciation instructions like listening and repeating, electronic visual feedback (EVF) systems such as ultrasound technology have been employed in new approaches. Recently, an ultrasound-enhanced multimodal method has been developed for visualizing tongue movements of a language learner overlaid on the face-side of the s…
Breast cancer is the most common invasive cancer in women. Besides the primary B-mode ultrasound screening, sonographers have explored the inclusion of Doppler, strain and shear-wave elasticity imaging to advance the diagnosis. However, recognizing useful patterns in all types of images and weighing up the significance of each modality can elude less-experienced clinicians. In this paper, we explore, for the first time, an automatic way to combine the four types of ultrasonography to discriminate between benign and malignant breast nodules. A novel multimodal network is proposed, along with pr…
Due to the high complexity and occlusion, insufficient perception in the crowded urban intersection can be a serious safety risk for both human drivers and autonomous algorithms, whereas CVIS (Cooperative Vehicle Infrastructure System) is a proposed solution for full-participants perception in this scenario. However, the research on roadside multimodal perception is still in its infancy, and there is no open-source dataset for such scenario. Accordingly, this paper fills the gap. Through an IPS (Intersection Perception System) installed at the diagonal of the intersection, this paper proposes …
FetalCLIP: A Visual-Language Foundation Model for Fetal Ultrasound Image Analysis
Department of Computer Vision（计算机视觉系）；Mohamed bin Zayed University of Artificial Intelligence（马尔代夫比兹人工智能大学）；Department of Machine Learning（机器学习系）；Corniche Hospital, Abu Dhabi Health Services Company (SEHA)（阿布扎赫尔医院，阿布扎赫健康服务公司（SEHA））
PulseLM: A Foundation Dataset and Benchmark for PPG-Text Learning
Singapore Management University（新加坡管理大学）；Queen Mary University of London（伦敦玛丽女王大学）；Eindhoven University of Technology（埃因霍温理工大学）；University of Cambridge（剑桥大学）
Rensselaer Polytechnic Institute（伦斯勒理工学院）；National Institutes of Health（美国国立卫生研究院）
Fusing intra-operative 2D transrectal ultrasound (TRUS) image with pre-operative 3D magnetic resonance (MR) volume to guide prostate biopsy can significantly increase the yield. However, such a multimodal 2D/3D registration problem is a very challenging task. In this paper, we propose an end-to-end frame-to-volume registration network (FVR-Net), which can efficiently bridge the previous research gaps by aligning a 2D TRUS frame with a 3D TRUS volume without requiring hardware tracking. The proposed FVR-Net utilizes a dual-branch feature extraction module to extract the information from TRUS fr…
We describe a point-set registration algorithm based on a novel free point transformer (FPT) network, designed for points extracted from multimodal biomedical images for registration tasks, such as those frequently encountered in ultrasound-guided interventional procedures. FPT is constructed with a global feature extractor which accepts unordered source and target point-sets of variable size. The extracted features are conditioned by a shared multilayer perceptron point transformer module to predict a displacement vector for each source point, transforming it into the target space. The point …
Force Sensing Guided Artery-Vein Segmentation via Sequential Ultrasound Images
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Centre for Artificial Intelligence and Robotics, HK Institute of Science & Innovation, Chinese Academy of Sciences（中国科学院香港创新研究院人工智能与机器人中心）；City University of Hong Kong（香港城市大学）
