Kaapana: A Comprehensive Open-Source Platform for Integrating AI in Medical Imaging Research Environments
Division of Medical Image Computing, German Cancer Research Center, Heidelberg, Germany（德国癌症研究中心医学影像计算部，海德堡，德国）；Medical Faculty, University of Heidelberg, Heidelberg, Germany（海德堡大学医学系，海德堡，德国）；Computer Science Faculty, University of Heidelberg, Heidelberg, Germany（海德堡大学计算机科学系，海德堡，德国）；Pattern Analysis and Learning Group, Department of Radiation Oncology, Heidelberg University Hospital, Heidelberg, Germany（放射肿瘤学部门模式分析与学习组，海德堡大学医院，海德堡，德国）；German Cancer Consortium (DKTK), Partner Site Heidelberg, Heidelberg, Germany（德国癌症联盟（DKTK），海德堡合作伙伴站点，海德堡，德国）；Heidelberg Institute of Radiation Oncology (HIRO), National Center for Radiation Research in Oncology (NCRO), Heidelberg, Germany（海德堡放射肿瘤学研究所（HIRO），肿瘤放射研究国家中心（NCRO），海德堡，德国）；National Center for Tumor Diseases (NCT), NCT Heidelberg, a partnership between DKFZ and the University Medical Center Heidelberg, 69120 Heidelberg, Germany（肿瘤疾病国家中心（NCT），海德堡NCT，德克萨斯儿童医学研究中心（DKFZ）与海德堡大学医学中心之间的合作，海德堡，德国）
Ünal Akünal, Markus Bujotzek, Stefan Denner, Benjamin Hamm, Klaus Kades, Philipp Schader, Jonas Scherer, Marco Nolden, Peter Neher, Ralf Floca, Klaus Maier-Hein
Developing generalizable AI for medical imaging requires both access to large, multi-center datasets and standardized, reproducible tooling within research environments. However, leveraging real-world imaging data in clinical research environments is still hampered by strict regulatory constraints, fragmented software infrastructure, and the challenges inherent in conducting large-cohort multicentre studies. This leads to projects that rely on ad-hoc toolchains that are hard to reproduce, difficult to scale beyond single institutions and poorly suited for collaboration between clinicians and data scientists. We present Kaapana, a comprehensive open-source platform for medical imaging research that is designed to bridge this gap. Rather than building single-use, site-specific tooling, Kaapana provides a modular, extensible framework that unifies data ingestion, cohort curation, processing workflows and result inspection under a common user interface. By bringing the algorithm to the data, it enables institutions to keep control over their sensitive data while still participating in distributed experimentation and model development. By integrating flexible workflow orchestration with user-facing applications for researchers, Kaapana reduces technical overhead, improves reproducibility and enables conducting large-scale, collaborative, multi-centre imaging studies. We describe the core concepts of the platform and illustrate how they can support diverse use cases, from local prototyping to nation-wide research networks. The open-source codebase is available at https://github.com/kaapana/kaapana
An Open-Source Monitoring Framework for Data Exploration and Progress Tracking in Multi-Center Radiology Studies
Germany Cancer Research Center（德国癌症研究中心）；University of Heidelberg（海德堡大学）；University Hospital Frankfurt（法兰克福大学医院）；Charite Universitätsmedizin Berlin（柏林夏里特医学院）；Berlin Institute of Health（柏林健康研究所）
Efficient Large Scale Medical Image Dataset Preparation for Machine Learning Applications
German Cancer Research Center (DKFZ)（德国癌症研究中心（DKFZ））；University of Heidelberg（海德堡大学）；Goethe University Frankfurt（法兰克福歌德大学）；Charité - Universitätsmedizin Berlin（柏林夏里特医学院）
Heidelberg University（海德堡大学）；German Cancer Research Center（德国癌症研究中心）
One of the greatest challenges in the medical imaging domain is to successfully transfer deep learning models into clinical practice. Since models are often trained on a specific body region, a robust transfer into the clinic necessitates the selection of images with body regions that fit the algorithm to avoid false-positive predictions in unknown regions. Due to the insufficient and inaccurate nature of manually-defined imaging meta-data, automated body part recognition is a key ingredient towards the broad and reliable adoption of medical deep learning models. While some approaches to this …
Integrating MedCLIP and Cross-Modal Fusion for Automatic Radiology Report Generation
The Bishop Strachan School（毕夏普斯特拉坎学校）；Baidu Inc.（百度公司）；Beihang University（北京航空航天大学）；Institute of Future Education（未来教育研究所）；TechArena Canada Inc.（加拿大泰克Arena公司）；College of Engineering（工程学院）；Northeastern University（东北大学）
University of Central Florida（中佛罗里达大学）；National Institutes of Health（美国国立卫生研究院）；Soochow University（苏州大学）
This study introduced a novel system, called Gaze2Segment, integrating biological and computer vision techniques to support radiologists' reading experience with an automatic image segmentation task. During diagnostic assessment of lung CT scans, the radiologists' gaze information were used to create a visual attention map. This map was then combined with a computer-derived saliency map, extracted from the gray-scale CT images. The visual attention map was used as an input for indicating roughly the location of a object of interest. With computer-derived saliency information, on the other hand…
Beihang University（北京航空航天大学）；Beijing Advanced Innovation Center for Big Data and Brain Computing（北京大数据与脑计算高精尖创新中心）；Hangzhou Innovation Institute of Beihang University（北京航空航天大学杭州创新研究院）；The University of Texas at Dallas（德克萨斯大学达拉斯分校）；University of Technology Sydney（悉尼科技大学）
Although deep learning models like CNNs have achieved great success in medical image analysis, the small size of medical datasets remains a major bottleneck in this area. To address this problem, researchers have started looking for external information beyond current available medical datasets. Traditional approaches generally leverage the information from natural images via transfer learning. More recent works utilize the domain knowledge from medical doctors, to create networks that resemble how medical doctors are trained, mimic their diagnostic patterns, or focus on the features or areas …
Integrating Medical Imaging and Clinical Reports Using Multimodal Deep Learning for Advanced Disease Analysis
Penn State University（宾夕法尼亚州立大学）；Mississippi State University（密西西比州立大学）；Northwest Missouri State University（西北密苏里州立大学）；University of California, Los Angeles（加利福尼亚大学洛杉矶分校）；University of California, Berkeley（加利福尼亚大学伯克利分校）；Colorado State University（科罗拉多州立大学）
Text-guided Foundation Model Adaptation for Long-Tailed Medical Image Classification
Southern University of Science and Technology（南方科技大学）；The University of Hong Kong（香港大学）；University of British Columbia（不列颠哥伦比亚大学）；Jiaxing Research Institute, Southern University of Science and Technology（南方科技大学嘉兴研究院）
SMFusion: Semantic-Preserving Fusion of Multimodal Medical Images for Enhanced Clinical Diagnosis
College of Information and Intelligence, Hunan Agricultural University（湖南农业大学信息与智能学院）；College of Artificial Intelligence, Nankai University（南开大学人工智能学院）
Lung nodules suffer large variation in size and appearance in CT images. Nodules less than 10mm can easily lose information after down-sampling in convolutional neural networks, which results in low sensitivity. In this paper, a combination of 3D image and feature pyramid is exploited to integrate lower-level texture features with high-level semantic features, thus leading to a higher recall. However, 3D operations are time and memory consuming, which aggravates the situation with the explosive growth of medical images. To tackle this problem, we propose a general curriculum training strategy …
Large language models (LLMs) have recently demonstrated their potential in clinical applications, providing valuable medical knowledge and advice. For example, a large dialog LLM like ChatGPT has successfully passed part of the US medical licensing exam. However, LLMs currently have difficulty processing images, making it challenging to interpret information from medical images, which are rich in information that supports clinical decisions. On the other hand, computer-aided diagnosis (CAD) networks for medical images have seen significant success in the medical field by using advanced deep-le…
Multi-Scale Transformer Architecture for Accurate Medical Image Classification
Tulane University（杜兰大学）；University of Houston（休斯顿大学）；University of Pennsylvania（宾夕法尼亚大学）；Shanghai Jiao Tong University（上海交通大学）；New York University（纽约大学）；New York University Shanghai（纽约大学上海分校）
Technion Institute of Technology（以色列理工学院）；Rambam Health Care Campus（兰巴姆医疗中心）
A key concern in integrating machine learning models in medicine is the ability to interpret their reasoning. Popular explainability methods have demonstrated satisfactory results in natural image recognition, yet in medical image analysis, many of these approaches provide partial and noisy explanations. Recently, attention mechanisms have shown compelling results both in their predictive performance and in their interpretable qualities. A fundamental trait of attention is that it leverages salient parts of the input which contribute to the model's prediction. To this end, our work focuses on …
INESC-ID（INESC-ID（葡萄牙国家系统与计算机工程研究所））；Instituto Superior Técnico（里斯本高等技术学院）；Universidade de Lisboa（里斯本大学）；School of Information Systems（信息系统学院）；Queensland University of Technology（昆士兰科技大学）；Grupo Lusíadas（卢西亚达斯集团）
This paper proposes a novel multimodal DL architecture incorporating medical images and eye-tracking data for abnormality detection in chest x-rays. Our results show that applying eye gaze data directly into DL architectures does not show superior predictive performance in abnormality detection chest X-rays. These results support other works in the literature and suggest that human-generated data, such as eye gaze, needs a more thorough investigation before being applied to DL architectures.
Random Token Fusion for Multi-View Medical Diagnosis
KTH Royal Institute of Technology（皇家理工学院）；Science for Life Laboratory（生命科学实验室）；Karolinska Institutet（卡罗林斯卡学院）；Karolinska University Hospital（卡罗林斯卡大学医院）
INESC TEC（葡萄牙系统与计算机工程研究所）；University of Porto（波尔图大学）
The increasing popularity of attention mechanisms in deep learning algorithms for computer vision and natural language processing made these models attractive to other research domains. In healthcare, there is a strong need for tools that may improve the routines of the clinicians and the patients. Naturally, the use of attention-based algorithms for medical applications occurred smoothly. However, being healthcare a domain that depends on high-stake decisions, the scientific community must ponder if these high-performing algorithms fit the needs of medical applications. With this motto, this …
Microsoft Research（微软研究院）；TuSimple（图森未来）；Baidu Research（百度研究院）；Tencent Medical AI Labs（腾讯医疗AI实验室）；PingAn Tech（平安科技）
Medical imaging is an essential tool in many areas of medical applications, used for both diagnosis and treatment. However, reading medical images and making diagnosis or treatment recommendations require specially trained medical specialists. The current practice of reading medical images is labor-intensive, time-consuming, costly, and error-prone. It would be more desirable to have a computer-aided system that can automatically make diagnosis and treatment recommendations. Recent advances in deep learning enable us to rethink the ways of clinician diagnosis based on medical images. In this t…
MTL-MAD: Multi-Task Learners are Effective Medical Anomaly Detectors
Rayscape；University of Bucharest（布加勒斯特大学）；Polytechnic University of Timişoara（蒂米什瓦拉工业大学）；Colţea Clinical Hospital（科尔切亚医院）；Carol Davila University of Medicine and Pharmacy（卡罗尔·迪瓦ila医学院和药理学大学）
CDPDNet: Integrating Text Guidance with Hybrid Vision Encoders for Medical Image Segmentation
J. Crayton Pruitt Family Department of Biomedical Engineering, University of Florida（J. Crayton Pruitt家庭生物医学工程系，佛罗里达大学）；Department of Medicine, University of Florida（医学系，佛罗里达大学）
Towards Clinician-Preferred Segmentation: Leveraging Human-in-the-Loop for Test Time Adaptation in Medical Image Segmentation
Northwestern Polytechnical University（西北工业大学）；Ningbo Institute of Northwestern Polytechnical University（西北工业大学宁波研究院）；Research and Development Institute of Northwestern Polytechnical University in Shenzhen（西北工业大学深圳研究院）
Lightweight Medical Image Restoration via Integrating Reliable Lesion-Semantic Driven Prior
University of Electronic Science and Technology of China（电子科技大学）；City University of Hong Kong（香港城市大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Shenzhen Institute For Advanced Study, University of Electronic Science and Technology of China（电子科技大学深圳研究院）
Integrating Vision and Location with Transformers: A Multimodal Deep Learning Framework for Medical Wound Analysis
University of Zanjan（赞詹大学）；Islamic Azad University Science and Research Branch（伊斯兰阿扎德大学科学与研究分支）；Samarkand State University named after Sharof Rashidov（沙罗夫·拉希多夫命名的撒马尔罕州立大学）；Kimyo International University In Tashkent（塔什干基莫国际大学）；University of Tehran（德黑兰大学）；Drexel University（德雷塞尔大学）
Enhancing Pneumonia Diagnosis and Severity Assessment through Deep Learning: A Comprehensive Approach Integrating CNN Classification and Infection Segmentation
Med-CMR: A Fine-Grained Benchmark Integrating Visual Evidence and Clinical Logic for Medical Complex Multimodal Reasoning
National University of Singapore（新加坡国立大学）；Nanjing University（南京大学）；Tongji University（同济大学）；Ruijin Hospital（瑞金医院）；Technical University of Munich（慕尼黑工业大学）；Zhejiang University（浙江大学）
