Radiological analysis increasingly benefits from pretrained visual representations that can support heterogeneous downstream tasks across imaging modalities. In this work, we introduce OmniRad, a self-supervised radiological foundation model pretrained on 1.2 million medical images, designed with radiology-inspired principles emphasizing representation reuse and cross-task transferability. We evaluate the pretrained encoder under multiple downstream adaptation regimes, including lightweight task-specific adapters with a frozen backbone as well as full end-to-end fine-tuning for classification, allowing us to assess both representation quality and task-specific performance. OmniRad is evaluated on a broad suite of public benchmarks spanning classification and segmentation across multiple modalities. On the MedMNISTv2 collection, OmniRad improves classification F1 by up to 2.05% over competing foundation models. For dense prediction, OmniRad attains mean Dice score improvements across six MedSegBench datasets when using frozen representations. Qualitative analyses and latent-space visualizations suggest improved feature clustering and modality-related separation.
University of Oxford（牛津大学）；Plexalis Ltd（Plexalis有限公司）
This paper proposes a novel transformer-based model architecture for medical imaging problems involving analysis of vertebrae. It considers two applications of such models in MR images: (a) detection of spinal metastases and the related conditions of vertebral fractures and metastatic cord compression, (b) radiological grading of common degenerative changes in intervertebral discs. Our contributions are as follows: (i) We propose a Spinal Context Transformer (SCT), a deep-learning architecture suited for the analysis of repeated anatomical structures in medical imaging such as vertebral bodies…
SD-MAD: Sign-Driven Few-shot Multi-Anomaly Detection in Medical Images
Shanghai Academy of AI for Science University of Queensland（上海人工智能科学研究院 华南理工大学）；Fudan University Shanghai Academy of AI for Science（复旦大学 上海人工智能科学研究院）；University of Queensland（昆士兰大学）
RadSAM: Segmenting 3D radiological images with a 2D promptable model
Raidium；Université de Paris Cité（巴黎大学）；AP-HP（法国国家卫生与医学研究院）；Hôpital Européen Georges Pompidou（欧仁·蓬皮杜医院）；Department of Vascular and Oncological Interventional Radiology（血管和肿瘤介入放射科）；HEKA INRIA；INSERM PARCC U 970（法国国家医学研究院 PARCC U 970）
Agentic Large Language Models for Training-Free Neuro-Radiological Image Analysis
Department of Radiation Oncology, TUM University Hospital, Munich, Germany（放射肿瘤科，慕尼黑技术大学医院，德国）；Chair for AI in Healthcare and Medicine, Technical University of Munich (TUM)（人工智能在医疗和健康领域的主任，慕尼黑技术大学）；Chair for AI for Image-Guided Diagnosis and Therapy, Technical University of Munich (TUM)（人工智能在影像引导诊断和治疗领域的主任，慕尼黑技术大学）；Munich Center for Machine Learning (MCML), Munich, Germany（慕尼黑机器学习中心（MCML），德国慕尼黑）；Department of Computing, Imperial College London, London, UK（计算系，伦敦帝国学院，英国伦敦）；Deutsches Konsortium für Translationale Krebsforschung (DKTK), Partner Site Munich, Munich, Germany（德国转化癌症研究联盟（DKTK），慕尼黑分部，德国慕尼黑）
Convolutional neural networks (CNNs) have been successfully employed in recent years for the detection of radiological abnormalities in medical images such as plain x-rays. To date, most studies use CNNs on individual examinations in isolation and discard previously available clinical information. In this study we set out to explore whether Long-Short-Term-Memory networks (LSTMs) can be used to improve classification performance when modelling the entire sequence of radiographs that may be available for a given patient, including their reports. A limitation of traditional LSTMs, though, is tha…
The manifold hypothesis is a core mechanism behind the success of deep learning, so understanding the intrinsic manifold structure of image data is central to studying how neural networks learn from the data. Intrinsic dataset manifolds and their relationship to learning difficulty have recently begun to be studied for the common domain of natural images, but little such research has been attempted for radiological images. We address this here. First, we compare the intrinsic manifold dimensionality of radiological and natural images. We also investigate the relationship between intrinsic dime…
Technion – Israel Institute of Technology（以色列理工学院）
Medical imaging analysis plays a critical role in the diagnosis and treatment of various medical conditions. This paper focuses on chest X-ray images and their corresponding radiological reports. It presents a new model that learns a joint X-ray image & report representation. The model is based on a novel alignment scheme between the visual data and the text, which takes into account both local and global information. Furthermore, the model integrates domain-specific information of two types -- lateral images and the consistent visual structure of chest images. Our representation is shown to b…
Sierra Research S.L（塞拉研究有限公司）；Universidad de Alicante（阿利坎特大学）；Medbravo（Medbravo医疗公司）；Fundación para el Fomento de la Investigación Sanitario y Biomédica de la Comunidad Valenciana（瓦伦西亚自治区卫生与生物医学研究促进基金会）；Hospital San Juan de Alicante（阿利坎特圣胡安医院）；Universidad Miguel Hernández（米格尔·埃尔南德斯大学）
In this work we present a method for the detection of radiological findings, their location and differential diagnoses from chest x-rays. Unlike prior works that focus on the detection of few pathologies, we use a hierarchical taxonomy mapped to the Unified Medical Language System (UMLS) terminology to identify 189 radiological findings, 22 differential diagnosis and 122 anatomic locations, including ground glass opacities, infiltrates, consolidations and other radiological findings compatible with COVID-19. We train the system on one large database of 92,594 frontal chest x-rays (AP or PA, st…
A Modular Agent for Reliable and Auditable Spatial Relation Verification in CT Scans
Ulm University（乌尔姆大学）；Ulm University Hospital（乌尔姆大学医院）；Technical University of Munich (TUM)（慕尼黑工业大学（TUM））；TUM University Hospital（慕尼黑工业大学医院）；Department of Radiation Oncology, TUM University Hospital（慕尼黑工业大学医院放射肿瘤科）
Unidad Mixta de Imagen Biomédica FISABIO-CIPF（FISABIO-CIPF生物医学成像联合单元）；Universidad de Alicante（阿利坎特大学）；Medbravo；Hospital San Juan de Alicante（阿利坎特圣胡安医院）；Universidad Miguel Hernández（米格尔·埃尔南德斯大学）；Sierra Research SL（Sierra Research有限公司）；Bioinformatics & Biostatistics Unit, Principe Felipe Research Center（费利佩王子研究中心生物信息学与生物统计学单元）
This paper describes BIMCV COVID-19+, a large dataset from the Valencian Region Medical ImageBank (BIMCV) containing chest X-ray images CXR (CR, DX) and computed tomography (CT) imaging of COVID-19+ patients along with their radiological findings and locations, pathologies, radiological reports (in Spanish), DICOM metadata, Polymerase chain reaction (PCR), Immunoglobulin G (IgG) and Immunoglobulin M (IgM) diagnostic antibody tests. The findings have been mapped onto standard Unified Medical Language System (UMLS) terminology and cover a wide spectrum of thoracic entities, unlike the considerab…
Efficient Large Scale Medical Image Dataset Preparation for Machine Learning Applications
German Cancer Research Center (DKFZ)（德国癌症研究中心（DKFZ））；University of Heidelberg（海德堡大学）；Goethe University Frankfurt（法兰克福歌德大学）；Charité - Universitätsmedizin Berlin（柏林夏里特医学院）
Imperial College London（帝国理工学院）；Technische Universität München（慕尼黑工业大学）；FAU Erlangen–Nürnberg（埃尔朗根-纽伦堡大学）；National Institutes of Health（美国国立卫生研究院）
Chest radiographs are one of the most common diagnostic modalities in clinical routine. It can be done cheaply, requires minimal equipment, and the image can be diagnosed by every radiologists. However, the number of chest radiographs obtained on a daily basis can easily overwhelm the available clinical capacities. We propose RATCHET: RAdiological Text Captioning for Human Examined Thoraces. RATCHET is a CNN-RNN-based medical transformer that is trained end-to-end. It is capable of extracting image features from chest radiographs, and generates medically accurate text reports that fit seamless…
SAM: Self-supervised Learning of Pixel-wise Anatomical Embeddings in Radiological Images
Alibaba Group DAMO Academy（阿里巴巴集团达摩院）；PAII Inc.（PAII公司）；Ping An Insurance (Group) Company of China, Ltd.（中国平安保险（集团）股份有限公司）；Beijing United Family Hospital（北京和睦家医院）
We present in this paper a new approach for the automatic annotation of medical images, using the approach of "bag-of-words" to represent the visual content of the medical image combined with text descriptors based approach tf.idf and reduced by latent semantic to extract the co-occurrence between terms and visual terms. A medical report is composed of a text describing a medical image. First, we are interested to index the text and extract all relevant terms using a thesaurus containing MeSH medical concepts. In a second phase, the medical image is indexed while recovering areas of interest w…
ViT3D Alignment of LLaMA3: 3D Medical Image Report Generation
School of Electronic Engineering and Computer Science, Queen Mary University of London（伦敦玛丽女王大学电子工程与计算机科学学院）；Department of Bioengineering, Imperial College London（伦敦帝国理工学院生物工程系）；Meta Inc.（Meta公司）；School of Engineering, College of Engineering and Physical Sciences, University of Birmingham（伯明翰大学工程学院、工程与物理科学学院）
Any-to-Any Vision-Language Model for Multimodal X-ray Imaging and Radiological Report Generation
Department of Diagnostics and Intervention, Biomedical Engineering and Radiation Physics, Umeå University（诊断与介入部门，生物医学工程与放射物理，乌梅大学）；College of Computer Science and Software Engineering, Shenzhen University（计算机科学与软件工程学院，深圳大学）
Decision support tools that rely on supervised learning require large amounts of expert annotations. Using past radiological reports obtained from hospital archiving systems has many advantages as training data above manual single-class labels: they are expert annotations available in large quantities, covering a population-representative variety of pathologies, and they provide additional context to pathology diagnoses, such as anatomical location and severity. Learning to auto-generate such reports from images present many challenges such as the difficulty in representing and generating long…
Technical University of Munich（慕尼黑工业大学）；Imperial College London（帝国理工学院）
Contrastive learning has proven effective for pre-training image models on unlabeled data with promising results for tasks such as medical image classification. Using paired text (like radiological reports) during pre-training improves the results even further. Still, most existing methods target image classification downstream tasks and may not be optimal for localized tasks like semantic segmentation or object detection. We therefore propose Localized representation learning from Vision and Text (LoVT), to our best knowledge, the first text-supervised pre-training method that targets localiz…
National and Kapodistrian University of Athens（雅典国立卡波迪斯特里亚大学）
Medical Informatics and the application of modern signal processing in the assistance of the diagnostic process in medical imaging is one of the more recent and active research areas today. This thesis addresses a variety of issues related to the general problem of medical image analysis, specifically in mammography, and presents a series of algorithms and design approaches for all the intermediate levels of a modern system for computer-aided diagnosis (CAD). The diagnostic problem is analyzed with a systematic approach, first defining the imaging characteristics and features that are relevant…
DeepTek；Dr. D.Y. Patil Medical College, Hospital and Research Centre（D.Y. Patil 医学院、医院与研究中心）
Radiology, being one of the younger disciplines of medicine with a history of just over a century, has witnessed tremendous technological advancements and has revolutionized the way we practice medicine today. In the last few decades, medical imaging modalities have generated seismic amounts of medical data. The development and adoption of Artificial Intelligence (AI) applications using this data will lead to the next phase of evolution in radiology. It will include automating laborious manual tasks such as annotations, report-generation, etc., along with the initial radiological assessment of…
Eyes Tell the Truth: GazeVal Highlights Shortcomings of Generative AI in Medical Imaging
Northwestern University（西北大学）；University of Illinois at Chicago（伊利诺伊大学芝加哥分校）；Stanford University（斯坦福大学）；Université Sorbonne Paris Nord（巴黎北索邦大学）；Loyola University Chicago（芝加哥洛约拉大学）；DuPage Medical Group（杜佩奇医疗集团）；University of Pennsylvania（宾夕法尼亚大学）；Emory University（埃默里大学）
针对医学影像中合成数据评估与临床真实性脱节的问题，提出 GazeVal 框架，结合专家眼动追踪与放射学评估，实验发现 96.6% 的生成图像被识破，凸显生成式 AI 的不足。
A multimodal vision foundation model for generalizable knee pathology
Beihang University（北京航空航天大学）；Peking University Third Hospital（北京大学第三医院）；Institute of Sports Medicine of Peking University（北京大学运动医学研究所）；Ministry of Education（教育部）
MicarVLMoE: A Modern Gated Cross-Aligned Vision-Language Mixture of Experts Model for Medical Image Captioning and Report Generation
Faculty of Computer Science and Information Technology, Universiti Malaya（马来亚大学计算机科学与信息学院）；School of Computing and Information Systems, The University of Melbourne（墨尔本大学计算机与信息学院）
MedicalPatchNet: A Patch-Based Self-Explainable AI Architecture for Chest X-ray Classification
Department of Diagnostic and Interventional Radiology of the University Hospital RWTH Aachen（鲁尔大学亚琛医院诊断与介入放射科）；Else Kroener Fresenius Center for Digital Health, Technical University Dresden（德累斯顿技术大学埃勒克罗纳弗雷斯内尔数字健康中心）；Department of Medicine III, University Hospital RWTH Aachen（鲁尔大学亚琛医院第三医学部）；Pathology & Data Analytics, Leeds Institute of Medical Research at St James’s, University of Leeds（利兹大学圣詹姆斯医院医学研究所以及病理学与数据科学）；Department of Medicine I, University Hospital Dresden（德累斯顿大学第一医学部）；Medical Oncology, National Center for Tumor Diseases (NCT), University Hospital Heidelberg（海德堡大学国家肿瘤疾病中心（NCT）肿瘤医学部）
Tencent Jarvis Lab（腾讯Jarvis实验室）；Institute of Computing Technology Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；The First Affiliated Hospital of Anhui Medical University（安徽医科大学第一附属医院）；University of Science and Technology of China（中国科学技术大学）
Fast screening and diagnosis are critical in COVID-19 patient treatment. In addition to the gold standard RT-PCR, radiological imaging like X-ray and CT also works as an important means in patient screening and follow-up. However, due to the excessive number of patients, writing reports becomes a heavy burden for radiologists. To reduce the workload of radiologists, we propose DeltaNet to generate medical reports automatically. Different from typical image captioning approaches that generate reports with an encoder and a decoder, DeltaNet applies a conditional generation process. In particular…
Curia-2: Scaling Self-Supervised Learning for Radiology Foundation Models
Raidium；Department of Vascular and Oncological Interventional Radiology, Hôpital Européen Georges Pompidou, AP-HP, Paris, France（巴黎欧洲乔治·蓬皮杜医院血管与肿瘤介入放射科，AP-HP）；Faculté de Santé, Université Paris-Cité, Paris, France（巴黎西岱大学健康学院）
Department of Radiology and Imaging Sciences, Clinical Center（临床中心放射与影像科学系）
Chest X-rays are one of the most common radiological examinations in daily clinical routines. Reporting thorax diseases using chest X-rays is often an entry-level task for radiologist trainees. Yet, reading a chest X-ray image remains a challenging job for learning-oriented machine intelligence, due to (1) shortage of large-scale machine-learnable medical image datasets, and (2) lack of techniques that can mimic the high-level reasoning of human radiologists that requires years of knowledge accumulation and professional training. In this paper, we show the clinical free-text radiological repor…
Recent advances in deep learning have led to a promising performance in many medical image analysis tasks. As the most commonly performed radiological exam, chest radiographs are a particularly important modality for which a variety of applications have been researched. The release of multiple, large, publicly available chest X-ray datasets in recent years has encouraged research interest and boosted the number of publications. In this paper, we review all studies using deep learning on chest radiographs, categorizing works by task: image-level prediction (classification and regression), segme…
CT-GRAPH: Hierarchical Graph Attention Network for Anatomy-Guided CT Report Generation
Institute for AI in Medicine (IKIM), University Hospital Essen (AöR)（人工智能医学研究所（IKIM）、埃森大学医院（AöR））；Department of Nuclear Medicine, University Hospital Essen (AöR)（核医学系、埃森大学医院（AöR））；Department of Physics, TU Dortmund（物理系、多特蒙德技术大学）
Comprehensive language-image pre-training for 3D medical image understanding
Microsoft（微软公司）；German Cancer Research Center (DKFZ)（德国癌症研究中心）；Department of Radiology, University of Cambridge and Cambridge University Hospitals NHS Foundation Trust（剑桥大学放射科及剑桥大学医院国家健康服务体系基金会信托）；Pattern Analysis and Learning Group, Heidelberg University Hospital（海德堡大学医院模式分析与学习组）；Department of Radiology, Mayo Clinic（梅奥诊所放射科）
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Weill Cornell Medicine（威尔康奈尔医学院）
Chest X-ray becomes one of the most common medical diagnoses due to its noninvasiveness. The number of chest X-ray images has skyrocketed, but reading chest X-rays still have been manually performed by radiologists, which creates huge burnouts and delays. Traditionally, radiomics, as a subfield of radiology that can extract a large number of quantitative features from medical images, demonstrates its potential to facilitate medical imaging diagnosis before the deep learning era. In this paper, we develop an end-to-end framework, ChexRadiNet, that can utilize the radiomics features to improve t…
In medical imaging, radiological scans of different modalities serve to enhance different sets of features for clinical diagnosis and treatment planning. This variety enriches the source information that could be used for outcome prediction. Deep learning methods are particularly well-suited for feature extraction from high-dimensional inputs such as images. In this work, we apply a CNN classification network augmented with a FCN preprocessor sub-network to a public TCIA head and neck cancer dataset. The training goal is survival prediction of radiotherapy cases based on pre-treatment FDG PET-…
