Radiological services are experiencing unprecedented demand, leading to increased interest in automating radiology report generation. Existing Vision-Language Models (VLMs) suffer from hallucinations, lack interpretability, and require expensive fine-tuning. We introduce SAE-Rad, which uses sparse autoencoders (SAEs) to decompose latent representations from a pre-trained vision transformer into human-interpretable features. Our hybrid architecture combines state-of-the-art SAE advancements, achieving accurate latent reconstructions while maintaining sparsity. Using an off-the-shelf language model, we distil ground-truth reports into radiological descriptions for each SAE feature, which we then compile into a full report for each image, eliminating the need for fine-tuning large models for this task. To the best of our knowledge, SAE-Rad represents the first instance of using mechanistic interpretability techniques explicitly for a downstream multi-modal reasoning task. On the MIMIC-CXR dataset, SAE-Rad achieves competitive radiology-specific metrics compared to state-of-the-art models while using significantly fewer computational resources for training. Qualitative analysis reveals that SAE-Rad learns meaningful visual concepts and generates reports aligning closely with expert interpretations. Our results suggest that SAEs can enhance multimodal reasoning in healthcare, providing a more interpretable alternative to existing VLMs.
Structure is Supervision: Multiview Masked Autoencoders for Radiology
Department of Computer Science, ETH Zurich（苏黎世联邦理工学院计算机科学系）；Department of Computer Science, UC Irvine（加州大学尔湾分校计算机科学系）；Department of Quantitative Biomedicine, University of Zurich（苏黎世大学定量生物医学系）
StRegA: Unsupervised Anomaly Detection in Brain MRIs using a Compact Context-encoding Variational Autoencoder
Otto von Guericke University Magdeburg（马格德堡奥托·冯·格里克大学）；Center for Behavioral Brain Sciences（行为脑科学研究中心）；German Center for Neurodegenerative Disease（德国神经退行性疾病研究中心）
Semi-supervised classification of dental conditions in panoramic radiographs using large language model and instance segmentation: A real-world dataset evaluation
Institute of Computing, Federal University of Bahia（巴伊亚联邦大学计算学院）；Federal Institute of Bahia（巴伊亚联邦学院）；Federal University of Maranhão（马拉尼昂联邦大学）；Harvard School of Public Health（哈佛大学公共卫生学院）；Instituto de Ciência e Inovação em Engenharia Mecânica e Engenharia Industrial（机械与工业工程科学与创新研究所）；Faculdade de Engenharia, Universidade do Porto（波尔图大学工程学院）；Dentistry Faculty, Federal University of Bahia（巴伊亚联邦大学牙科学院）
Multiparametric Deep Learning Tissue Signatures for a Radiological Biomarker of Breast Cancer: Preliminary Results
The Johns Hopkins University School of Medicine（约翰霍普金斯大学医学院）；Sidney Kimmel Comprehensive Cancer Center（西德尼·基梅尔综合癌症中心）；The Johns Hopkins University（约翰霍普金斯大学）；University of Kentucky（肯塔基大学）；University of Wisconsin School of Medicine and Public Health（威斯康星大学医学与公共卫生学院）
Universal Boosts, Specific Suppressors: Sparse Autoencoder Steering of Medical Vision-Language Models
University of Zurich and University Hospital of Zurich（苏黎世大学及苏黎世大学医院）；Kobe University（Kobe大学）；ETH AI Center（苏黎世联邦理工学院人工智能中心）；ETH Zurich（苏黎世联邦理工学院）；Stanford University（斯坦福大学）；Zurich University of Applied Sciences（苏黎世应用科学大学）
Cardiac-CLIP: A Vision-Language Foundation Model for 3D Cardiac CT Images
Southeast University（东南大学）；Jinling Hospital, Affiliated Hospital of Medical School, Nanjing University（南京大学医学院附属金陵医院）；The First Affiliated Hospital of Nanjing Medical University（南京医科大学第一附属医院）；Shanghai AI Laboratory（上海人工智能实验室）；Nanjing First Hospital, Nanjing Medical University（南京医科大学附属南京医院）；Northern Jiangsu People’s Hospital（苏北人民医院）；the Affiliated Hospital of Jiangnan University（江南大学附属医院）；Sir Run Run Shaw Hospital, Zhejiang University School of Medicine（浙江大学医学院附属邵逸夫医院）；Beijing Anzhen Hospital, Capital Medical University（首都医科大学附属北京安贞医院）；Shanghai General Hospital, Shanghai Jiao Tong University School of Medicine（上海交通大学医学院附属第一人民医院）；Guangdong Provincial People’s Hospital（广东省人民医院）；Xijing Hospital, Air Force Medical University（空军军医大学西京医院）；Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College（中国医学科学院北京协和医学院北京协和医院）；Sir Run Run Hospital, Nanjing Medical University（南京医科大学附属逸夫医院）；Taizhou People’s Hospital（泰州市人民医院）；Case Western Reserve University（凯斯西储大学）
