Sparse autoencoders (SAEs) offer a natural path toward comparable explanations across different representation spaces. However, current SAEs are trained per modality, producing dictionaries whose features are not directly understandable and whose explanations do not transfer across domains. In this study, we introduce LUCID (Learning Unified vision-language sparse Codes for Interpretable concept Discovery), a unified vision-language sparse autoencoder that learns a shared latent dictionary for image patch and text token representations, while reserving private capacity for modality-specific details. We achieve feature alignment by coupling the shared codes with a learned optimal transport matching objective without the need of labeling. LUCID yields interpretable shared features that support patch-level grounding, establish cross-modal neuron correspondence, and enhance robustness against the concept clustering problem in similarity-based evaluation. Leveraging the alignment properties, we develop an automated dictionary interpretation pipeline based on term clustering without manual observations. Our analysis reveals that LUCID's shared features capture diverse semantic categories beyond objects, including actions, attributes, and abstract concepts, demonstrating a comprehensive approach to interpretable multimodal representations.
Verifying the Linear Representation Hypothesis: How Interpretable Are Vision SAEs?
Fraunhofer SIT(弗劳恩霍夫安全信息技术研究所);ATHENE National Research Center for Applied Cybersecurity(ATHENE国家应用网络安全研究中心);AUMOVIO;University of Bamberg(班贝格大学);Berlin University of Applied Sciences(柏林应用科学大学);Einstein Center Digital Future(爱因斯坦数字未来中心)
Evaluating the Interpretability of Sparse Autoencoders with Concept Annotations
The Berlin Institute for the Foundations of Learning and Data (BIFOLD)(柏林学习与数据基础研究所);Technische Universität Berlin(柏林工业大学);INRAE(法国国家农业、食品与环境研究院);Inria, EVERGREEN(法国国家信息与自动化研究所,EVERGREEN);UMR TETIS, Univ Montpellier(UMR TETIS,蒙彼利埃大学)
MonoLoss: A Training Objective for Interpretable Monosemantic Representations
Department of Computer Science and Software Engineering, Concordia University, Canada(康科迪亚大学计算机科学与软件工程系);Department of Computer Science, Stony Brook University, United States(石溪大学计算机科学系);Department of Electrical and Computer Engineering, Concordia University, Canada(康科迪亚大学电气与计算机工程系);Mila--Quebec AI Institute, Canada(魁北克人工智能研究所)
Similarity-as-Evidence: Calibrating Overconfident VLMs for Interpretable and Label-Efficient Medical Active Learning
School of Electronic Technology and Engineering, Xiamen University, Xiamen, China(厦门大学电子技术与工程学院,厦门,中国);School of Informatics, Xiamen University, Xiamen, China(厦门大学信息学院,厦门,中国);School of Engineering, Case Western Reserve University, Cleveland, USA(凯斯西储大学工程学院,克利夫兰,美国)
Sparse Autoencoders for Interpretable Medical Image Representation Learning
Stanford Center for Artificial Intelligence in Medicine and Imaging(斯坦福大学人工智能医学与成像中心);Department of Radiology, Stanford University(斯坦福大学放射科);Chair of AI in Healthcare and Medicine, Technical University of Munich(慕尼黑技术大学人工智能医疗与医学主任);TUM University Hospital, Munich, Germany(慕尼黑技术大学医院)
Archetypal SAE: Adaptive and Stable Dictionary Learning for Concept Extraction in Large Vision Models
Kempner Institute(肯普纳研究所);Harvard University(哈佛大学);CBS-NTT Program in Physics of Intelligence(CBS-NTT 智能物理项目);Dept. of Psychology(心理学系);FAR AI;CerCo;York University(约克大学);CNRS(法国国家科学研究中心);Google DeepMind(谷歌DeepMind)
CytoSAE: Interpretable Cell Embeddings for Hematology
Institute of AI for Health, Helmholtz Munich Institute of Computational Biology, Helmholtz Munich Korea Advanced Institute of Science \& Technology Medical Department for Hematology;Oncology, Technical University Munich Munich Center for Machine Learning (MCML)
SegCompass: Exploring Interpretable Alignment with Sparse Autoencoders for Enhanced Reasoning Segmentation
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences(深圳先进技术研究院,中国科学院);Peng Cheng Laboratory(鹏城实验室);Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳));Meituan, Beijing(美团,北京);University of Chinese Academy of Sciences(中国科学院大学);College of Computer Science and Technology, Jilin University(吉林大学计算机科学与技术学院)