arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.07311cs.CVcs.AI
收藏

LUCID-SAE:学习统一的视觉-语言稀疏代码以发现可解释的概念

LUCID-SAE: Learning Unified Vision-Language Sparse Codes for Interpretable Concept Discovery

Rutgers University(罗杰斯大学);Stanford University(斯坦福大学)

Difei Gu, Yunhe Gao, Gerasimos Chatzoudis, Zihan Dong, Guoning Zhang, Bangwei Guo, Yang Zhou, Mu Zhou, Dimitris Metaxas

AI总结:

LUCID-SAE通过统一视觉-语言稀疏编码方法,实现了跨模态的可解释概念发现与多模态表示的鲁棒性提升。

摘要:

稀疏自编码器(SAEs)提供了一条自然路径,以在不同的表示空间中实现可比的解释。然而,当前的SAEs是按模态训练的,生成的字典特征不是直接可理解的,其解释也无法跨领域转移。在本研究中,我们引入了LUCID(学习统一的视觉-语言稀疏代码以发现可解释的概念),一种统一的视觉-语言稀疏自编码器,它学习了一个共享的潜在字典,用于图像块和文本令牌的表示,同时保留了模态特定的细节容量。我们通过将共享代码与学习的最优传输匹配目标相结合,实现了特征对齐,而无需标注。LUCID产生了可解释的共享特征,支持块级 grounding,建立了跨模态神经元对应关系,并增强了在基于相似度的评估中的鲁棒性,以对抗概念聚类问题。利用对齐属性,我们开发了一种基于术语聚类的自动化字典解释流程,而无需手动观察。我们的分析表明,LUCID的共享特征捕捉了超越对象的多样语义类别,包括动作、属性和抽象概念,展示了全面的可解释多模态表示方法。

英文摘要:

Sparse autoencoders (SAEs) offer a natural path toward comparable explanations across different representation spaces. However, current SAEs are trained per modality, producing dictionaries whose features are not directly understandable and whose explanations do not transfer across domains. In this study, we introduce LUCID (Learning Unified vision-language sparse Codes for Interpretable concept Discovery), a unified vision-language sparse autoencoder that learns a shared latent dictionary for image patch and text token representations, while reserving private capacity for modality-specific details. We achieve feature alignment by coupling the shared codes with a learned optimal transport matching objective without the need of labeling. LUCID yields interpretable shared features that support patch-level grounding, establish cross-modal neuron correspondence, and enhance robustness against the concept clustering problem in similarity-based evaluation. Leveraging the alignment properties, we develop an automated dictionary interpretation pipeline based on term clustering without manual observations. Our analysis reveals that LUCID's shared features capture diverse semantic categories beyond objects, including actions, attributes, and abstract concepts, demonstrating a comprehensive approach to interpretable multimodal representations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑