Randomized-MLP 正则化提升 DINOv2 的领域适应性与可解释性
Randomized-MLP Regularization Improves Domain Adaptation and Interpretability in DINOv2
AI总结：
本文提出 Randomized-MLP 正则化方法，通过在微调 DINOv2 时引入对比学习约束，在医学与自然图像任务中提升或保持性能，并增强注意力图可解释性。
摘要：
视觉 Transformer（ViT），例如 DINOv2，在跨领域任务中取得了强大性能，但常常重新利用低信息量的图像块 token，从而降低注意力图和特征图的可解释性。这一挑战在医学影像中尤为明显，因为领域偏移会同时损害性能和透明度。本文提出 Randomized-MLP（RMLP）正则化，这是一种基于对比学习的方法，旨在促使模型学习更具语义对齐性的表示。我们在对 DINOv2 进行医学图像和自然图像模态的微调时使用 RMLP，结果表明它能够提升或保持下游性能，同时生成更具可解释性的注意力图。我们还对 RMLP 进行了数学分析，深入揭示其在增强基于 ViT 的模型以及推进对比学习理解方面的作用。
英文摘要：
Vision Transformers (ViTs), such as DINOv2, achieve strong performance across domains but often repurpose low-informative patch tokens in ways that reduce the interpretability of attention and feature maps. This challenge is especially evident in medical imaging, where domain shifts can degrade both performance and transparency. In this paper, we introduce Randomized-MLP (RMLP) regularization, a contrastive learning-based method that encourages more semantically aligned representations. We use RMLPs when fine-tuning DINOv2 to both medical and natural image modalities, showing that it improves or maintains downstream performance while producing more interpretable attention maps. We also provide a mathematical analysis of RMLPs, offering insights into its role in enhancing ViT-based models and advancing our understanding of contrastive learning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
