冻结的脑MRI基础模型是站点指纹
Site Is Decodable Before Pretraining: Negative Controls for Probing Frozen Brain-MRI Foundation Models
浏览论文内容
中文总结 AI 辅助
该研究发现冻结脑MRI基础模型的嵌入包含采集站点的指纹，该指纹可线性解码且与预训练无关，可通过特定方法移除，同时指出其对共享嵌入的影响及密集分割的代价，并发布了审计工具包。
中文摘要 AI 辅助
冻结的基础模型（FM）嵌入正日益被用作现成的脑MRI表示，其前提是这些嵌入捕捉了解剖结构。我们对它们实际编码的内容进行了审计，发现采集站点是该表示的一个大的内在组成部分。在两个独立队列（ABIDE-I、ABIDE-II）、三个冻结的3D编码器（脑预训练、CT预训练和随机初始化）以及所有网络深度下，在深层站点可被线性解码，平衡准确率约为0.9，超过了各层所有临床或人口统计学变量（性别、年龄、自闭症诊断）的可解码性。该效应是内在的而非学习得到的：随机初始化的编码器在两个队列以及三个架构家族（Swin、ViT、ResNet）上已经是约0.9的站点分类器，且在不使用编码器的情况下，直接从下采样的原始图像可约0.95地解码出站点，因此该指纹反映的是任何编码器都会保留的低级图像统计信息，而非预训练的产物。对测得的人口协变量进行残差处理后，站点可解码性基本保持不变，表明这是由采集而非人口驱动的效应。非线性探针与线性探针匹配，因此该指纹完全可线性访问。该站点子空间可通过迭代零空间投影或ComBat事后移除（站点可解码性从0.94降至0.07/0.00），这对共享或联邦嵌入而言是站点归因方面的问题；但对于密集分割而言，这种移除并非无代价，因为站点与解剖结构占据纠缠的线性子空间（匹配秩的随机方向投影对Dice系数无影响，而移除站点子空间具有破坏性）。我们建议对冻结的脑MRI FM进行站点审计使用，并发布了一个开放的审计工具包。
英文摘要
Brain foundation models are often tested with a probe. The model is frozen, a simple classifier is trained on its output, and the classifier's accuracy is taken as a sign of what pretraining learned. We show that this reading can be wrong unless two controls are reported with it. We probed three frozen 3-D brain-MRI models at five depths, on two independent multi-site cohorts (ABIDE-I and ABIDE-II). The site where a scan was acquired could be predicted at about 0.9, on a scale where 0 is chance and 1 is perfect. No clinical or demographic target came close. An untrained copy of the same model predicted site just as well on both cohorts. Untrained models of three architectures, each built with three random seeds, did the same on ABIDE-I. The raw image, shrunk to 12x12x12 voxels and given to no model at all, predicted site at 0.95. Pretraining did not create the site signal. It was already in the scans. We also looked for any clinical target on which pretraining beats an untrained model, and found no reliable case. Adjusting for age, sex and diagnosis left site prediction almost unchanged, and so did a 4.6-fold increase in the number of voxels. Resting-state fMRI from the same people showed the same pattern. Site was the most predictable attribute of brain connectivity (0.81), and it survived five untrained layers while age did not. We recommend reporting two controls with every probe of a brain foundation model: running the probe again on the untrained model, and on the raw input. We release the code for both.
发表机构
- University of British Columbia（不列颠哥伦比亚大学）
机构由 AI 辅助整理，请以论文原文为准。
