发表机构
University of California Santa Barbara（加州大学圣塔芭芭拉分校）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究发现，在嵌套编码模型中仅重构条件预测器即可逆转多模态神经对比的符号，表明预测器构建属于实验设计，表征主张需匹配对照。
AI 中文摘要
基础模型特征越来越多地被用于探究神经活动所表征的信息，通常通过比较嵌套编码模型之间的预测增益来实现。我们表明，当仅重构条件预测器时，此类多模态对比的符号可能发生改变。利用自然场景数据集的fMRI数据、DINOv2视觉特征以及MS COCO标题和局部叙述的MPNet嵌入，在视觉额外预测贡献中，标题-叙述对比在将一个短标题与一个长叙述进行比较时偏向叙述（在Places区域为+0.012/+0.015），但在近似词数匹配后偏向标题（-0.031/-0.023）。这种转变发生在两个受试者的所有测量ROI中，且主要由仅语言预测的差异驱动。在几个ROI中，去除图像特定内容词身份后，类似的对比仍然存在。这些结果表明，嵌套神经对比本身并不能识别所表征的内容：预测器构建是实验设计的一部分，表征主张需要匹配的对照。
英文摘要
Foundation-model features are increasingly used to ask what information neural activity represents, often by comparing prediction gains between nested encoding models. We show that such multimodal contrasts can change sign when only the conditioning predictor is reconstructed. Using fMRI from the Natural Scenes Dataset, DINOv2 visual features, and MPNet embeddings of MS COCO captions and Localized Narratives, a caption-narrative contrast in the additional predictive contribution of vision favors narratives when one short caption is compared with a long narrative (+0.012/+0.015 in Places), but favors captions after approximate word-count matching (-0.031/-0.023). The shift occurs across every measured ROI in both subjects and is driven primarily by differences in language-only prediction. Comparable contrasts also survive removal of image-specific content-word identity in several ROIs. These results show that nested neural contrasts do not identify represented content by themselves: predictor construction is part of the experimental design, and matched controls are required for representational claims.
