脑图像对齐的最优传输：揭示神经信息处理中的冗余与协同
Optimal Transport for Brain-Image Alignment: Unveiling Redundancy and Synergy in Neural Information Processing
AI总结：
本文提出基于最优传输的脑图像对齐方法，替代均方误差实现全局匹配，控制传输量抑制冗余，应用于脑字幕生成，在十项指标上超越先前最佳，并揭示大脑信息处理的冗余与协同。
摘要：
人工神经网络（ANNs）的设计灵感来源于人脑的结构，反过来，ANNs 也为解释和理解大脑信号提供了一种潜在手段。现有方法主要使用均方误差（MSE）将大脑信号与刺激信号对齐，该方法仅关注局部逐点对齐，忽略了全局匹配，导致大脑信号解码的解释粗糙且不准确。\n在本文中，我们通过最优传输（OT）解决这些问题，并从理论上证明为何 OT 比 MSE 能提供更有效的对齐策略。具体而言，我们在大脑体素嵌入和图像嵌入之间构建一个传输计划，实现更精确的匹配。通过控制传输量，我们减轻了冗余信息的影响。我们将对齐模型直接应用于脑字幕生成任务，将大脑信号而非图像输入到大语言模型（LLM）中。我们的方法在十项评估指标上达到了最先进的性能，在单受试者训练中平均超过之前最佳方法 6.11%，在跨受试者训练中平均超过 3.81%。此外，我们发现了几个与现有脑研究一致的深刻结论。通过区域掩蔽和数据降维可视化实验，我们揭示了大脑信息处理的冗余性和协同性。我们相信我们的方法为未来更精确地理解大脑信号铺平了道路。代码可在 https://github.com/NKUShaw/OT-Alignment4brain-to-image 获取。
英文摘要：
The design of artificial neural networks (ANNs) is inspired by the structure of the human brain, and in turn, ANNs offer a potential means to interpret and understand brain signals. Existing methods primarily align brain signals with stimulus signals using Mean Squared Error (MSE), which focuses only on local point-wise alignment and ignores global matching, leading to coarse interpretations and inaccuracies in brain signal decoding. In this paper, we address these issues through optimal transport (OT) and theoretically demonstrate why OT provides a more effective alignment strategy than MSE. Specifically, we construct a transport plan between brain voxel embeddings and image embeddings, enabling more precise matching. By controlling the amount of transport, we mitigate the influence of redundant information. We apply our alignment model directly to the Brain Captioning task by feeding brain signals into a large language model (LLM) instead of images. Our approach achieves state-of-the-art performance across ten evaluation metrics, surpassing the previous best method by an average of 6.11\% in single-subject training and 3.81\% in cross-subject training. Additionally, we have uncovered several insightful conclusions that align with existing brain research. We unveil the redundancy and synergy of brain information processing through region masking and data dimensionality reduction visualization experiments. We believe our approach paves the way for a more precise understanding of brain signals in the future. The code is available at https://github.com/NKUShaw/OT-Alignment4brain-to-image.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
