NeuralDiffuser：用于fMRI视觉重建的神经科学启发扩散引导
NeuralDiffuser: Neuroscience-inspired Diffusion Guidance for fMRI Visual Reconstruction
AI总结：
本文提出NeuralDiffuser，通过初级视觉特征的梯度引导弥补稳定扩散自下而上细节感知的不足，并设计一致性引导策略，在NSD上显著提升fMRI视觉重建的细节保真度与可重复性。
摘要：
从功能性磁共振成像（fMRI）重建视觉刺激，能够实现对大脑活动的细粒度检索。然而，准确重建结构、背景、纹理、颜色等多种细节仍然具有挑战性。稳定扩散模型不可避免地会导致重建图像出现变异，即使在相同条件下也是如此。为应对这一挑战，我们首先揭示了扩散方法的神经科学视角：这些方法主要利用来自大规模图像数据集的预训练知识进行自上而下的创造，但往往缺乏由细节驱动的自下而上感知，从而导致忠实细节的丢失。本文提出NeuralDiffuser，其引入初级视觉特征引导，以梯度形式提供细节线索。这种对扩散模型自下而上过程的扩展，在重建视觉刺激时同时实现语义连贯性和细节保真度。此外，我们为重建任务开发了一种新颖的引导策略，确保重复输出与原始图像保持一致，而不是与各种不同输出保持一致。在Natural Senses Dataset（NSD）上的大量实验结果，通过与基线方法和最先进方法进行横向比较，并开展纵向消融研究，从定性和定量两方面证明了NeuralDiffuser的先进性。
英文摘要：
Reconstructing visual stimuli from functional Magnetic Resonance Imaging fMRI enables fine-grained retrieval of brain activity. However, the accurate reconstruction of diverse details, including structure, background, texture, color, and more, remains challenging. The stable diffusion models inevitably result in the variability of reconstructed images, even under identical conditions. To address this challenge, we first uncover the neuroscientific perspective of diffusion methods, which primarily involve top-down creation using pre-trained knowledge from extensive image datasets, but tend to lack detail-driven bottom-up perception, leading to a loss of faithful details. In this paper, we propose NeuralDiffuser, which incorporates primary visual feature guidance to provide detailed cues in the form of gradients. This extension of the bottom-up process for diffusion models achieves both semantic coherence and detail fidelity when reconstructing visual stimuli. Furthermore, we have developed a novel guidance strategy for reconstruction tasks that ensures the consistency of repeated outputs with original images rather than with various outputs. Extensive experimental results on the Natural Senses Dataset (NSD) qualitatively and quantitatively demonstrate the advancement of NeuralDiffuser by comparing it against baseline and state-of-the-art methods horizontally, as well as conducting longitudinal ablation studies.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
