基于形状-语义GAN的脑活动感知图像重建
Reconstructing Perceptive Images from Brain Activity by Shape-Semantic GAN
摘要:
从功能磁共振成像记录中重建视觉图像是神经科学中一个引人注目的研究领域,并提供了一种潜在的大脑阅读技术。挑战在于大脑中的视觉编码是高度复杂的,并且没有完全揭示。受视觉特征在大脑皮层的分层表征理论的启发,我们提出将复杂的视觉信号分解为多个层次的分量,并对每个分量分别进行解码。具体来说,我们解码的形状和语义表示从较低和较高的视觉皮层分别,并合并的形状和语义信息的图像生成对抗网络(形状-语义GAN)。这种“分而治之”的策略可以更准确地捕捉视觉信息。实验结果表明,形状语义GAN提高了重建相似度和图像质量,达到了最新的图像重建性能。
英文摘要:
Reconstructing seeing images from fMRI recordings is an absorbing research area in neuroscience and provides a potential brain-reading technology. The challenge lies in that visual encoding in brain is highly complex and not fully revealed. Inspired by the theory that visual features are hierarchically represented in cortex, we propose to break the complex visual signals into multi-level components and decode each component separately. Specifically, we decode shape and semantic representations from the lower and higher visual cortex respectively, and merge the shape and semantic information to images by a generative adversarial network (Shape-Semantic GAN). This 'divide and conquer' strategy captures visual information more accurately. Experiments demonstrate that Shape-Semantic GAN improves the reconstruction similarity and image quality, and achieves the state-of-the-art image reconstruction performance.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。