DreamCatcher：利用GPT嵌入通过fMRI揭示大脑的语言
DreamCatcher: Revealing the Language of the Brain with fMRI using GPT Embedding
AI总结：
本文提出DreamCatcher框架，利用GPT嵌入和fMRI数据生成字幕，以揭示视觉感知的神经机制，并在实验中展现出强大性能。
摘要：
人类大脑在视觉处理方面拥有卓越的能力，包括图像识别和场景总结。人们已经努力理解视觉大脑的认知能力，但对潜在机制的全面理解仍有待发现。脑解码技术的进步催生了诸如fMRI到图像重建等复杂方法，这对认知神经科学和医学影像具有重要意义。然而，fMRI到图像重建仍面临挑战，例如整合全局上下文和情境信息。在本文中，我们提出fMRI字幕生成，即基于fMRI数据生成字幕，以深入了解视觉感知的神经相关性。本研究提出了DreamCatcher，一种用于fMRI字幕生成的新型框架。DreamCatcher由表示空间编码器（RSE）和RevEmbedding解码器组成，分别将fMRI向量转换为潜在空间并生成字幕。我们通过可视化、数据集训练和受试者测试对框架进行了评估，展示了强大的性能。基于fMRI的字幕生成具有多种应用，包括理解神经机制、人机交互以及增强学习和训练过程。
英文摘要：
The human brain possesses remarkable abilities in visual processing, including image recognition and scene summarization. Efforts have been made to understand the cognitive capacities of the visual brain, but a comprehensive understanding of the underlying mechanisms still needs to be discovered. Advancements in brain decoding techniques have led to sophisticated approaches like fMRI-to-Image reconstruction, which has implications for cognitive neuroscience and medical imaging. However, challenges persist in fMRI-to-image reconstruction, such as incorporating global context and contextual information. In this article, we propose fMRI captioning, where captions are generated based on fMRI data to gain insight into the neural correlates of visual perception. This research presents DreamCatcher, a novel framework for fMRI captioning. DreamCatcher consists of the Representation Space Encoder (RSE) and the RevEmbedding Decoder, which transform fMRI vectors into a latent space and generate captions, respectively. We evaluated the framework through visualization, dataset training, and testing on subjects, demonstrating strong performance. fMRI-based captioning has diverse applications, including understanding neural mechanisms, Human-Computer Interaction, and enhancing learning and training processes.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
