Uni-Neur2Img：通过扩散变换器实现神经信号引导的图像生成、编辑和风格化
Uni-Neur2Img: Unified Neural Signal-Guided Image Generation, Editing, and Stylization via Diffusion Transformers
AI总结：
Uni-Neur2Img通过扩散变换器实现神经信号引导的图像生成、编辑和风格化，提供统一且高效的解决方案。
摘要：
直接从神经信号生成或编辑图像在神经科学、视觉和脑机交互的交汇处具有巨大潜力。在本文中，我们提出了Uni-Neur2Img，一个统一的框架，用于神经信号驱动的图像生成和编辑。该框架引入了一个参数高效的LoRA基于神经信号注入模块，该模块独立处理每个条件信号作为可插拔组件，从而实现灵活的多模态条件设置而不改变基础模型参数。此外，我们采用因果注意力机制以满足条件生成任务的长序列建模需求。现有的神经驱动生成研究主要集中在文本模态作为条件或中间表示，导致对视觉模态作为直接条件信号的探索有限。为了填补这一研究空白，我们引入了EEG-Style数据集。我们对公开基准和自收集的神经信号数据集进行了全面评估：（1）在公共CVPR40数据集上进行EEG驱动的图像生成；（2）在公共Loongx数据集上进行神经信号引导的图像编辑，用于语义感知的局部修改；（3）在我们自收集的EEG-Style数据集上进行EEG驱动的风格迁移。广泛的实验结果表明，在保持低计算开销和强大的多模态扩展能力的同时，生成保真度、编辑一致性以及风格迁移质量均显著提升。因此，Uni-Neur2Img为连接神经信号和视觉内容生成提供了一个统一、高效且可扩展的解决方案。
英文摘要：
Generating or editing images directly from Neural signals has immense potential at the intersection of neuroscience, vision, and Brain-computer interaction. In this paper, We present Uni-Neur2Img, a unified framework for neural signal-driven image generation and editing. The framework introduces a parameter-efficient LoRA-based neural signal injection module that independently processes each conditioning signal as a pluggable component, facilitating flexible multi-modal conditioning without altering base model parameters. Additionally, we employ a causal attention mechanism accommodate the long-sequence modeling demands of conditional generation tasks. Existing neural-driven generation research predominantly focuses on textual modalities as conditions or intermediate representations, resulting in limited exploration of visual modalities as direct conditioning signals. To bridge this research gap, we introduce the EEG-Style dataset. We conduct comprehensive evaluations across public benchmarks and self-collected neural signal datasets: (1) EEG-driven image generation on the public CVPR40 dataset; (2) neural signal-guided image editing on the public Loongx dataset for semantic-aware local modifications; and (3) EEG-driven style transfer on our self-collected EEG-Style dataset. Extensive experimental results demonstrate significant improvements in generation fidelity, editing consistency, and style transfer quality while maintaining low computational overhead and strong scalability to additional modalities. Thus, Uni-Neur2Img offers a unified, efficient, and extensible solution for bridging neural signals and visual content generation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
