发表机构
CVML Lab（CVML实验室）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
RAG-Audio通过检索真实音频样本初始化生成器采样轨迹，缓解脑电到音频生成的先验主导问题，在Brain2Music数据集上提升了刺激识别准确率并大幅降低Fréchet音频距离。
AI 中文摘要
脑电信号到音频的重建受“先验主导”问题限制：当预训练生成器以微弱神经信号为条件时，会生成逼真但与刺激不符的音频。本文提出RAG-Audio，它将fMRI解码成语义音频嵌入，检索匹配的真实音频样本，并以该样本初始化冻结生成器的采样轨迹，同时保留解码的嵌入作为条件。在Brain2Music数据集上，RAG-Audio将10种刺激识别准确率从直接生成的0.14-0.18（接近0.10的随机水平）提升至0.40-0.43，与检索方法相当；还将AudioLDM的Fréchet音频距离降低约一个数量级，从13.49降至1.25。RAG-Audio在识别任务上接近最近邻检索的性能，同时保持生成特性；其更高的FAD是可预期的，因为检索会直接回放真实音频。作为对照，缺乏可初始化潜轨迹的自回归模型未显示可比提升，表明改进源于轨迹初始化。这些结果说明，检索引导的初始化可缓解脑电到音频生成中的先验主导问题。
英文摘要
Brain-to-audio reconstruction is limited by \emph{prior domination}: when a pretrained generator is conditioned on a weak neural signal, it produces realistic but stimulus-inaccurate audio. We introduce RAG-Audio, which decodes fMRI into a semantic audio embedding, retrieves a matching real-audio exemplar, and initializes the frozen generator's sampling trajectory from that exemplar while retaining the decoded embedding as conditioning. On Brain2Music, RAG-Audio improves 10-way stimulus identification from $0.14$--$0.18$ for direct generation, near the $0.10$ chance level, to $0.40$--$0.43$, comparable to retrieval. It also reduces Fréchet Audio Distance by roughly an order of magnitude, from $13.49$ to $1.25$ for AudioLDM. RAG-Audio approaches nearest-neighbor retrieval in identification while remaining generative; its higher FAD is expected because retrieval directly replays real audio. An autoregressive negative control, which lacks an initializable latent trajectory, shows no comparable gain, attributing the improvement to trajectory initialization. These results suggest that retrieval-guided initialization can mitigate prior domination in brain-to-audio generation.
