双视预测扩散：通过频谱-图像协同实现轻量语音增强
Dual-View Predictive Diffusion: Lightweight Speech Enhancement via Spectrogram-Image Synergy
AI总结：
DVPD通过频谱-图像协同实现轻量语音增强，利用双视预测扩散模型在保持高质量语音的同时显著降低计算复杂度。
摘要：
扩散模型最近在语音增强（SE）中设定了新的基准。然而，大多数现有的基于分数的模型将语音频谱图仅仅视为通用的2D图像，采用统一的处理方式，忽略了音频固有的结构稀疏性，导致不高效的频谱表示和不可接受的计算复杂性。为弥合这一差距，我们提出了DVPD，一个极其轻量的双视预测扩散模型，它独特地利用频谱图作为视觉纹理和物理频域表示的双重性质，在训练和推理阶段均如此。具体来说，在训练过程中，我们通过频率自适应非均匀压缩（FANC）编码器优化频谱利用，该编码器在保留关键低频谐波的同时修剪高频冗余。同时，我们引入了一个轻量级基于图像的频谱意识（LISA）模块，以最小的开销从视觉角度捕捉特征。在推理过程中，我们提出了一种无需训练的无损提升（TLB）策略，该策略利用相同的双视先验来提升生成质量，而无需任何额外的微调。在各种基准上的广泛实验表明，DVPD在仅需35%的参数和40%的推理MACs的情况下，实现了最先进的性能，相比最先进的轻量模型PGUSE。这些结果突显了DVPD在高保真语音质量与极端架构效率之间卓越的平衡能力。代码和音频示例可在匿名网站上获得：{https://anonymous.4open.science/r/dvpd_demo-E630}
英文摘要：
Diffusion models have recently set new benchmarks in Speech Enhancement (SE). However, most existing score-based models treat speech spectrograms merely as generic 2D images, applying uniform processing that ignores the intrinsic structural sparsity of audio, which results in inefficient spectral representation and prohibitive computational complexity. To bridge this gap, we propose DVPD, an extremely lightweight Dual-View Predictive Diffusion model, which uniquely exploits the dual nature of spectrograms as both visual textures and physical frequency-domain representations across both training and inference stages. Specifically, during training, we optimize spectral utilization via the Frequency-Adaptive Non-uniform Compression (FANC) encoder, which preserves critical low-frequency harmonics while pruning high-frequency redundancies. Simultaneously, we introduce a Lightweight Image-based Spectro-Awareness (LISA) module to capture features from a visual perspective with minimal overhead. During inference, we propose a Training-free Lossless Boost (TLB) strategy that leverages the same dual-view priors to refine generation quality without any additional fine-tuning. Extensive experiments across various benchmarks demonstrate that DVPD achieves state-of-the-art performance while requiring only 35% of the parameters and 40% of the inference MACs compared to SOTA lightweight model, PGUSE. These results highlight DVPD's superior ability to balance high-fidelity speech quality with extreme architectural efficiency. Code and audio samples are available at the anonymous website: {https://anonymous.4open.science/r/dvpd_demo-E630}
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
