AnyBand：基于频率感知上下文频谱补全的统一多带宽语音扩展
AnyBand: Unified Multi-Bandwidth Speech Extension via Frequency-Aware In-Context Spectral Infilling
浏览论文内容
中文总结 AI 辅助
该研究提出统一多带宽语音扩展框架AnyBand，采用频率感知扩散Transformer等技术，可在连续输入带宽范围实现高质量语音带宽扩展，性能优于现有基准。
中文摘要 AI 辅助
带宽扩展（BWE）旨在从限带语音中恢复缺失的高频内容。现有方法通常将BWE表述为固定或预定义的带宽转换问题，当输入带宽变化时，可能需要特定截止频率的模型或重新训练。这一假设限制了它们在实际场景中的适用性，实际场景中语音可能以不同的截止频率输入。我们提出AnyBand，这是一个将带宽扩展重新表述为上下文频谱补全的统一BWE框架。受基于提示的零样本语音生成启发，AnyBand将高频生成条件化于观测到的低频频谱，使用可用频段作为频域提示，传达内容、说话人、韵律和频谱包络线索。该表述使单个模型能够在连续范围的输入带宽上执行截止频率条件生成。AnyBand通过缺失频段条件流匹配和在连续采样的截止频率上的易到平衡截止课程进行训练。为更好地利用频谱提示，我们引入了频率感知扩散Transformer，它对跨频交互和长程时间依赖进行建模，随后是受物理启发的多视图对抗细化阶段，以增强频谱真实性、包络一致性和谐波一致性。在多个数据集和带宽设置上的实验表明，AnyBand在现有基准上持续改善了频谱重建，同时在标准和不规则输入截止频率上实现了有竞争力的感知质量。音频样本可用。
英文摘要
Bandwidth extension (BWE) aims to recover missing high-frequency content from band-limited speech. Existing methods often formulate BWE as a fixed or predefined bandwidth conversion problem, potentially requiring cutoff-specific models or retraining when the input bandwidth changes. This assumption limits their applicability to practical scenarios where speech may arrive with diverse cutoff frequencies. We propose AnyBand, a unified BWE framework that recasts bandwidth extension as in-context spectral infilling. Motivated by prompt-based zero-shot speech generation, AnyBand conditions high-frequency generation on the observed low-frequency spectrum, using the available band as a frequency-domain prompt that conveys content, speaker, prosodic, and spectral-envelope cues. This formulation enables a single model to perform cutoff-conditioned generation over a continuous range of input bandwidths. AnyBand is trained with missing-band conditional flow matching and an Easy-to-Balanced cutoff curriculum over continuously sampled cutoff frequencies. To better exploit the spectral prompt, we introduce a frequency-aware Diffusion Transformer that models cross-frequency interactions and long-range temporal dependencies, followed by a physically motivated multi-view adversarial refinement stage to enhance spectral realism, envelope coherence, and harmonic consistency. Experiments on multiple datasets and bandwidth settings show that AnyBand consistently improves spectral reconstruction over existing baselines while achieving competitive perceptual quality across both standard and irregular input cutoffs. Audio samples are available.
