DialogueSidon: 从真实场景对话音频中恢复全双工对话轨道
DialogueSidon: Recovering Full-Duplex Dialogue Tracks from In-the-Wild Dialogue Audio
AI总结：
DialogueSidon通过联合恢复与分离降质单通道对话音频，提升识别与分离质量，并实现更快推理。
摘要：
全双工对话音频，即每个说话人单独录制在不同轨道上，是口语对话研究的重要资源，但难以大规模收集。大多数真实场景双说话人对话仅以降质的单通道混合物形式存在，不适合需要清洁说话人信号的系统。我们提出DialogueSidon，一种联合恢复与分离降质单通道双说话人对话音频的模型。DialogueSidon结合了基于变分自编码器（VAE）的操作于语音自监督学习（SSL）模型特征，将SSL模型特征压缩到紧凑的潜在空间，与基于扩散的潜在预测器相结合，从降质混合物中恢复说话人特定的潜在表示。在英语、多语言和真实场景对话数据集上的实验表明，DialogueSidon在基线模型上显著提高了可识别性和分离质量，同时实现了更快的推理。
英文摘要：
Full-duplex dialogue audio, in which each speaker is recorded on a separate track, is an important resource for spoken dialogue research, but is difficult to collect at scale. Most in-the-wild two-speaker dialogue is available only as degraded monaural mixtures, making it unsuitable for systems requiring clean speaker-wise signals. We propose DialogueSidon, a model for joint restoration and separation of degraded monaural two-speaker dialogue audio. DialogueSidon combines a variational autoencoder (VAE) operates on the speech self-supervised learning (SSL) model feature, which compresses SSL model features into a compact latent space, with a diffusion-based latent predictor that recovers speaker-wise latent representations from the degraded mixture. Experiments on English, multilingual, and in-the-wild dialogue datasets show that DialogueSidon substantially improves intelligibility and separation quality over a baseline, while also achieving much faster inference.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
