IRAF：面向噪声鲁棒的端到端全双工口语对话系统的抗干扰自适应融合
IRAF: Interference-Resilient Adaptive Fusion for Noise-Robust End-to-End Full-Duplex Spoken Dialogue Systems
AI总结：
提出IRAF模块，通过逐帧预测可靠性门控来调节用户音频对LLM的贡献，提升全双工对话系统在干扰说话人环境下的响应质量和交互稳定性。
摘要：
全双工口语对话模型允许语音代理同时听和说，实现具有实时重叠的自然交互。然而，联合编码用户和代理流的端到端双通道模型在现实声学环境中可能会退化：干扰说话人泄漏到用户麦克风中，会被编码为用户查询的一部分，破坏LLM的条件，导致不稳定的轮流说话和响应质量下降。我们提出抗干扰自适应融合（IRAF），一个轻量级、流兼容的模块，逐帧调节用户音频对LLM的贡献。IRAF从目标说话人和用户音频嵌入中预测一个标量可靠性门控，并在与代理嵌入融合之前重新缩放用户表示。在MS-MARCO和InstructS2S-200K上的实验表明，在干扰说话人条件下，响应质量和全双工交互获得一致提升。
英文摘要：
Full-duplex spoken dialogue models allow voice agents to listen and speak concurrently, enabling natural interaction with real-time overlap. However, end-to-end dual-channel models that jointly encode user and agent streams may degrade in realistic acoustic environments: interfering speakers leaking into the user microphone can be encoded as part of the user query, corrupting the LLM's conditioning and causing unstable turn-taking and reduced response quality. We propose Interference-Resilient Adaptive Fusion (IRAF), a lightweight, streaming-compatible module that modulates the contribution of user audio to the LLM frame by frame. IRAF predicts a scalar reliability gate from target-speaker and user audio embeddings and rescales user representations before fusion with agent embeddings. Experiments on MS-MARCO and InstructS2S-200K show consistent gains in response quality and full-duplex interaction under interfering-speaker conditions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
