STAM-ASR：带记忆的说话人-时间锚定用于多说话人ASR
STAM-ASR: Speaker-Temporal Anchoring with Memory for Multi-Speaker ASR
- Qatar Computing Research Institute (QCRI)（卡塔尔计算研究所）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
STAM-ASR通过说话人-时间锚定与记忆机制，在不依赖外部日志和语音分离的情况下扩展预训练AudioLLM，实现多说话人ASR，并在多个基准上验证了其有效性。
AI中文摘要：
自然对话使得语音识别和说话人归属对ASR系统都构成挑战，因为说话人会轮流发言、重叠发言并随时间重新出现。我们提出STAM-ASR，即带记忆的说话人-时间锚定，这是一个轻量级框架，扩展了已预训练的AudioLLM以用于多说话人ASR。在不依赖外部说话人日志系统的情况下，STAM-ASR直接从AudioLLM的中间特征中学习说话人活动和说话人感知表示。因此，它提供了明确的谁和何时线索来调制AudioLLM的语义表示，而无需显式的语音分离。STAM-ASR进一步维护固定大小的说话人和对话记忆，以在轮流发言之间携带互补的上下文。我们在AMI、ICSI、LibriCSS和NOTSOFAR-1上评估了STAM-ASR，覆盖近讲、远场、重叠和跨域条件。我们报告的结果表明，说话人-时间条件化和记忆提供了互补的益处，而参考与预测说话人活动之间的差距表明，稳健的说话人跟踪是一个关键剩余挑战。
英文摘要：
Natural conversations make both speech recognition and speaker attribution challenging for ASR, as speakers take turns, overlap, and reappear over time. We propose STAM-ASR, Speaker-Temporal Anchoring with Memory, a lightweight framework that extends an already pretrained AudioLLM for multi-speaker ASR. Without relying on an external diarization system, STAM-ASR learns speaker activity and speaker-aware representations directly from intermediate AudioLLM features. Hence providing explicit who and when cues to modulate the AudioLLM's semantic representation without explicit speech separation. STAM-ASR further maintains fixed-size speaker and conversational memories to carry complementary context across turns. We evaluate STAM-ASR on AMI, ICSI, LibriCSS, and NOTSOFAR-1 across close-talk, far-field, overlapping, and cross-domain conditions. Our reported results shows that speaker-temporal conditioning and memory provide complementary benefits, while the gap between reference and predicted speaker activity identifies robust speaker tracking as a key remaining challenge.
