arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.08412cs.SDcs.AIeess.AS
收藏

选择性注意系统(SAS):面向实时设备语音AI的设备地址语音检测

Selective Attention System (SAS): Device-Addressed Speech Detection for Real-Time On-Device Voice AI

Attention Labs

David Joohun Kim, Daniyal Anjum, Bonny Banerjee, Omar Abbasi

AI总结:

本文提出SAS系统,通过设备地址语音检测解决边缘部署中的语音识别问题,利用序列路由方法提升多说话人环境下的检测性能,实验显示融合音频视频信息可显著提升F1分数。

摘要:

我们研究了在预ASR边缘部署约束下设备地址语音检测的问题,系统必须在严格延迟和计算限制下决定是否在转录前转发音频。我们表明,在多说话人环境和时间上模糊的语音中,此任务更应建模为交互历史上的序列路由问题,而非语音局部分类任务。我们将其形式化为序列设备地址路由(SDAR),并提出选择性注意系统(SAS),一种在设备上实现该形式化的系统。在60小时的多说话人英语测试集上,主要音频-only配置实现F1=0.86(精确度=0.89,召回率=0.83);在有可选摄像头的情况下,音频+视频融合将F1提升至0.95(精确度=0.97,召回率=0.93)。移除因果交互历史(阶段3)使音频+视频配置的F1从0.95降至0.57±0.03,表明短时间交互历史在评估设置中携带重要决策相关信息。SAS在ARM Cortex-A类硬件上完全运行(<150毫秒延迟,<20MB占用空间)。所有结果均来自内部评估的专有数据集,主要以英语评估;5小时评估子集可用于独立验证(第8.8节)

英文摘要:

We study device-addressed speech detection under pre-ASR edge deployment constraints, where systems must decide whether to forward audio before transcription under strict latency and compute limits. We show that, in multi-speaker environments with temporally ambiguous utterances, this task is more effectively modelled as a sequential routing problem over interaction history than as an utterance-local classification task. We formalize this as Sequential Device-Addressed Routing (SDAR) and present the Selective Attention System (SAS), an on-device implementation that instantiates this formulation. On a held-out 60-hour multi-speaker English test set, the primary audio-only configuration achieves F1=0.86 (precision=0.89, recall=0.83); with an optional camera, audio+video fusion raises F1 to 0.95 (precision=0.97, recall=0.93). Removing causal interaction history (Stage~3) reduced F1 from 0.95 to 0.57+/-0.03 in the audio+video configuration under our evaluation protocol. Among the tested components, this was the largest observed ablation effect, indicating that short-horizon interaction history carries substantial decision-relevant information in the evaluated setting. SAS runs fully on-device on ARM Cortex-A class hardware (<150 ms latency, <20 MB footprint). All results are from internal evaluation on a proprietary dataset evaluated primarily in English; a 5-hour evaluation subset may be shared for independent verification (Section 8.8).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑