arXiv 2602.047022026-02-04cs.SD

多头自注意力机制中的细粒度帧建模用于语音深度伪造检测

Fine-Grained Frame Modeling in Multi-head Self-Attention for Speech Deepfake Detection

College of Computing and Data Science, Nanyang Technological University, Singapore（计算与数据科学学院，南洋理工大学）

Tuan Dat Phuong, Duc-Tuan Truong, Long-Vu Hoang, Trang Nguyen Thi Thu

备注：: Accepted by ICASSP 2026

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出细粒度帧建模方法，通过多头投票和跨层细化模块提升语音深度伪造检测的准确性。

摘要：

基于Transformer的模型在语音深度伪造检测中表现出强大的性能，这主要归功于多头自注意力（MHSA）机制的有效性。MHSA提供帧级注意力分数，这特别有价值，因为深度伪造伪影通常出现在语音时间维度上的小而局部化的区域。这使得精细的帧建模对于准确检测细微的伪造线索至关重要。在本工作中，我们为基于MHSA的语音深度伪造检测提出了细粒度帧建模（FGFM），其中首先通过多头投票（MHV）模块选择最信息丰富的帧。这些选定的帧随后通过跨层细化（CLR）模块进行细化，以增强模型学习细微伪造线索的能力。实验结果表明，我们的方法优于基线模型，并在LA21、DF21和ITW数据集上分别实现了等错误率（EER）为0.90%、1.88%和6.64%。这些在多个基准测试中的一致改进突显了我们细粒度建模在稳健语音深度伪造检测中的有效性。

英文摘要：

Transformer-based models have shown strong performance in speech deepfake detection, largely due to the effectiveness of the multi-head self-attention (MHSA) mechanism. MHSA provides frame-level attention scores, which are particularly valuable because deepfake artifacts often occur in small, localized regions along the temporal dimension of speech. This makes fine-grained frame modeling essential for accurately detecting subtle spoofing cues. In this work, we propose fine-grained frame modeling (FGFM) for MHSA-based speech deepfake detection, where the most informative frames are first selected through a multi-head voting (MHV) module. These selected frames are then refined via a cross-layer refinement (CLR) module to enhance the model's ability to learn subtle spoofing cues. Experimental results demonstrate that our method outperforms the baseline model and achieves Equal Error Rate (EER) of 0.90%, 1.88%, and 6.64% on the LA21, DF21, and ITW datasets, respectively. These consistent improvements across multiple benchmarks highlight the effectiveness of our fine-grained modeling for robust speech deepfake detection.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

AI总结：

摘要：

英文摘要：

相关论文