arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.19949eess.AS
收藏

Indic-CodecFake meets SATYAM:迈向检测印度语言神经音频编解码器合成语音深度伪造

Indic-CodecFake meets SATYAM: Towards Detecting Neural Audio Codec Synthesized Speech Deepfakes in Indic Languages

UPES, India(乌塔拉克德联合教育研究所);Veer Bahadur Singh Purvanchal University, India(维尔·巴哈杜尔·辛格普尔万恰尔大学);IIIT-Delhi, India(德里印度信息技术学院)

Girish, Mohd Mujtaba Akhtar, Orchid Chetia Phukan, Arun Balaji Buduru

AI总结:

本文提出Indic-CodecFake数据集,用于检测印度语言中神经音频编解码器合成语音深度伪造,发现现有检测器在印度语音上表现不佳,并提出SATYAM模型以提升性能。

摘要:

音频大语言模型(ALMs)的快速发展推动了神经音频编解码器(NACs)的应用,导致高度逼真的语音深度伪造(CodecFakes, CFs)的出现。现有研究主要关注英语或中文,忽略了印度语言的脆弱性。本文引入Indic-CodecFake(ICF)数据集,包含多种印度语言的真实语音和NAC合成语音,通过IndicSUPERB生成。实验表明,基于英语数据集的SOTA CF检测器无法泛化到ICF,凸显了印度语音的语音多样性与语调变化挑战。进一步评估了SOTA ALMs在零样本设置下的表现,发现其性能不佳。为此,提出SATYAM,一种针对印度语言CF检测的超几何ALM,结合Whisper的语义表示和TRILLsson的语调表示,通过Bhattacharya距离在超几何空间中融合,并执行相同对齐程序。双阶段融合框架使SATYAM能够有效建模语音内部(语义-语调)和跨模态(语音-文本)的层次关系。大量评估显示,SATYAM在ICF基准上优于端到端和ALM基线。

英文摘要:

The rapid advancement of Audio Large Language Models (ALMs), driven by Neural Audio Codecs (NACs), has led to the emergence of highly realistic speech deepfakes, commonly referred to as CodecFakes (CFs). Consequently, CF detection has attracted increasing attention from the research community. However, existing studies predominantly focus on English or Chinese, leaving the vulnerability of Indic languages largely unexplored. To bridge this gap, we introduce Indic-CodecFake (ICF) dataset, the first large-scale benchmark comprising real and NAC-synthesized speech across multiple Indic languages, diverse speaker profiles, and multiple NAC types. We use IndicSUPERB as the real speech corpus for generation of ICF dataset. Our experiments demonstrate that state-of-the-art (SOTA) CF detectors trained on English-centric datasets fail to generalize to ICF, underscoring the challenges posed by phonetic diversity and prosodic variability in Indic speech. Further, we present systematic evaluation of SOTA ALMs in a zero-shot setting on ICF dataset. We evaluate these ALMs as they have shown effectiveness for different speech tasks. However, our findings reveal that current ALMs exhibit consistently poor performance. To address this, we propose SATYAM, a novel hyperbolic ALM tailored for CF detection in Indic languages. SATYAM integrates semantic representations from Whisper and prosodic representations from TRILLsson using through Bhattacharya distance in hyperbolic space and subsequently performs the same alignment procedure between the fused speech representation and an input conditioning prompt. This dual-stage fusion framework enables SATYAM to effectively model hierarchical relationships both within speech (semantic-prosodic) and across modalities (speech-text). Extensive evaluations show that SATYAM consistently outperforms competitive end-to-end and ALM-based baselines on the ICF benchmark.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑