通过图增强的实例学习桥接归因与开放集检测在合成语音中
Bridging Attribution and Open-Set Detection using Graph-Augmented Instance Learning in Synthetic Speech
AI总结:
SIGNAL通过结合图神经网络和开放集检测方法,实现了合成语音来源的归因与开放集检测的统一。
摘要:
我们提出了一种统一的框架,不仅将合成语音归因于其来源,还能够检测在训练期间未遇到的合成器生成的语音。这需要方法超越简单的检测,以支持详细的取证分析和开放集泛化。为此,我们引入了SIGNAL,一种结合语音基础模型(SFMs)和基于图的建模以及开放集感知推理的混合框架。我们的框架整合了图神经网络(GNNs)和k-最近邻(KNN)分类器,使其能够捕捉语音之间的有意义关系,并识别不属于任何已知生成器的语音。它构建了一个查询条件的图,覆盖生成器类原型,使GNN能够在候选生成器之间进行推理,同时KNN分支通过基于置信度的阈值检测开放集。我们使用DiffSSD数据集评估SIGNAL,该数据集提供了真实语音和来自开源和商业扩散式TTS系统的合成音频的多样化混合。为了进一步评估泛化能力,我们还测试了SingFake基准。我们的结果表明,SIGNAL在两项任务上都表现出色,Mamba-based嵌入在其中表现尤为突出。据我们所知,这是首次将基于图的学习与开放集检测统一起来,用于追溯合成语音的起源。
英文摘要:
We propose a unified framework for not only attributing synthetic speech to its source but also for detecting speech generated by synthesizers that were not encountered during training. This requires methods that move beyond simple detection to support both detailed forensic analysis and open-set generalization. To address this, we introduce SIGNAL, a hybrid framework that combines speech foundation models (SFMs) with graph-based modeling and open-set-aware inference. Our framework integrates Graph Neural Networks (GNNs) and a k-Nearest Neighbor (KNN) classifier, allowing it to capture meaningful relationships between utterances and recognize speech that doesn`t belong to any known generator. It constructs a query-conditioned graph over generator class prototypes, enabling the GNN to reason over relationships among candidate generators, while the KNN branch supports open-set detection via confidence-based thresholding. We evaluate SIGNAL using the DiffSSD dataset, which offers a diverse mix of real speech and synthetic audio from both open-source and commercial diffusion-based TTS systems. To further assess generalization, we also test on the SingFake benchmark. Our results show that SIGNAL consistently improves performance across both tasks, with Mamba-based embeddings delivering especially strong results. To the best of our knowledge, this is the first study to unify graph-based learning and open-set detection for tracing synthetic speech back to its origin.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。