一种基于Transformer的音频亲属关系验证新方法及一个新的非受控普通话亲属语音数据集
A New Transformer-Based Approach for Audio-Based Kinship Verification and a New Uncontrolled Mandarin Kinship Speech Dataset
查看机构详情
- Imperial College London（帝国理工学院）
- TUM University Hospital（慕尼黑工业大学附属医院）
机构由 AI 辅助整理，请以论文原文为准。
浏览论文内容
中文总结 AI 辅助
本文提出基于Transformer的音频亲属关系验证新架构CONVTRAP-TN，并构建非受控普通话数据集ARKIN，通过基线实验证明现有方法跨数据集不鲁棒。
中文摘要 AI 辅助
亲属关系验证是一项涉及确定两个人是否具有一级亲属关系的任务。为了解决这一任务，我们提出了CONVTRAP-TN，一种用于基于音频的亲属关系验证的新架构，并对所提出的模型进行了消融研究。据我们所知，我们是首次将成功的Transformer架构应用于基于音频的亲属关系验证任务。此外，我们还收集了一个自定义语音数据集ARKIN，该数据集准确反映了日常录音条件。我们这样做是因为目前只有少数带有亲属标签的语音数据集，这些数据集要么从互联网上获取极其嘈杂的野外数据，要么指示说话者在特定环境中录音。这些设置无法反映用户在不受约束的条件下在个人设备上录音的真实场景。此外，我们在收集的数据集上进行了一系列初步基线实验，包括说话人验证和识别、语音识别、年龄估计和亲属关系验证，以及跨数据集亲属关系验证实验，以表明现有方法在不同数据集之间不具有鲁棒性。
英文摘要
Kinship verification is a task involving determining whether two individuals share a first-order kin relation. To tackle this task, we propose CONVTRAP-TN, a new architecture for audio-based kinship verification, and conduct an ablation study on the proposed model. To the best of our knowledge, we are the first to apply the successful transformer architecture to the task of audio-based kinship verification. Furthermore, we also collect a custom speech dataset, ARKIN, which accurately reflects everyday recording conditions. We do this because only a few speech datasets with kinship labels currently exist, all of which either source extremely noisy in-the-wild data from the internet, or instruct speakers to record in specific environments. These settings fail to reflect real-world scenarios where users record on personal devices under unrestrained conditions. Additionally, we perform a series of preliminary baseline experiments on the collected dataset, including speaker verification and recognition, speech recognition, age estimation, and kinship verification, as well as cross-dataset kinship verification experiments to show that existing methods are not robust across datasets.
