WASD：一个更广泛的主动说话人检测数据集
WASD: A Wilder Active Speaker Detection Dataset
摘要：
目前的主动说话人检测（ASD）模型在AVA-ActiveSpeaker（AVA）上取得了很好的效果，仅使用声音和面部特征。虽然这种方法适用于电影设置（AVA），但它不适合约束较少的条件。为了证明这一局限性，我们提出了一个Wilder Active Speaker Detection（WASD）数据集，通过针对当前ASD的两个关键组成部分：音频和面部，增加了难度。WASD分为5类，从最佳条件到监视设置，包含ASD的音频和面部数据战术损伤的增量挑战。我们选择了最先进的模型，并评估了它们在两组WASD中的性能：简单（合作设置）和困难（音频和/或面部被专门降级）。结果表明：1）AVA训练的模型在WASD Easy组中保持最先进的性能，而在Hard组中表现不佳，显示2）AVA和Easy数据之间的相似性;以及3）WASD中的训练不会将模型性能提高到AVA水平，特别是对于音频损伤和监视设置。这表明AVA没有为野生ASD准备模型，并且目前的方法不足以处理这种情况。建议的数据集还包含身体数据注释，为ASD提供新的来源，并可在https://github.com/Tiago-Roxo/WASD上获得。
英文摘要：
Current Active Speaker Detection (ASD) models achieve great results on AVA-ActiveSpeaker (AVA), using only sound and facial features. Although this approach is applicable in movie setups (AVA), it is not suited for less constrained conditions. To demonstrate this limitation, we propose a Wilder Active Speaker Detection (WASD) dataset, with increased difficulty by targeting the two key components of current ASD: audio and face. Grouped into 5 categories, ranging from optimal conditions to surveillance settings, WASD contains incremental challenges for ASD with tactical impairment of audio and face data. We select state-of-the-art models and assess their performance in two groups of WASD: Easy (cooperative settings) and Hard (audio and/or face are specifically degraded). The results show that: 1) AVA trained models maintain a state-of-the-art performance in WASD Easy group, while underperforming in the Hard one, showing the 2) similarity between AVA and Easy data; and 3) training in WASD does not improve models performance to AVA levels, particularly for audio impairment and surveillance settings. This shows that AVA does not prepare models for wild ASD and current approaches are subpar to deal with such conditions. The proposed dataset also contains body data annotations to provide a new source for ASD, and is available at https://github.com/Tiago-Roxo/WASD.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
