OpenSVBench：诊断跨部署场景的说话人验证泛化能力
OpenSVBench: Diagnosing Speaker Verification Generalization Across Deployment Scenarios
AI总结：
本文提出OpenSVBench跨场景说话人验证基准，评估18种主流SV模型，发现系统存在明显跨场景泛化差距，为SV系统部署泛化诊断提供了工具。
摘要：
说话人验证（SV）系统在标准公开协议上已实现极低的错误率，但这些结果无法完整反映部署泛化能力。VoxCeleb式评估仍是英文野外媒体语音的有用基准，特定场景的评估与挑战也颇具价值，不过两种设置均无法直接体现同一系统在不同部署场景下的可靠性。本文提出OpenSVBench，这是一个用于跨场景SV评估的公开基准：其固定公开核心包含26个试集，分为11个场景组，共915万次验证试次，含冻结的目标/非目标密钥及用于场景级报告的元数据；核心之外的在线排行榜兼具诊断评估接口功能，而非仅为排名列表，它接收新系统的分数包，通过冻结密钥与通用评分规则进行评估，并报告试集细分、场景组指标、宏平均等错误率（macro EER）、最小检测成本函数（minDCF）、模型特征及场景间对比。本文评估了18个公开检查点，涵盖x-vector、ECAPA-TDNN、CAMPPlus、ResNet及ERes变体、ReDimNet2、SAM-ResNet和预训练Transformer说话人方案，结果显示存在明显的跨场景差距：宏平均等错误率相近的系统可能有不同的场景特征，某一场景的优异表现无法保证在另一场景也表现良好，且同场景试集仍可暴露源数据与协议的差距。
英文摘要：
Speaker verification (SV) systems now achieve very low error rates on standard public protocols, but those results do not fully describe deployment generalization. VoxCeleb-style evaluations remain useful anchors for English in-the-wild media speech. Scenario-specific evaluations and challenges are also valuable. However, neither setting directly shows whether the same system remains reliable across diverse deployment scenarios. We introduce OpenSVBench, a public benchmark for cross-scenario SV evaluation. Its fixed public core contains 26 trial sets organized into 11 scenario groups, with 9.15M verification trials, frozen target/non-target keys, and metadata for scenario-wise reporting. Around this core, the online leaderboard acts as a diagnostic evaluation interface rather than only a ranked list. It accepts score packages from new systems, evaluates them with frozen keys and common scoring, and reports trial-set breakdowns, scenario-group metrics, macro EER/minDCF, model-wise profiles, and scenario-wise comparisons. We evaluate 18 public checkpoints spanning x-vector, ECAPA-TDNN, CAMPPlus, ResNet and ERes variants, ReDimNet2, SAM-ResNet, and pretrained Transformer speaker recipes. The results show clear cross-scenario gaps: systems with similar macro EER can have different scenario profiles, strong performance in one scenario does not guarantee strong performance in another, and same-scenario trial sets can still expose source and protocol gaps.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
