PashtoTTS-Bench：低资源非拉丁文字文本转语音的自动化筛选
PashtoTTS-Bench: automated screening for low-resource non-Latin-script text-to-speech
AI总结：
针对低资源非拉丁文字TTS评估中单一ASR往返WER的不足，提出INSV报告框架及其自动化筛选子集INSV-A，并实例化为PashtoTTS-Bench基准，通过多指标评估多个TTS系统。
摘要：
对于低资源非拉丁文字语言，当文本转语音（TTS）评估依赖于单一的ASR往返词错误率（WER）时可能会失败。系统可能不产生音频、说出邻近语言、仅在ASR转录中保留目标文字脚本，或者对母语者来说听起来不自然。我们引入了INSV（可懂度、自然度、脚本保真度和验证）报告框架，将这些情况分开。本文报告了INSV-A，即自动化筛选子集：合成完成度、ASR WER/CER、转录脚本保真率和音频语言识别。原生MOS和语音标注已指定但未在此版本中声明。我们将INSV-A实例化为PashtoTTS-Bench，一个针对普什图语TTS的带日期基准。2026年4月至5月的运行评估了Edge GulNawaz、Edge Latifa、OmniVoice clone、OmniVoice auto和一个乌尔都语阴性对照，使用200个FLEURS和200个过滤后的Common Voice 24提示。在独立的omniASR_CTC_300M_v2下，OmniVoice auto的WER最低（FLEURS 24.1%，CV24 27.4%），其次是Edge GulNawaz（32.8%，39.5%）、Edge Latifa（35.6%，47.7%）和OmniVoice clone（45.4%，34.8%）。低于自然语音基线的WER反映了干净的合成音频，不应被解读为优于原生语音。Whisper Large V3在检查的普什图语TTS音频上返回0.0%的普什图语标签，而MMS-LID-4017和SpeechBrain VoxLingua107将普什图语输出与乌尔都语对照区分开。该版本提供了提供者元数据、每句分数、LID审计、失败日志和用于添加系统的脚本。
英文摘要：
Text-to-speech (TTS) evaluation for low-resource non-Latin-script languages can fail when it relies on a single ASR round-trip word error rate (WER). A system may produce no audio, speak a neighbouring language, preserve target script text only in an ASR transcript, or sound unnatural to native listeners. We introduce INSV (Intelligibility, Naturalness, Script fidelity, and Verification), a reporting framework that separates these cases. This paper reports INSV-A, the automated screening subset: synthesis completion, ASR WER/CER, transcript Script Fidelity Rate, and audio language identification. Native MOS and phonetic annotation are specified but not claimed in this release. We instantiate INSV-A as PashtoTTS-Bench, a dated benchmark for Pashto TTS. The April-May 2026 run evaluates Edge GulNawaz, Edge Latifa, OmniVoice clone, OmniVoice auto, and an Urdu negative control on 200 FLEURS and 200 filtered Common Voice 24 prompts. Under the independent omniASR_CTC_300M_v2, OmniVoice auto has the lowest WER (24.1% FLEURS, 27.4% CV24), followed by Edge GulNawaz (32.8%, 39.5%), Edge Latifa (35.6%, 47.7%), and OmniVoice clone (45.4%, 34.8%). WER below the natural-speech baseline reflects clean synthetic audio and should not be read as better than native speech. Whisper Large V3 returns 0.0% Pashto labels on checked Pashto TTS audio, while MMS-LID-4017 and SpeechBrain VoxLingua107 separate Pashto outputs from the Urdu control. The release provides provider metadata, per-sentence scores, LID audits, failure logs, and scripts for adding systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
