Whisper 具有内部词对齐器
Whisper Has an Internal Word Aligner
AI总结：
该研究提出一种无监督方法，通过过滤 Whisper 在字符教师强制下的注意力头来提取词对齐，无需额外训练，在 20-100 ms 严格容差下比先前工作更准确。
摘要：
人们对从强大的自动语音识别器（特别是 Whisper）中获取准确的词级时间戳的兴趣日益浓厚。现有方法要么需要额外训练，要么效果不佳。先前工作中的评估也相对宽松，通常使用超过 200 ms 的容差。在这项工作中，我们发现了 Whisper 中的注意力头能够捕获准确的词对齐，并且与不能捕获词对齐的注意力头明显不同。此外，我们发现使用字符比使用词元产生更精细和更准确的对齐。基于这些发现，我们提出了一种无监督方法，通过在用字符对 Whisper 进行教师强制的同时过滤注意力头来提取词对齐。我们的方法不仅不需要训练，而且在 20 ms 到 100 ms 的更严格容差下，产生的词对齐比先前的工作更准确。
英文摘要：
There is an increasing interest in obtaining accurate word-level timestamps from strong automatic speech recognizers, in particular Whisper. Existing approaches either require additional training or are simply not competitive. The evaluation in prior work is also relatively loose, typically using a tolerance of more than 200 ms. In this work, we discover attention heads in Whisper that capture accurate word alignments and are distinctively different from those that do not. Moreover, we find that using characters produces finer and more accurate alignments than using wordpieces. Based on these findings, we propose an unsupervised approach to extracting word alignments by filtering attention heads while teacher forcing Whisper with characters. Our approach not only does not require training but also produces word alignments that are more accurate than prior work under a stricter tolerance between 20 ms and 100 ms.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
