使用开源工具包和公开可用数据复现Whisper风格训练
Reproducing Whisper-Style Training Using an Open-Source Toolkit and Publicly Available Data
AI总结：
为解决OpenAI Whisper开发流程未公开的问题，本工作使用开源工具包和公开数据复现了Whisper风格训练，提出OWSM模型，支持更多翻译方向且训练更高效，并公开全部脚本与模型以促进开放科学。
摘要：
在大规模数据上预训练语音模型已取得显著成功。OpenAI Whisper是一个在680k小时监督语音数据上训练的多语言多任务模型。即使在零样本设置下，它也能很好地泛化到各种语音识别和翻译基准。然而，开发此类模型的完整流程（从数据收集到训练）并未公开，这使得研究人员难以进一步提升其性能并解决与训练相关的问题，如效率、鲁棒性、公平性和偏见。本工作提出了Open Whisper-style Speech Model (OWSM)，它使用开源工具包和公开可用数据复现了Whisper风格训练。OWSM甚至支持更多翻译方向，并且训练效率更高。我们将公开发布用于数据准备、训练、推理和评分的所有脚本，以及预训练模型和训练日志，以促进开放科学。
英文摘要：
Pre-training speech models on large volumes of data has achieved remarkable success. OpenAI Whisper is a multilingual multitask model trained on 680k hours of supervised speech data. It generalizes well to various speech recognition and translation benchmarks even in a zero-shot setup. However, the full pipeline for developing such models (from data collection to training) is not publicly accessible, which makes it difficult for researchers to further improve its performance and address training-related issues such as efficiency, robustness, fairness, and bias. This work presents an Open Whisper-style Speech Model (OWSM), which reproduces Whisper-style training using an open-source toolkit and publicly available data. OWSM even supports more translation directions and can be more efficient to train. We will publicly release all scripts used for data preparation, training, inference, and scoring as well as pre-trained models and training logs to promote open science.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
