优化多口吃语音分类：利用Whisper编码器实现自动评估中的高效参数缩减
Optimizing Multi-Stuttered Speech Classification: Leveraging Whisper's Encoder for Efficient Parameter Reduction in Automated Assessment
AI总结：
本研究利用Whisper编码器进行多标签分类，通过层冻结策略仅微调单层，将可训练参数减少83.7%，在Fluency-Bank上实现高F1分数，高效识别多口吃语音。
摘要：
口吃语音的自动分类对于及时评估并为语言病理学家提供辅助具有重要意义。尽管该领域已取得显著进展，但语音中出现多种不流畅（disfluencies）的情况仍需关注。我们采取了一种渐进式方法，通过更高效地分类多口吃语音来填补这一空白。该问题首先通过从开源数据集SEP-28k音频片段中整理一个多口吃不流畅数据集来解决。其次，利用Whisper（一种最先进的语音识别模型），通过使用其编码器并将问题视为多标签分类来处理。第三，使用6层编码器的Whisper，并尝试各种层冻结策略，确定了计算效率较高的模型配置。所提出的配置在外部测试数据集（即Fluency-Bank）上分别取得了微平均、宏平均和加权F1分数0.88、0.85和0.87。此外，通过层冻结策略，我们仅微调单个编码器层即可实现上述结果，从而将模型的可训练参数从2027万减少到329万。本研究揭示了最后一个编码器层在识别口吃语音不流畅中的贡献。因此，它带来了一种计算效率高的方法，训练参数减少83.7%，使所提出的方法更适用于各种方言和语言。
英文摘要：
The automated classification of stuttered speech has significant implications for timely assessments providing assistance to speech language pathologists. Despite notable advancements in the field, the cases in which multiple disfluencies occur in speech require attention. We have taken a progressive approach to fill this gap by classifying multi-stuttered speech more efficiently. The problem has been addressed by firstly curating a dataset of multi-stuttered disfluencies from open source dataset SEP-28k audio clips. Secondly, employing Whisper, a state-of-the-art speech recognition model has been leveraged by using its encoder and taking the problem as multi label classification. Thirdly, using a 6 encoder layer Whisper and experimenting with various layer freezing strategies, a computationally efficient configuration of the model was identified. The proposed configuration achieved micro, macro, and weighted F1-scores of 0.88, 0.85, and 0.87, correspondingly on an external test dataset i.e. Fluency-Bank. In addition, through layer freezing strategies, we were able to achieve the aforementioned results by fine-tuning a single encoder layer, consequently, reducing the model's trainable parameters from 20.27 million to 3.29 million. This research study unveils the contribution of the last encoder layer in the identification of disfluencies in stuttered speech. Consequently, it has led to a computationally efficient approach, 83.7% less parameters to train, making the proposed approach more adaptable for various dialects and languages.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
