CLiFT-ASR：面向低资源台语（台湾闽南语）语音识别的跨语言微调框架
CLiFT-ASR: A Cross-Lingual Fine-Tuning Framework for Low-Resource Taiwanese Hokkien Speech Recognition
AI总结：
针对低资源台语语音识别中汉字与罗马字标注难以兼顾的问题，提出基于普通话 HuBERT 的两阶段跨语言微调框架 CLiFT-ASR，在 TAT-MOE 上实现 CER 相对降低 24.88%。
摘要：
由于标注数据稀缺，台语（台湾闽南语）等低资源语言的自动语音识别（ASR）十分困难。然而，直接在汉字转写上进行微调往往难以捕捉细致的语音和声调线索，而仅基于罗马字进行训练又缺乏词汇和句法覆盖。此外，以往研究很少探索整合两种标注类型的分阶段策略。为弥补这一空白，我们提出 CLiFT-ASR，一个基于普通话 HuBERT 模型并逐步将其适配到台语的跨语言微调框架。该框架采用两阶段流程：首先从语音式台罗拼音标注中学习声学和声调表示，然后从汉字转写中获取词汇和句法信息。这种渐进式适配能够有效对齐语音与正字法结构。在 TAT-MOE 语料库上的实验表明，与强基线相比，CLiFT-ASR 在字错误率（CER）上实现了 24.88% 的相对降低。结果表明，CLiFT-ASR 为台语 ASR 提供了一种有效且参数高效的解决方案，并有望惠及其他低资源语言场景。
英文摘要：
Automatic speech recognition (ASR) for low-resource languages such as Taiwanese Hokkien is difficult due to the scarcity of annotated data. However, direct fine-tuning on Han-character transcriptions often fails to capture detailed phonetic and tonal cues, while training only on romanization lacks lexical and syntactic coverage. In addition, prior studies have rarely explored staged strategies that integrate both annotation types. To address this gap, we present CLiFT-ASR, a cross-lingual fine-tuning framework that builds on Mandarin HuBERT models and progressively adapts them to Taiwanese Hokkien. The framework employs a two-stage process in which it first learns acoustic and tonal representations from phonetic Tai-lo annotations and then captures vocabulary and syntax from Han-character transcriptions. This progressive adaptation enables effective alignment between speech sounds and orthographic structures. Experiments on the TAT-MOE corpus demonstrate that CLiFT-ASR achieves a 24.88\% relative reduction in character error rate (CER) compared with strong baselines. The results indicate that CLiFT-ASR provides an effective and parameter-efficient solution for Taiwanese Hokkien ASR and that it has potential to benefit other low-resource language scenarios.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
