Generative speech enhancement offers a promising alternative to traditional discriminative methods by modeling the distribution of clean speech conditioned on noisy inputs. Post-training alignment via reinforcement learning (RL) effectively aligns generative models with human preferences and downstream metrics in domains such as natural language processing, but its use in speech enhancement remains limited, especially for online RL. Prior work explores offline methods like Direct Preference Optimization (DPO); online methods such as Group Relative Policy Optimization (GRPO) remain largely uninvestigated. In this paper, we present the first successful integration of online GRPO into a flow-matching speech enhancement framework, enabling efficient post-training alignment to perceptual and task-oriented metrics with few update steps. Unlike prior GRPO work on Large Language Models, we adapt the algorithm to the continuous, time-series nature of speech and to the dynamics of flow-matching generative models. We show that optimizing a single reward yields rapid metric gains but often induces reward hacking that degrades audio fidelity despite higher scores. To mitigate this, we propose a multi-metric reward optimization strategy that balances competing objectives, substantially reducing overfitting and improving overall performance. Our experiments validate online GRPO for speech enhancement and provide practical guidance for RL-based post-training of generative audio models.
TokAN: Accent Normalization Using Self-Supervised Speech Tokens
School of Data Science (SDS), The Chinese University of Hong Kong, Shenzhen (CUHKSZ)（香港中文大学（深圳）数据科学学院）；Tencent Ethereal Audio Lab, Tencent（腾讯虚幻音频实验室）；School of Intelligence Science and Technology, Nanjing University, Suzhou（南京大学苏州校区智能科学与技术学院）；Shenzhen Loop Area Institute（深圳河套学院）；X-LANCE Lab, School of Computer Science, Shanghai Jiao Tong University（上海交通大学计算机科学与工程学院X-LANCE实验室）；School of Artificial Intelligence (SAI), CUHKSZ（香港中文大学（深圳）人工智能学院）；Shenzhen Research Institute of Big Data（深圳大数据研究院）
