Although Large Language Models (LLMs) excel in many tasks, their application to Speech-to-Speech Translation (S2ST) is underexplored and hindered by data scarcity. To bridge this gap, we propose PROST-LLM (PROgressive Speech-to-speech Translation) to enhance the S2ST capabilities in LLMs progressively. First, we fine-tune the LLMs with the CVSS corpus, employing designed tri-task learning and chain of modality methods to boost the initial performance. Then, leveraging the fine-tuned model, we generate preference pairs through self-sampling and back-translation without human evaluation. Finally, these preference pairs are used for preference optimization to enhance the model's S2ST capability further. Extensive experiments confirm the effectiveness of our proposed PROST-LLM in improving the S2ST capability of LLMs.
HPRO: Hierarchical Progressive Reward Optimization via Preference Extraction for Emotional Text-to-Speech
South China University of Technology, China（华南理工大学，中国）；Huya Inc., China（欢丫公司，中国）；Tongyi Fun Team, Alibaba Group, China（通义 Fun 团队，阿里巴巴集团，中国）；Foshan University, China（佛山大学，中国）
Bridging the Stability-Expressivity Gap: Synthetic Data Scaling and Preference Alignment for Low-Resource Spoken Language Models
Beijing University of Posts（北京邮电大学）；University of California, USA（美国加州大学）；Northwestern University, USA（美国西北大学）；Eastern Institute of Technology, Ningbo, China（宁波工程技术学院）
