发表机构
Kyushu University; hon9kon9ize; The Education University of Hong Kong; Votee AI; Beever AI（九州大学; hon9kon9ize; 香港教育大学; Votee AI; Beever AI）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对粤语低资源推理数据缺失问题，提出基于Qwen3的CantoneseLLM v2，通过CPT、SFT、DPO和RLVR训练，在HKCanto-Eval上达73.16，恢复推理性能并保留粤语推理行为。
AI 中文摘要
粤语使用广泛，但在书面数据方面仍属低资源语言，目前没有可用于模型训练的大型原生粤语推理语料库。我们开发并发布了 CantoneseLLM v2，包括基于 Qwen3 8B 和 30B-A3B 的模型。这些模型通过 CPT（在 7.84 亿个粤语和香港相关 token 上）、chat-vector 合并、SFT、DPO 和 RLVR 进行训练。跨训练阶段的评估显示，chat-vector 合并转移了指令跟随能力，但保留了捐赠模型的推理语言，而使用有限的粤语推理数据进行 SFT 会大幅缩短或移除推理痕迹并降低基准性能。DPO 恢复了推理块格式，尤其是对于 8B 模型，但仅恢复了部分丢失的性能。使用粤语和繁体中文脚本作为乘法约束的 RLVR 训练引入了粤语语言对齐并恢复了丢失的性能。30B-A3B 模型在 HKCanto-Eval 上达到 73.16，与其合并检查点相差 1.20 分，同时保留了该检查点所缺乏的粤语推理行为。我们发布了模型检查点、训练环境和一个十三年份的繁体中文 Common Crawl 数据集。模型可通过此 https URL 访问。
英文摘要
Cantonese is widely spoken but remains low-resource in written data, with no large corpus of native Cantonese reasoning traces available for model training. We develop and release CantoneseLLM v2, comprising models based on Qwen3 8B and 30B-A3B. The models are trained through CPT on 784 million Cantonese and Hong Kong-related tokens, chat-vector merging, SFT, DPO, and RLVR. Evaluation across the training stages shows that chat-vector merging transfers instruction following but preserves the donor model's reasoning language, while SFT with limited Cantonese reasoning data substantially shortens or removes reasoning traces and reduces benchmark performance. DPO restores the reasoning-block format, particularly for the 8B model, but recovers only part of the lost performance. The RLVR training with Cantonese language and Traditional Chinese scripts as multiplicative constraints introduced Cantonese language alignment and restored the lost performance. The 30B-A3B model reaches 73.16 on HKCanto-Eval, within 1.20 points of its merged checkpoint, while retaining the Cantonese reasoning behaviour absent from that checkpoint. We release the model checkpoints, the training environments, and a thirteen-year Traditional Chinese Common Crawl dataset. The models can be accessed at https://huggingface.co/collections/hon9kon9ize/cantonesellm-v20
