Qwenfant:基于简洁平行文本训练大型字符级BabyLMs
Qwenfant: Train Big Character-level BabyLMs on Concise Parallel Text
AI总结:
该研究针对2026 BabyLM Challenge多语言赛道,用LLM处理的合成高质量数据、更大模型及字符级词汇(需LoRA微调)训练出Qwenfant,获该赛道排名第一。
摘要:
本研究涉及我们提交的2026 BabyLM Challenge多语言赛道参赛作品。我们使用经LLM总结和翻译的数据改进了基线模型,表明更高质量的数据可来自合成来源。我们通过两项简单改动进一步提升性能:使用更大的模型,以及采用字符级词汇。这两项改动均需LoRA风格微调以充分发挥其潜力。我们将最终基于Qwen3的模型命名为Qwenfant,提交时该模型在多语言赛道排名第一。
英文摘要:
This work concerns our submission for the Multilingual track of the 2026 BabyLM Challenge. We improve on our baseline using data that has been summarized and translated by an LLM, showing that higher-quality data can come from synthetic sources. We further improve with 2 simple changes: using a larger model, and using a character-level vocabulary. Both of these changes require LoRA-style fine-tuning for realizing their full potential. We dub our final Qwen3-based model Qwenfant, which, at the time of submission, ranks 1st in the Multilingual track.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。