HyperCLOVA X THINK 技术报告
HyperCLOVA X THINK Technical Report
AI总结：
HyperCLOVA X THINK 是 HyperCLOVA X 系列首个推理大模型，以 6 万亿 token 预训练、128K 上下文和可验证奖励强化学习，在多项韩国基准上以更低算力达到同等规模模型的竞争性能。
摘要：
我们介绍了 HyperCLOVA X THINK，这是 HyperCLOVA X 系列中首个以推理为核心的大语言模型，在大约 6 万亿高质量韩语和英语 token 上进行了预训练，并补充了针对性的合成韩语数据。该模型实现为计算-内存均衡的 Peri-LN Transformer，并通过 μP 进行扩展，采用三阶段课程预训练将上下文窗口扩展到 128K token，随后通过监督微调和基于可验证奖励的强化学习进行后训练，同时支持详细推理过程和简洁答案两种模式。该模型在 KMMLU、CSAT、KoBALT-700、HAERAE-1.0 和 KoBigBench 等以韩国为中心的基准测试上，与同等规模模型相比具有竞争力的性能，同时保持了稳健的双语一致性和翻译质量。此外，一个视觉增强变体在 KCSAT STEM 基准测试上达到或超过了 GPT-4.1，而所有这些成果所使用的训练计算量都远低于现有同等规模模型。我们还提出了一种剪枝和蒸馏技术，该技术将很快应用于 HyperCLOVA X THINK，以构建开源且商业友好的基础模型。总体而言，这些能力使 HyperCLOVA X THINK 成为韩国人工智能创新的坚实基础，也是全球研究社区的宝贵资源。
英文摘要：
We introduce HyperCLOVA X THINK, the first reasoning-focused large language model in the HyperCLOVA X family, pre-trained on roughly $6$ trillion high-quality Korean, and English tokens, augmented with targeted synthetic Korean data. It was implemented as a compute-memory-balanced Peri-LN Transformer scaled with $μ$P, pre-trained through a three-stage curriculum that expands the context window to $128$K tokens, and post-trained via supervised fine-tuning with Reinforcement Learning from Verifiable Rewards supports both detailed rationale and concise-answer modes. It delivers competitive performance against similarly sized models on Korea-focused benchmarks such as KMMLU, CSAT, KoBALT-700, HAERAE-1.0, and KoBigBench, while preserving robust bilingual consistency and translation quality. In addition, a vision-augmented variant matches or exceeds GPT-4.1 on the KCSAT STEM benchmark, all of which are achieved with substantially lower training compute than existing models of similar sizes. We also present a pruning and distillation technique that will soon be applied to HyperCLOVA X THINK for an open-source and business-friendly foundation model. Altogether, these capabilities position HyperCLOVA X THINK as a robust foundation for Korean AI innovation and a valuable resource for the global research community.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
