arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2601.09200cs.CLcs.AI
收藏

A.X K1 技术报告

A.X K1 Technical Report

SK Telecom(SK电信)
Sung Jun Cheon, Jaekyung Cho, Seongho Choi, Hyunjun Eun, Seokhwan Jo, Jaehyun Jun, Minsoo Kang, Jin Kim, Jiwon Kim, Minsang Kim, Seungsik Kim, Sungwan Kim, Tae … 展开作者

Sung Jun Cheon, Jaekyung Cho, Seongho Choi, Hyunjun Eun, Seokhwan Jo, Jaehyun Jun, Minsoo Kang, Jin Kim, Jiwon Kim, Minsang Kim, Seungsik Kim, Sungwan Kim, Tae Yoon Kim, Youngrang Kim, Hyeongmun Lee, Sangyeol Lee, Sungeun Lee, Youngsoon Lee, Yujin Lee, Seongmin Ok, Chanyong Park, Hyewoong Park, Junyoung Park, Hyunho Yang, Subin Yi, Dhammiko Arya, Soohyun Bae, Dongyeon Cho, Seungmo Cho, Sangho Choi, Yongseok Choi, Gyoungeun Han, Yong-jin Han, Seokyoung Hong, Hyeon Hwang, Wonbeom Jang, Minjeong Ju, Wonjin Jung, Keummin Ka, Sungil Kang, Dongnam Kim, Jonghwi Kim, Joonghoon Kim, SaeRom Kim, Sangjin Kim, Seongwon Kim, Youngjin Kim, Seojin Lee, Sunwoo Lee, Taehoon Lee, Chanwoo Park, Sohee Park, Sooyeon Park, Yohan Ra, Sereimony Sek, Seungyeon Seo, Gun Song, Sanghoon Woo, Janghan Yoon, Sungbin Yoon

AI总结:

A.X K1 是一个5190亿参数的混合专家语言模型,通过Think-Fusion训练配方实现可控推理,具备在韩语基准测试中优于开源模型的性能。

摘要:

我们介绍了A.X K1,一个拥有5190亿参数的混合专家(MoE)语言模型,从头开始训练。我们的设计利用扩展定律来优化训练配置和词汇表大小,在固定计算预算下进行优化。A.X K1 在约10万亿个标记的语料库上进行预训练,该语料库通过多阶段的数据处理流程进行整理。设计目的是弥合推理能力和推理效率之间的差距,A.X K1 支持显式可控的推理,以促进在各种现实世界场景中的可扩展部署。我们提出了一种简单而有效的Think-Fusion训练配方,使用户能够在单一统一模型中控制切换到思考和非思考模式。广泛评估表明,A.X K1 在性能上与领先的开源模型相媲美,同时在韩语基准测试中建立了独特的优势。

英文摘要:

We introduce A.X K1, a 519B-parameter Mixture-of-Experts (MoE) language model trained from scratch. Our design leverages scaling laws to optimize training configurations and vocabulary size under fixed computational budgets. A.X K1 is pre-trained on a corpus of approximately 10T tokens, curated by a multi-stage data processing pipeline. Designed to bridge the gap between reasoning capability and inference efficiency, A.X K1 supports explicitly controllable reasoning to facilitate scalable deployment across diverse real-world scenarios. We propose a simple yet effective Think-Fusion training recipe, enabling user-controlled switching between thinking and non-thinking modes within a single unified model. Extensive evaluations demonstrate that A.X K1 achieves performance competitive with leading open-source models, while establishing a distinctive advantage in Korean-language benchmarks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑