Compass-V2 技术报告
Compass-V2 Technical Report
AI总结：
本文介绍 Compass-v2，一个专为东南亚语言和电子商务设计的轻量级 MoE 模型，通过混合推理和高质量数据集，在子 30B 模型中实现最优性能并降低推理成本。
摘要：
主流的大型语言模型主要关注高资源语言，而忽视了低资源语言，尤其是东南亚（SEA）地区的语言。此外，这些模型是通用型的，对电子商务领域的关注有限。为了克服这些局限性，我们推出了 Compass-v2，一个专为东南亚语言和电子商务应用设计的轻量级混合专家（MoE）模型。为了平衡模型性能和推理成本，该模型设计为总参数 300 亿、激活参数 50 亿，并采用了细粒度和共享专家模块。为了增强多语言性能，我们据我们所知，策划并构建了一个高质量、行业领先的东南亚数据集。为了提升电子商务领域的性能，我们构建了一个包含数千亿个令牌的数据集，这些数据来源于外部数据挖掘和内部平台收集。此外，我们开创了一种混合推理模型，在统一框架内支持快速思考和深度思考，以增强推理能力，这与业界通常部署两个独立模型的惯例不同。通过广泛的实验评估，我们的模型在 300 亿参数以下的模型中展示了最先进的东南亚多语言和电子商务性能，同时保持了显著较低的推理成本。
英文摘要：
Predominant LLMs focus on high-resource languages while leaving low-resource languages, particularly those in Southeast Asia (SEA), underrepresented. In addition, those models are general-purpose and pay limited attention to the e-commerce domain. To overcome these limitations, we introduce Compass-v2, a lightweight Mixture-of-Experts (MoE) model specifically designed for Southeast Asian languages and e-commerce applications. To balance model performance and inference cost, the model is designed with 30B total parameters and 5B active parameters, incorporating both fine-grained and shared expert modules. To enhance multilingual performance, we curated and constructed a high-quality, industry-leading SEA dataset, to the best of our knowledge. To boost performance in the e-commerce domain, we built a dataset comprising hundreds of billions of tokens, sourced through external data mining and internal platform collection. Besides, we pioneered a hybrid reasoning model that supports both fast thinking and deep thinking within a unified framework to enhance the reasoning capabilities, diverging from the conventional industry practice of deploying two separate models. Through extensive experimental evaluations, our model demonstrates state-of-the-art SEA multilingual and e-commerce performance among sub-30B models, while maintaining significantly lower inference cost.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
