arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.14060cs.CL
收藏

LM-Lexicon:通过和谐语义专家提升定义建模

LM-Lexicon: Improving Definition Modeling via Harmonizing Semantic Experts

BIGAI;Baidu Inc.(百度公司);Peking University(北京大学)

Yang Liu, Jiaye Yang, Weikang Li, Jiahui Liang, Yang Li, Lingyong Yan

AI总结:

LM-Lexicon通过语义专家学习和混合架构提升定义建模效果,实现BLEU分数提升7%,并优化领域级路由机制

摘要:

我们介绍了LM-Lexicon,一种创新的定义建模方法,该方法结合了数据聚类、语义专家学习和模型融合,使用稀疏混合专家架构。通过将定义建模任务分解为专门的语义领域,其中小型语言模型被训练为领域专家,LM-Lexicon在五个广泛使用的基准上相比现有方法实现了显著的改进(+7% BLEU分数)。实证表明,1)聚类策略能够实现细粒度专家专业化,使定义质量提高近10%;2)语义感知的领域级路由机制比传统令牌级路由更有效(+1%);3)通过测试时计算和语义专家扩展可以进一步提升性能。我们的工作推进了定义建模,同时为语义密集型应用的高效语言模型发展提供了见解。

英文摘要:

We introduce LM-Lexicon, an innovative definition modeling approach that incorporates data clustering, semantic expert learning, and model merging using a sparse mixture-of-experts architecture. By decomposing the definition modeling task into specialized semantic domains, where small language models are trained as domain experts, LM-Lexicon achieves substantial improvements (+7% BLEU score compared with the prior state-of-the-art model) over existing methods on five widely used benchmarks. Empirically, we demonstrate that 1) the clustering strategy enables fine-grained expert specialization with nearly 10% improvement in definition quality; 2) the semantic-aware domain-level routing mechanism achieves higher expert efficacy (+1%) than conventional token-level routing; and 3) further performance gains can be obtained through test-time compute and semantic expert scaling. Our work advances definition modeling while providing insights into the development of efficient language models for semantic-intensive applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑