arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.03359cs.LGcs.AIcs.CL
收藏

MeKi:基于记忆的专家知识注入用于高效的LLM扩展

MeKi: Memory-based Expert Knowledge Injection for Efficient LLM Scaling

Samsung Research(三星研究)

Ning Ding, Fangcheng Liu, Kyungrae Kim, Linji Hao, Kyeng-Hun Lee, Hyeonmok Ko, Yehui Tang

AI总结:

MeKi通过利用存储空间而非计算量扩展LLM容量,实现边缘设备上的高效推理,验证了基于内存的扩展方法在设备端LLM中的有效性。

摘要:

大规模语言模型(LLM)的扩展通常依赖于增加参数数量或测试时计算量以提升性能。然而,这些策略在边缘设备部署中不切实际,因为RAM和NPU资源有限。尽管存在硬件限制,将高性能LLM部署在智能手机等边缘设备上仍对用户体验至关重要。为此,我们提出了MeKi(基于记忆的专家知识注入),一种新的系统,通过存储空间而非FLOPs来扩展LLM的容量。MeKi为每个Transformer层配备token级内存专家,将预存的语义知识注入生成过程。为了弥合训练容量与推理效率之间的差距,我们采用重参数化策略,将训练期间使用的参数矩阵折叠成紧凑的静态查找表。通过将知识转移到ROM,MeKi将模型容量与计算成本解耦,引入零推理延迟开销。大量实验表明,MeKi在相同推理速度下显著优于密集LLM基线,验证了基于内存的扩展范式在设备端LLM中的有效性。项目主页位于https://github.com/ningding-o/MeKi。

英文摘要:

Scaling Large Language Models (LLMs) typically relies on increasing the number of parameters or test-time computations to boost performance. However, these strategies are impractical for edge device deployment due to limited RAM and NPU resources. Despite hardware constraints, deploying performant LLM on edge devices such as smartphone remains crucial for user experience. To address this, we propose MeKi (Memory-based Expert Knowledge Injection), a novel system that scales LLM capacity via storage space rather than FLOPs. MeKi equips each Transformer layer with token-level memory experts that injects pre-stored semantic knowledge into the generation process. To bridge the gap between training capacity and inference efficiency, we employ a re-parameterization strategy to fold parameter matrices used during training into a compact static lookup table. By offloading the knowledge to ROM, MeKi decouples model capacity from computational cost, introducing zero inference latency overhead. Extensive experiments demonstrate that MeKi significantly outperforms dense LLM baselines with identical inference speed, validating the effectiveness of memory-based scaling paradigm for on-device LLMs. Project homepage is at https://github.com/ningding-o/MeKi.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑