SPA:迈向一种计算友好的云端与设备端协作的Seq2seq个性化生成与因果推断
SPA: Towards A Computational Friendly Cloud-Base and On-Devices Collaboration Seq2seq Personalized Generation with Casual Inference
AI总结:
本文提出SPA架构,通过云端预训练LLMs与设备端附加参数交互,实现计算友好的云端与设备端协作Seq2seq个性化生成,解决低资源设备上的内存与计算限制。
摘要:
大型语言模型(LLMs)在各种任务和问答中展现了其卓越的能力。然而,LLMs在低资源设备上需要大量的内存存储。更关键的是,这些设备上的计算速度也受到严重限制。在本文中,我们提出了SPA(Side Plugin Adaption,侧边插件适配),一种在严格的设备端计算和内存约束下用于快速设备端推理的轻量级架构。与其他设备端seq2seq生成相比,SPA能够在低资源约束下进行快速且稳定的推理,从而使其获得成本效益。我们的方法在云端预训练的LLMs与设备端的附加参数之间建立交互,能够同时提供预训练LLMs的知识和特定的个性化特征。此外,SPA提供了一个框架,将基于特征的参数保留在低计算设备上,而将包含通用信息的参数留在高计算设备上。
英文摘要:
Large language models(LLMs) have shown its outperforming ability on various tasks and question answering. However, LLMs require substantial memory storage on low-resource devices. More critically, the computational speed on these devices is also severely limited. In this paper, we propose SPA(Side Plugin Adaption), a lightweight architecture for fast on-devices inference on the constraints of strict on-devices computation and memory constraints. Compared with other on-devices seq2seq generation, SPA could make a fast and stable inference on low-resource constraints, allowing it to obtain cost effiency. Our method establish an interaction between a pretrained LLMs on-cloud and additive parameters on-devices, which could provide the knowledge on both pretrained LLMs and featured personal feature. Further more, SPA provides a framework to keep feature-base parameters on low computational devices while leave the parameters containing general information on the high computational devices.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。