解锁多LoRA支持的多用途基础大语言模型的边缘部署与设备端加速
Unlocking the Edge deployment and ondevice acceleration of multi-LoRA enabled one-for-all foundational LLM
AI总结：
本文提出一种硬件感知框架，实现基于LLaMA的多语言基础模型在三星S24/S25设备上的高效设备端推理，通过动态任务切换和多流解码机制，提升内存和延迟性能，验证了多用途大语言模型在边缘设备上的可行性。
摘要：
将大型语言模型（LLMs）部署到智能手机上面临严峻的工程挑战，因为内存、延迟和运行时灵活性有严格限制。本文提出一种硬件感知框架，用于在三星Galaxy S24和S25设备上高效执行基于LLaMA的多语言基础模型，支持多种用例，采用应用特定的LoRAs作为运行时输入到单一冻结的推理图中，实现动态任务切换而无需重新编译或增加内存开销。我们进一步引入多流解码机制，在单次前向传递中同时生成风格变化，如正式、礼貌或欢乐的响应，从而将延迟降低高达6倍。为加速令牌生成，我们应用动态自我推测解码（DS2D），一种基于树的策略，可在不需草案模型的情况下预测未来令牌，从而在解码时间上获得高达2.3倍的加速。结合量化到INT4和架构级优化，我们的系统在内存和延迟方面实现了4-6倍的总体改进，同时在9种语言和8种任务上保持准确性。这些结果证明了在边缘设备上部署多用途LLMs的实用性，推动了生成式AI在移动平台上的商业化可行性。
英文摘要：
Deploying large language models (LLMs) on smartphones poses significant engineering challenges due to stringent constraints on memory, latency, and runtime flexibility. In this work, we present a hardware-aware framework for efficient on-device inference of a LLaMA-based multilingual foundation model supporting multiple use cases on Samsung Galaxy S24 and S25 devices with SM8650 and SM8750 Qualcomm chipsets respectively. Our approach integrates application-specific LoRAs as runtime inputs to a single frozen inference graph, enabling dynamic task switching without recompilation or memory overhead. We further introduce a multi-stream decoding mechanism that concurrently generates stylistic variations - such as formal, polite, or jovial responses - within a single forward pass, reducing latency by up to 6x. To accelerate token generation, we apply Dynamic Self-Speculative Decoding (DS2D), a tree-based strategy that predicts future tokens without requiring a draft model, yielding up to 2.3x speedup in decode time. Combined with quantization to INT4 and architecture-level optimizations, our system achieves 4-6x overall improvements in memory and latency while maintaining accuracy across 9 languages and 8 tasks. These results demonstrate practical feasibility of deploying multi-use-case LLMs on edge devices, advancing the commercial viability of Generative AI in mobile platforms.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
