发表机构
Paderborn University（帕德博恩大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对大语言模型在边缘设备部署的挑战，提出SelectInfer框架，通过离线分析识别特定任务和通用神经元，实施选择性加载和计算两项关键优化，可显著减少内存与计算量，同时保持任务性能，推动LLM在边缘设备的部署。
AI 中文摘要
大语言模型（LLMs）在自然语言处理（NLP）任务中展现出卓越能力，但在资源受限的边缘设备上部署时，其高计算和内存需求带来巨大挑战。现有模型压缩与优化方法常依赖粗粒度剪枝或量化，可能影响精度或需重新训练与微调。本文介绍了SelectInfer，这是一个神经元级优化框架，通过选择性神经元加载和计算，实现边缘设备上高效的LLM推理。利用离线LLM分析器分析并识别特定任务和通用神经元，实施两项关键优化：选择性加载，通过选择性加载离线阶段确定的最重要神经元子集来减少内存占用；选择性计算，在运行时仅动态计算最相关神经元。多个数据集评估表明，SelectInfer在保留任务性能的同时，显著减少内存占用和计算量，朝着在边缘设备上部署LLM迈出了实际一步。
英文摘要
Large Language Models (LLMs) have demonstrated remarkable capabilities across a range of Natural Language Processing (NLP) tasks, but their high computational and memory demands pose significant challenges for deployment on resource-constrained edge devices. Existing approaches to model compression and optimization often rely on coarse-grained pruning or quantization, which can compromise accuracy or require re-training and fine-tuning. In this work, we introduce SelectInfer, a neuron-level optimization framework that enables efficient LLM inference on edge devices through selective neuron loading and computation. By profiling and identifying both task-specific and general-purpose neurons using an offline LLM profiler, SelectInfer implements two key optimizations: selective loading, which reduces memory footprint by selectively loading a subset of neurons that were identified to be most important during the offline stage, and selective computation, which dynamically computes only the most relevant neurons at runtime. Evaluation across multiple datasets shows that SelectInfer achieves significant reductions in memory footprint and computation while preserving task performance, making it a practical step towards enabling LLM deployment on edge devices
