大型语言模型的网络边缘推断:原理、技术与机遇
Network Edge Inference for Large Language Models: Principles, Techniques, and Opportunities
AI总结:
本文探讨了大型语言模型在边缘网络中的推断挑战,综述了系统架构、模型优化与资源管理等最新进展,旨在提升边缘环境下的LLM性能。
摘要:
大型语言模型(LLMs)已迅速发展,凭借其卓越的语言理解、生成和推理能力,成为跨领域的多功能工具。然而,由于其巨大的内存和计算需求,执行LLM推断仍面临网络边缘的挑战。本文综述了LLM边缘推断的具体挑战,提供了近期进展的全面概述,涵盖系统架构、模型优化和部署以及资源管理和调度。通过综合最新技术并映射未来方向,本文旨在解锁LLM在资源受限边缘环境中的潜力。
英文摘要:
Large language models (LLMs) have advanced rapidly, emerging as versatile tools across fields thanks to their exceptional language understanding, generation, and reasoning capabilities. However, performing LLM inference at the network edge remains challenging due to their large memory and compute demands. This survey outlines the challenges specific to LLM edge inference and provides a comprehensive overview of recent progress, covering system architectures, model optimization and deployment, and resource management and scheduling. By synthesizing state-of-the-art techniques and mapping future directions, this survey aims to unlock the potential of LLMs in resource-constrained edge environments.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。