硅的对决：消费级大语言模型推理中的性能、效率与生态系统障碍
Silicon Showdown: Performance, Efficiency, and Ecosystem Barriers in Consumer-Grade LLM Inference
AI总结：
本文比较了Nvidia和Apple Silicon生态系统在部署大规模大语言模型时的性能、效率和生态系统挑战，揭示了不同架构在内存容量和计算密度上的差异及影响。
摘要：
本地大语言模型（LLM）推理的运行环境已从轻量级模型转向数据中心级模型，参数超过70B，给消费级硬件带来深远的系统挑战。本文系统地分析了Nvidia和Apple Silicon生态系统，具体描述了部署这些大规模模型所需的内在架构权衡。在Nvidia Blackwell架构上，我们识别出TensorRT-LLM堆栈中的关键“后端二元性”：虽然新的NVFP4量化格式在优化BF16基线基础上提供了1.6倍的吞吐量优势（151 tokens/s vs. 92 tokens/s），但实现这一性能需要在复杂的运行时约束中权衡启动延迟与生成速度。此外，我们描述了70B+模型的“VRAM墙”：在离散GPU上，用户面临在VRAM中降级模型智能的激进量化（例如Q2）或受PCIe瓶颈限制的CPU卸载，后者相比全GPU执行的吞吐量减少了超过90%。相反，Apple的统一内存架构（UMA）绕过了这些瓶颈，使80B参数模型在实用的4位精度下实现线性扩展。这种架构差异延伸到操作可持续性，Apple的SoC设计在能耗效率方面表现突出，达到全GPU执行的23倍（tokens/joule）。我们得出结论，对于消费级推理，最佳硬件由计算密度（Nvidia）和内存容量（Apple）之间的复杂交互定义，并受到显著的“生态系统摩擦”影响，即专有量化工作流。
英文摘要：
The operational landscape of local Large Language Model (LLM) inference has shifted from lightweight models to datacenter-class weights exceeding 70B parameters, creating profound systems challenges for consumer hardware. This paper presents a systematic empirical analysis of the Nvidia and Apple Silicon ecosystems, specifically characterizing the distinct intra-architecture trade-offs required to deploy these massive models. On the Nvidia Blackwell architecture, we identify a critical "Backend Dichotomy" within the TensorRT-LLM stack: while the new NVFP4 quantization format delivers a 1.6x throughput advantage over optimized BF16 baselines (151 tokens/s vs. 92 tokens/s), realizing this performance requires navigating complex runtime constraints that trade startup latency for generation speed. Furthermore, we characterize the "VRAM Wall" for 70B+ models: on discrete GPUs, users face a destructive choice between aggressive quantization (e.g., Q2) that degrades model intelligence to fit in VRAM, or PCIe-bottlenecked CPU offloading, which reduces throughput by over 90% compared to full-GPU execution. Conversely, Apple's Unified Memory Architecture (UMA) circumvents these bottlenecks, enabling linear scaling for 80B parameter models at practical 4-bit precisions. This architectural divergence extends to operational sustainability, where Apple's SoC design demonstrates up to a 23x advantage in energy efficiency (tokens/joule). We conclude that for consumer-grade inference, the optimal hardware is defined by a complex interplay between compute density (Nvidia) and memory capacity (Apple), moderated by the significant "ecosystem friction" of proprietary quantization workflows.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
