Horizon-LM: A RAM-Centric Architecture for LLM Training
Horizon-LM:一种以RAM为中心的LLM训练架构
专题命中 效率与部署 :LLM(title);large language model(abstract);language model(abstract);instruction tuning(abstract)
AI总结 Horizon-LM通过重新定义CPU和GPU的角色,以主机内存为核心参数存储,实现大模型训练的内存优化,提升训练吞吐量并降低对多GPU集群的依赖。
Comments This paper contained an error in the throughput computation used in the experimental evaluation. Specifically, the TFLOPS calculation omitted the 12HL term in the training FLOPs formula, which led to systematic underestimation of the reported throughput numbers in the experimental results. We are withdrawing this version to correct the evaluation and avoid confusion for readers