arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2605.17552cs.LG
收藏

Q-LocalAdam: 一种内存高效的边缘联邦学习客户端自适应优化方法

Q-LocalAdam: Memory-Efficient Client-Side Adaptive Optimization for Edge Federated Learning

IISER Bhopal(印度比哈尔州科学与技术研究院)

Vedant Waykole, Haroon R. Lone

AI总结:

本文提出Q-LocalAdam,一种针对边缘联邦学习中非独立同分布数据和内存限制的自适应优化方法,通过分布感知的8位量化块线性编码和对数空间编码实现内存高效优化,显著提升模型性能和并发工作负载能力。

摘要:

边缘设备上的联邦学习必须应对非独立同分布的客户端数据和严格的内存预算。像Adam这样的自适应优化器在数据异质性下稳定训练,但需要存储全精度动量和方差状态,通常使客户端内存开销增加三倍。这限制了在资源受限设备上可部署的模型大小和同时进行的联邦任务数量。我们实证发现,联邦Adam中的动量和方差在统计特性上存在根本差异:动量值对称且有界,而方差跨越八个数量级并具有对数正态结构。受这种不对称性启发,我们提出了Q-LocalAdam,它对动量应用分布感知的8位量化块线性编码,对方差应用对数空间编码,同时保持模型参数在全精度下。在CIFAR-10和CIFAR-100上,针对不同数据异质性(α∈{0.1, 0.5, 1.0, IID}),Q-LocalAdam在中等异质性下实现3.37倍的优化器内存减少,无精度损失,在极端异质性下(如CIFAR-100,α=0.1)实现显著提升(+5.74pp)。多种子验证确认统计显著性(p<0.01)。相比之下,朴素的均匀量化退化到随机性能,证明了分布感知设计的重要性。Q-LocalAdam在内存受限的边缘设备上无需修改联邦协议即可实现更大的模型和更多的并发工作负载。

英文摘要:

On-device federated learning (FL) on edge devices is limited by the memory required by client-side adaptive optimizers. Maintaining optimizer states locally increases memory usage and can restrict the size of models that can be deployed on resource-constrained edge devices. State quantization can reduce this memory footprint, but uniform compression methods may perform poorly under skewed, non-IID data distributions because they can lose small-variance values that are important for stable optimization. We present \textbf{Q-LocalAdam}, a distribution-aware optimizer for memory-constrained edge devices. It applies block-wise linear quantization to the bounded momentum state and log-space quantization to the heavy-tailed variance state. A post-dequantization variance floor is used to maintain numerical stability. Q-LocalAdam reduces the persistent optimizer-state memory footprint while achieving convergence close to full-precision training. We evaluate Q-LocalAdam on physical edge devices and GPU, measuring persistent optimizer-state storage, process-level memory, and the latency introduced by quantization across human activity, vision, and language workloads. Our results show reduced peak resident memory with modest latency overhead, indicating that distribution-aware state compression can make adaptive optimization practical for edge deployments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑