← 研踪 PaperTrailarXiv:2605.17552 · v2
Q-LocalAdam: Memory-Efficient Client-Side Adaptive Optimization for Edge Federated Learning
首次提交 2026-05-18 01:23
版本更新 2026-11-17 14:37
雷达观测 2026-11-18 19:43
摘要
On-device federated learning (FL) on edge devices is limited by the memory required by client-side adaptive optimizers. Maintaining optimizer states locally increases memory usage and can restrict the size of models that can be deployed on resource-constrained edge devices. State quantization can reduce this memory footprint, but uniform compression methods may perform poorly under skewed, non-IID data distributions because they can lose small-variance values that are important for stable optimization. We present \textbf{Q-LocalAdam}, a distribution-aware optimizer for memory-constrained edge devices. It applies block-wise linear quantization to the bounded momentum state and log-space quantization to the heavy-tailed variance state. A post-dequantization variance floor is used to maintain numerical stability. Q-LocalAdam reduces the persistent optimizer-state memory footprint while achieving convergence close to full-precision training. We evaluate Q-LocalAdam on physical edge devices and GPU, measuring persistent optimizer-state storage, process-level memory, and the latency introduced by quantization across human activity, vision, and language workloads. Our results show reduced peak resident memory with modest latency overhead, indicating that distribution-aware state compression can make adaptive optimization practical for edge deployments.
观测到的版本
相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-17 14:37 · 最新观测版本
Q-LocalAdam: Memory-Efficient Client-Side Adaptive Optimization for Edge Federated Learning
On-device federated learning (FL) on edge devices is limited by the memory required by client-side adaptive optimizers. Maintaining optimizer states locally increases memory usage and can restrict the size of models that can be deployed on resource-constrained edge devices. State quantization can reduce this memory footprint, but uniform compression methods may perform poorly under skewed, non-IID data distributions because they can lose small-variance values that are important for stable optimization. We present \textbf{Q-LocalAdam}, a distribution-aware optimizer for memory-constrained edge devices. It applies block-wise linear quantization to the bounded momentum state and log-space quantization to the heavy-tailed variance state. A post-dequantization variance floor is used to maintain numerical stability. Q-LocalAdam reduces the persistent optimizer-state memory footprint while achieving convergence close to full-precision training. We evaluate Q-LocalAdam on physical edge devices and GPU, measuring persistent optimizer-state storage, process-level memory, and the latency introduced by quantization across human activity, vision, and language workloads. Our results show reduced peak resident memory with modest latency overhead, indicating that distribution-aware state compression can make adaptive optimization practical for edge deployments.
查看此版本解读 →查看此版本原文 ↗