← 研踪 PaperTrailarXiv:2506.23210 · v6
Effective Unbounded Drift Control in Heterogeneous Federated Learning: Bayesian Fine-Tuning Using a Reference Model
首次提交 2025-06-29 20:41
版本更新 2026-11-07 16:57
雷达观测 2026-11-10 11:14
摘要
Federated learning (FL) enables collaborative model training across distributed clients while preserving data privacy. However, data and system heterogeneity often induce unbounded drift in model updates due to each clients' different multiple local steps, partial participation, and unbounded dynamics in model updates. These challenges severely degrade predictive performance, exacerbate catastrophic forgetting, and hinder global optimization forward to optimal updates. Related works (AdaBest, FedEve, A-FedPD) defined dual drift problems and proposed each regularization term to client optimization process. However, in aspect of temporal unbounded drift, they generally reflect last round temporal drift to client optimization while forgets prior temporal unbounded drift. To address these limitations, we propose FedRef, a robust optimization framework that leverages a reference model generalized from prior global models via Bayesian fine-tuning. FedRef integrates a Maximum A Posteriori (MAP)-based regularization mechanism that calibrates global updates toward a temporally aggregated reference model, thereby effectively controlling unbounded drift and maximizing empirical stability. Moreover, unlike prior approaches that rely on complex client-side optimization heuristics, FedRef achieves superior generalization to reflect recency weighting by seamlessly blending a robust probabilistic prior with global likelihood optimization. By maintaining a temporal moving average centered on previous global trajectories, the proposed framework provides a highly reliable optimization anchor.
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v6 · 2026-11-07 16:57 · 最新观测版本
Effective Unbounded Drift Control in Heterogeneous Federated Learning: Bayesian Fine-Tuning Using a Reference Model
Federated learning (FL) enables collaborative model training across distributed clients while preserving data privacy. However, data and system heterogeneity often induce unbounded drift in model updates due to each clients' different multiple local steps, partial participation, and unbounded dynamics in model updates. These challenges severely degrade predictive performance, exacerbate catastrophic forgetting, and hinder global optimization forward to optimal updates. Related works (AdaBest, FedEve, A-FedPD) defined dual drift problems and proposed each regularization term to client optimization process. However, in aspect of temporal unbounded drift, they generally reflect last round temporal drift to client optimization while forgets prior temporal unbounded drift. To address these limitations, we propose FedRef, a robust optimization framework that leverages a reference model generalized from prior global models via Bayesian fine-tuning. FedRef integrates a Maximum A Posteriori (MAP)-based regularization mechanism that calibrates global updates toward a temporally aggregated reference model, thereby effectively controlling unbounded drift and maximizing empirical stability. Moreover, unlike prior approaches that rely on complex client-side optimization heuristics, FedRef achieves superior generalization to reflect recency weighting by seamlessly blending a robust probabilistic prior with global likelihood optimization. By maintaining a temporal moving average centered on previous global trajectories, the proposed framework provides a highly reliable optimization anchor.
查看此版本解读 →查看此版本原文 ↗