arXiv:2610.21742 · v3
Sparse Federated Echo State Networks for Time-Series Classification via Smooth-L1 Regularization
摘要
Echo State Networks (ESNs), as lightweight implementations of reservoir computing, provide efficient solutions for temporal data processing in resource-constrained environments. While standard ESNs offer computational efficiency, their dense readout layers inflate the stored model and the per-sample inference cost. This paper introduces a Smooth-L1 regularized ESN (SL1ESN) that promotes sparsity in the readout layer while preserving differentiability for gradient-based optimization. The method employs a twice-differentiable Smooth-L1 approximation compatible with Newton's method, which we exploit both for centralized training and for a second-order (Newton-step) federated aggregation scheme. Under a single fixed shrinkage threshold shared by every dataset and both settings, experiments on seven time-series classification benchmarks show that SL1ESN learns readouts that are 77-95% sparse in the centralized setting and 75-95% sparse in the federated one. On five of the seven benchmarks, the mean accuracy of the sparse readout over ten seeds differs from that of the stronger dense baseline by at most 0.7 points in the centralized setting and 1.6 points in the federated one, differences of the same order as the seed-to-seed variation, while on the remaining two it trails by 2.4-5.8 points. The second-order federated update converges in only a few rounds. The resulting sparsity compresses the stored model, the per-sample inference cost, and the server-to-client broadcast by a comparable factor. These results demonstrate the practical value of sparse readouts for edge computing and IoT deployments.
作者备注：30 pages, 8 figures, 7 tables. Extended version of a paper presented at IDEAL 2025. Code: https://github.com/oroboro235/FedESN-SmoothL1. The abstract and the corresponding results and discussion sentences now state that the accuracy margins are differences of ten-seed means, not bounds that hold for other seeds