← 研踪 PaperTrailarXiv:2609.01126 · v2
How Evaluation Choices Shape the Measured Benefit of Online Adaptation in Time-Series Forecasting: Warmup, Learning-Rate Selection, and Adaptation-State Memory
首次提交 2026-09-01 20:04
版本更新 2026-11-15 22:57
雷达观测 2026-11-17 19:15
摘要
Online adaptation can help edge time-series forecasting under distribution drift, but its measured benefit is sensitive to evaluation choices. We study six public multivariate streams, including building-sensor and smart-meter data, under a leakage-free streaming protocol. We identify two sources of comparison bias. First, the warmup budget of the static baseline has a two-sided effect on seed-mean curves: insufficient warmup undertrains the baseline, whereas excessive warmup can degrade its pre-drift generalization. Across six dataset-backbone settings, the estimated adaptation benefit changes by 3.0 to 18.8 percentage points (pp) over the 1,000-20,000-step warmup range. Second, comparing SGD with momentum (SGD+m) and Adam at a shared default learning rate conflates optimizer quality with rate sensitivity. We select both the warmup budget and each optimizer's online rate using a held-out pre-drift validation slice without accessing test data. Under this procedure, Adam outperforms SGD+m in 310 of 360 cells, and only 4 of its 360 cells fall below the static baseline. Third, we characterize accuracy against adaptation-state memory and A100-measured per-update latency for full, head-only, and calibration-based adaptation. In the evaluated PatchTST frontier settings, several parameter-efficient variants are nondominated on the adaptation-state-memory axis. These findings support a validation-only commissioning procedure. Code, data, and all reported numbers: https://github.com/keiotakmin/tsf-edge-adaptation
作者备注:Accepted as a regular paper at the 2026 IEEE International Conference on Big Data (IEEE BigData 2026)
观测到的版本
相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-15 22:57 · 最新观测版本
How Evaluation Choices Shape the Measured Benefit of Online Adaptation in Time-Series Forecasting: Warmup, Learning-Rate Selection, and Adaptation-State Memory
Online adaptation can help edge time-series forecasting under distribution drift, but its measured benefit is sensitive to evaluation choices. We study six public multivariate streams, including building-sensor and smart-meter data, under a leakage-free streaming protocol. We identify two sources of comparison bias. First, the warmup budget of the static baseline has a two-sided effect on seed-mean curves: insufficient warmup undertrains the baseline, whereas excessive warmup can degrade its pre-drift generalization. Across six dataset-backbone settings, the estimated adaptation benefit changes by 3.0 to 18.8 percentage points (pp) over the 1,000-20,000-step warmup range. Second, comparing SGD with momentum (SGD+m) and Adam at a shared default learning rate conflates optimizer quality with rate sensitivity. We select both the warmup budget and each optimizer's online rate using a held-out pre-drift validation slice without accessing test data. Under this procedure, Adam outperforms SGD+m in 310 of 360 cells, and only 4 of its 360 cells fall below the static baseline. Third, we characterize accuracy against adaptation-state memory and A100-measured per-update latency for full, head-only, and calibration-based adaptation. In the evaluated PatchTST frontier settings, several parameter-efficient variants are nondominated on the adaptation-state-memory axis. These findings support a validation-only commissioning procedure. Code, data, and all reported numbers: https://github.com/keiotakmin/tsf-edge-adaptation
查看此版本解读 →查看此版本原文 ↗