跳到正文
研踪PaperTrail
← 研踪 PaperTrail

arXiv:2609.01126 · v2

How Evaluation Choices Shape the Measured Benefit of Online Adaptation in Time-Series Forecasting: Warmup, Learning-Rate Selection, and Adaptation-State Memory

首次提交 2026-09-01 20:04
版本更新 2026-11-15 22:57
雷达观测 2026-11-17 19:15
arXiv ↗PDF ↗

摘要

Online adaptation can help edge time-series forecasting under distribution drift, but its measured benefit is sensitive to evaluation choices. We study six public multivariate streams, including building-sensor and smart-meter data, under a leakage-free streaming protocol. We identify two sources of comparison bias. First, the warmup budget of the static baseline has a two-sided effect on seed-mean curves: insufficient warmup undertrains the baseline, whereas excessive warmup can degrade its pre-drift generalization. Across six dataset-backbone settings, the estimated adaptation benefit changes by 3.0 to 18.8 percentage points (pp) over the 1,000-20,000-step warmup range. Second, comparing SGD with momentum (SGD+m) and Adam at a shared default learning rate conflates optimizer quality with rate sensitivity. We select both the warmup budget and each optimizer's online rate using a held-out pre-drift validation slice without accessing test data. Under this procedure, Adam outperforms SGD+m in 310 of 360 cells, and only 4 of its 360 cells fall below the static baseline. Third, we characterize accuracy against adaptation-state memory and A100-measured per-update latency for full, head-only, and calibration-based adaptation. In the evaluated PatchTST frontier settings, several parameter-efficient variants are nondominated on the adaptation-state-memory axis. These findings support a validation-only commissioning procedure. Code, data, and all reported numbers: https://github.com/keiotakmin/tsf-edge-adaptation

作者备注:Accepted as a regular paper at the 2026 IEEE International Conference on Big Data (IEEE BigData 2026)

AI 论文解读

基于摘要 · 非全文审阅

此版本尚未生成 AI 解读。解读由后台任务生成并按论文版本保存,登录后共享已有结果。

版本差异

本站首次收录为 v2,暂无更早版本可对照。

观测到的版本

相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。

v2 · 2026-11-15 22:57 · 最新观测版本

How Evaluation Choices Shape the Measured Benefit of Online Adaptation in Time-Series Forecasting: Warmup, Learning-Rate Selection, and Adaptation-State Memory

Online adaptation can help edge time-series forecasting under distribution drift, but its measured benefit is sensitive to evaluation choices. We study six public multivariate streams, including building-sensor and smart-meter data, under a leakage-free streaming protocol. We identify two sources of comparison bias. First, the warmup budget of the static baseline has a two-sided effect on seed-mean curves: insufficient warmup undertrains the baseline, whereas excessive warmup can degrade its pre-drift generalization. Across six dataset-backbone settings, the estimated adaptation benefit changes by 3.0 to 18.8 percentage points (pp) over the 1,000-20,000-step warmup range. Second, comparing SGD with momentum (SGD+m) and Adam at a shared default learning rate conflates optimizer quality with rate sensitivity. We select both the warmup budget and each optimizer's online rate using a held-out pre-drift validation slice without accessing test data. Under this procedure, Adam outperforms SGD+m in 310 of 360 cells, and only 4 of its 360 cells fall below the static baseline. Third, we characterize accuracy against adaptation-state memory and A100-measured per-update latency for full, head-only, and calibration-based adaptation. In the evaluated PatchTST frontier settings, several parameter-efficient variants are nondominated on the adaptation-state-memory axis. These findings support a validation-only commissioning procedure. Code, data, and all reported numbers: https://github.com/keiotakmin/tsf-edge-adaptation

查看此版本解读 →查看此版本原文 ↗