面向全局时间序列预测的按序列交叉验证
Series-wise Cross-validation for Global Time Series Forecasting
Vitor Cerqueira, Mariana Oliveira, Luis Roque
AI 速读 基于摘要
论文形式化并提出按序列划分的交叉验证(series-wise CV),在11个数据集、10种神经网络架构上系统比较其与按时间划分的交叉验证在多个评估维度上的表现。
stat.ML · cs.LG
查看原文摘要
We consider the problem of cross-validation (CV) in time series forecasting, with a focus on global forecasting models based on deep learning. The prevailing CV strategy in forecasting is to partition data across time with the last part of each series used for testing. With global models trained on collections of time series, however, a fundamentally different approach becomes possible: partitioning across series and time. We formalize series-wise CV for global forecasting models and benchmark several instantiations, including K-fold CV, bootstrapping, and Monte Carlo CV, on 11 datasets with 10 neural network architectures. We evaluate CV procedures along five dimensions: performance estimation, overoptimism rate, model selection accuracy, regret, and downstream forecasting performance. We withhold a random fraction of series from training so that they appear only in the test set. This protocol simulates the emergence of new series not seen during training, a common scenario in global forecasting. Under this protocol, our results reveal a nuanced trade-off: series-wise CV yields better downstream forecasting accuracy by guiding model selection toward configurations that generalize across both time and series, while time-wise CV retains a slight advantage for estimating out-of-sample error.