并行计算时时间飞逝:基于线性递归的帕累托高效时间序列基础模型
Time Flies When You Parallelise: Pareto-efficient Time Series Foundation Models with Linear Recurrences
Torben Berndt, Lingyi Yang, Federico Nardi, Yi Heng Lim, AJ Shears, Benjamin Walker
AI 速读 基于摘要
作者提出基于 Mamba-3 线性状态递归的时间序列基础模型 ci-proto,在训练时支持分块并行、推理成本随序列长度线性增长,并在 fev-bench 上报告了新的最优结果与帕累托前沿。
cs.LG
查看原文摘要
Time series foundation models have demonstrated strong zero-shot forecasting performance, but computational efficiency remains a central architectural challenge. Temporal self-attention incurs quadratic cost in the number of temporal tokens, while nonlinear recurrent state transitions restrict parallel computation along the temporal dimension. We introduce ci-proto, a family of time series foundation models based on Mamba-3, whose linear state recurrences support chunk-wise parallel-in-time computation during training, and inference costs that scale linearly with sequence length. The architecture alternates recurrent temporal processing with cross-variate attention, supporting multivariate probabilistic forecasting with past-only and future-known covariates. Across fev-bench's 100 forecasting tasks, ci-proto XL sets a new state of the art among time series foundation models. The five ci-proto variants also make up the entire Pareto frontier for mean scaled quantile loss and mean absolute scaled error against both parameter count and median runtime.