← 研踪 PaperTrailarXiv:2605.10313 · v2
Signature Approach for Contextual Bandits with Nonlinear and Path-dependent Rewards: Regret Bound and Statistical Inference
首次提交 2026-05-11 18:13
版本更新 2026-11-18 21:47
雷达观测 2026-11-19 15:04
摘要
We study contextual bandits with nonlinear and path-dependent rewards through a novel signature-transform-based approach. Leveraging the universal nonlinearity property of signatures, we approximate continuous path-dependent reward functionals by linear functionals in the signature space. This representation enables the use of efficient linear contextual bandit methods while preserving expressive sequential structure. Building on this framework, we propose \texttt{DisSigUCB}, a signature-based disjoint upper confidence bound (UCB) algorithm. Under boundedness and non-degeneracy assumptions, we establish a high-probability regret bound of order \(\tilde\mathcal{O}\bigl(d_{\text{Sig}} \sqrt{KT} + \ varepsilon_N T \sqrt{d_\Sig}\bigr)\) with being the number of actions, the signature feature dimension, and the finite-depth approximation error. We further introduce -\texttt{DisSigUCB}, a randomized variant that ensures persistent exploration for statistical inference. Under additional margin and long-run design conditions, we prove consistency and asymptotic normality of the action-specific ridge estimators. Both the synthetic experiments and the numerical studies on temperature sensor monitoring, sleep-stage classification, and hospital nurse staffing demonstrate the effectiveness of \texttt{DisSigUCB} over linear and kernelized contextual bandit baselines in nonlinear and path-dependent settings.
观测到的版本
相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-18 21:47 · 最新观测版本
Signature Approach for Contextual Bandits with Nonlinear and Path-dependent Rewards: Regret Bound and Statistical Inference
We study contextual bandits with nonlinear and path-dependent rewards through a novel signature-transform-based approach. Leveraging the universal nonlinearity property of signatures, we approximate continuous path-dependent reward functionals by linear functionals in the signature space. This representation enables the use of efficient linear contextual bandit methods while preserving expressive sequential structure. Building on this framework, we propose \texttt{DisSigUCB}, a signature-based disjoint upper confidence bound (UCB) algorithm. Under boundedness and non-degeneracy assumptions, we establish a high-probability regret bound of order \(\tilde\mathcal{O}\bigl(d_{\text{Sig}} \sqrt{KT} + \ varepsilon_N T \sqrt{d_\Sig}\bigr)\) with being the number of actions, the signature feature dimension, and the finite-depth approximation error. We further introduce -\texttt{DisSigUCB}, a randomized variant that ensures persistent exploration for statistical inference. Under additional margin and long-run design conditions, we prove consistency and asymptotic normality of the action-specific ridge estimators. Both the synthetic experiments and the numerical studies on temperature sensor monitoring, sleep-stage classification, and hospital nurse staffing demonstrate the effectiveness of \texttt{DisSigUCB} over linear and kernelized contextual bandit baselines in nonlinear and path-dependent settings.
查看此版本解读 →查看此版本原文 ↗