← 研踪 PaperTrailarXiv:2605.19215 · v2
Not all uncertainty is alike: volatility, stochasticity, and exploration
首次提交 2026-05-19 08:38
版本更新 2026-11-09 07:54
雷达观测 2026-11-10 13:19
摘要
Adaptive decision-making in biological and artificial intelligence requires balancing the exploitation of known outcomes with the exploration of uncertain alternatives. Although prior work suggests that uncertainty generally promotes exploration, it has typically treated distinct sources of environmental uncertainty as equivalent. We consider environments with latent reward states that drift over time (volatility) and are observed through noisy outcomes (stochasticity). Both of them increase posterior uncertainty, yet we show they drive optimal exploration in opposite directions: volatility enhances it, stochasticity suppresses it. We establish this asymmetry formally by extending the Gittins index framework to Gaussian state-space bandits with latent dynamics. We further derive Cause-Aware Uncertainty-Sensitive Exploration (CAUSE), a closed-form exploration bonus obtained via control-as-inference that inherits the same monotonicities. CAUSE matches or exceeds both standard exploration strategies and numerically computed index policies in environments with heterogeneous noise structure. Learning and exploration are governed by the same noise-inference asymmetry, and the framework predicts that pathological noise inference misallocates exploration in an axis-specific way, with implications for computational accounts of psychiatric conditions.
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-09 07:54 · 最新观测版本
Not all uncertainty is alike: volatility, stochasticity, and exploration
Adaptive decision-making in biological and artificial intelligence requires balancing the exploitation of known outcomes with the exploration of uncertain alternatives. Although prior work suggests that uncertainty generally promotes exploration, it has typically treated distinct sources of environmental uncertainty as equivalent. We consider environments with latent reward states that drift over time (volatility) and are observed through noisy outcomes (stochasticity). Both of them increase posterior uncertainty, yet we show they drive optimal exploration in opposite directions: volatility enhances it, stochasticity suppresses it. We establish this asymmetry formally by extending the Gittins index framework to Gaussian state-space bandits with latent dynamics. We further derive Cause-Aware Uncertainty-Sensitive Exploration (CAUSE), a closed-form exploration bonus obtained via control-as-inference that inherits the same monotonicities. CAUSE matches or exceeds both standard exploration strategies and numerically computed index policies in environments with heterogeneous noise structure. Learning and exploration are governed by the same noise-inference asymmetry, and the framework predicts that pathological noise inference misallocates exploration in an axis-specific way, with implications for computational accounts of psychiatric conditions.
查看此版本解读 →查看此版本原文 ↗