熵风险感知的蒙特卡洛树搜索
Entropic Risk-Aware Monte Carlo Tree Search
Pedro P. Santos, Jacopo Silvestrin, Manuel Lopes, Alberto Sardinha, Francisco S. Melo
AI 速读 基于摘要
提出一种可证明正确的蒙特卡洛树搜索算法,用于求解以熵风险度量(ERM)为目标的风险感知马尔可夫决策过程。
cs.LG
查看原文摘要
We propose a provably correct Monte Carlo tree search (MCTS) algorithm for solving risk-aware Markov decision processes (MDPs) with entropic risk measure (ERM) objectives. We provide a non-asymptotic analysis of our proposed algorithm, showing that the algorithm: (i) is correct in the sense that the empirical ERM obtained at the root node converges to the optimal ERM; and (ii) enjoys polynomial regret concentration. Our algorithm successfully exploits the dynamic programming formulations for solving risk-aware MDPs with ERM objectives introduced by previous works in the context of an upper confidence bound-based tree search algorithm. Finally, we provide a set of illustrative experiments to evaluate our risk-aware MCTS method, showing it successfully trades off between risk-neutral and risk-averse behavior.