arXiv:2603.05066 · v4

Reward-Conditioned Reinforcement Learning

首次提交 2026-03-05 19:29

版本更新 2026-11-11 06:09

雷达观测 2026-11-13 11:12

摘要

Sample-efficient reinforcement learning typically uses a fixed reward function both to guide exploration and to supervise learning. However, the same experience can provide supervision under multiple related reward specifications, even when the objective of interest remains fixed. We introduce Reward-Conditioned Reinforcement Learning (RCRL), an off-policy approach that collects experience under a nominal objective while training a reward-conditioned agent using counterfactual rewards recomputed from replay. RCRL constructs alternative reward specifications by perturbing the nominal reward, providing diverse supervision without additional environment interaction. Across single-task, multi-task, and vision-based control benchmarks, RCRL improves nominal task performance and sample efficiency when integrated with strong RL backbones. As a secondary capability, the learned policies support zero-shot behavioral adjustment toward related objectives without further training. These results demonstrate that diverse reward supervision can improve learning of a predefined objective while also enabling policy steerability.

作者备注：NeurIPS 2026