让下一篇重要论文，不再错过。

原文摘要

Pretrained imitation policies have become a strong foundation for robot manipulation, but online adaptation can further improve policy performance within their pretrained behavioral support. A central question is therefore how reinforcement learning (RL) should adapt policies after offline pretraining. Existing lightweight methods commonly apply residual corrections directly in action space. We investigate how the steering interface affects online RL. In this work, we propose Z-Perturbation Reinforcement Learning (ZPRL), an approach that steers pretrained policies through a compact bottleneck latent rather than through policy weights or output actions. Building on an offline-learned bottleneck policy interface introduced by previous work, ZPRL turns this representation into an online RL control space through reward-driven residual perturbations while freezing the pretrained policy. We instantiate ZPRL on flow-matching policies and evaluate it on eight simulation tasks and four real-world tasks. Across diverse manipulation settings, ZPRL improves sample efficiency with competitive or better final performance. In the real world, ZPRL improves the average success rate on four tasks by 33.7 percentage points over imitation base policies. Our qualitative and quantitative results also indicate that ZPRL combines task success with smooth actions without explicit temporal filtering or smoothness regularization. These results suggest that a compact, task-aligned bottleneck latent provides an effective interface for online RL adaptation. Videos and code can be found at https://manutdmoon.github.io/ZPRL/.