面向稀疏奖励任务与多样奖励塑形的零样本协调
Zero Shot Coordination for Sparse Reward Tasks with Diverse Reward Shapings
Keenan Powell, Peihong Yu, Pratap Tokekar
AI 速览：作者提出用随机奖励塑形训练方法集成，以提升智能体与采用不同奖励塑形但稀疏目标相同的未知伙伴在稀疏奖励任务中的零样本协调表现。
Many Multi-Agent Reinforcement Learning (MARL) agents fail to adapt properly to cooperating with agents trained with the same objectives but different seeds, algorithms, or other training differences. This is the problem of Zero-Shot Coordination (ZSC), which focuses on training agents to cooperate well with unknown agents. ZSC has been studied for a variety of tabular cases and simple games such as Hanabi, achieving excellent results. However, existing solutions to ZSC only consider identical rewards for your trained agents and all future partners. This is not realistic for the trained agents, as they do not consider the problem of cooperating with agents that have identical sparse objectives but shape the rewards for those objectives in different manner. To address this issue, we show how to train an ensemble of methods using randomized reward shapings chosen using 4 selection algorithms. Experiments done on the Overcooked environment demonstrate consistent improvements of 62.2%-119.2% in sparse reward over baseline ZSC algorithms when playing with agents that have identical sparse rewards but different reward shapings.