← 研踪 PaperTrailarXiv:2604.24549 · v2
GradMAP: Gradient-Based Multi-Agent Proximal Learning for Grid-Edge Flexibility
首次提交 2026-04-27 22:43
版本更新 2026-11-08 21:28
雷达观测 2026-11-10 12:16
摘要
Coordinating large populations of grid-edge devices requires learning methods that remain decentralised in deployment while still respecting three-phase AC distribution-network constraints. This paper proposes gradient-based multi-agent proximal learning (GradMAP) to address this challenge. GradMAP trains independent neural network policies for each agent that makes online decisions without communication. During offline training, GradMAP embeds a differentiable power grid environment and uses implicit differentiation to propagate exact AC power flow gradients. To speed up training while maintaining performance, GradMAP reuses expensive environment gradients through a proximal surrogate within a trust region defined in the more direct policy-output space, instead of the probability distribution space used in PPO. Using a single workstation-class NVIDIA RTX PRO 5000 Blackwell GPU with 48~GB of memory, GradMAP trains 1,000 agents on the IEEE123 network in 15 minutes and 100,000 agents on the IEEE8500 network in 40 minutes, without parameter sharing. In both cases, GradMAP achieve the lowest mean daily maximum voltage violation and the lowest energy cost among all evaluated methods, including standard multi-agent reinforcement learning methods and gradient-based learning methods extended to the multi-agent setting by us. The training time to convergence is also the lowest, with a 5x to more than 10x training speedup compared to other methods.
作者备注：12 pages; under revision for IEEE Transactions on Smart Grid
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-08 21:28 · 最新观测版本
GradMAP: Gradient-Based Multi-Agent Proximal Learning for Grid-Edge Flexibility
Coordinating large populations of grid-edge devices requires learning methods that remain decentralised in deployment while still respecting three-phase AC distribution-network constraints. This paper proposes gradient-based multi-agent proximal learning (GradMAP) to address this challenge. GradMAP trains independent neural network policies for each agent that makes online decisions without communication. During offline training, GradMAP embeds a differentiable power grid environment and uses implicit differentiation to propagate exact AC power flow gradients. To speed up training while maintaining performance, GradMAP reuses expensive environment gradients through a proximal surrogate within a trust region defined in the more direct policy-output space, instead of the probability distribution space used in PPO. Using a single workstation-class NVIDIA RTX PRO 5000 Blackwell GPU with 48~GB of memory, GradMAP trains 1,000 agents on the IEEE123 network in 15 minutes and 100,000 agents on the IEEE8500 network in 40 minutes, without parameter sharing. In both cases, GradMAP achieve the lowest mean daily maximum voltage violation and the lowest energy cost among all evaluated methods, including standard multi-agent reinforcement learning methods and gradient-based learning methods extended to the multi-agent setting by us. The training time to convergence is also the lowest, with a 5x to more than 10x training speedup compared to other methods.
查看此版本解读 →查看此版本原文 ↗