Graph Attention Reinforcement Learning for Multicast Routing and Age-Optimal Scheduling
Yanning Zhang, Guocheng Liao, Shengbin Cao, Ning Yang, Nikolaos Pappas, Meng Zhang
原文摘要
Multicast routing is essential for real-time group applications, such as video streaming, virtual reality, and metaverse platforms, where the Age of Information (AoI) acts as a crucial metric to assess information timeliness. This paper studies dynamic multicast networks with the objective of minimizing the expected average AoI by jointly optimizing multicast routing and scheduling. The main challenges stem from the intricate coupling between routing and scheduling decisions, the inherent complexity of multicast operations, and the graph representation. We first decompose the original problem into two subtasks amenable to hierarchical reinforcement learning (RL) methods. We propose the first RL framework to address the multicast routing problem, also known as the Steiner Tree problem, by incorporating graph embedding and the successive addition of nodes and links. For graph embedding, we propose the Normalized Graph Attention mechanism (NGAT) framework and establish non-expansiveness of the complete embedding matrix in the Frobenius norm under fixed symmetric row-stochastic attention, while evaluating the learned model's stability and generalization empirically. We validate our framework through experiments on four datasets, including the real-world AS-733 dataset. The results demonstrate that our proposed scheme can be up to 9.85 times more computationally efficient than traditional multicast routing algorithms, achieving approximation ratios of 1.1-1.3. These results are not only comparable to state-of-the-art (SOTA) methods but also highlight its superior generalization capabilities, performing effectively on unseen and more complex tasks. Additionally, our age-optimal TGMS algorithm reduces the average weighted AoI by 38.7% and the weighted peak age by 37.1% under various scenarios.
cs.NI