跳到正文
研踪PaperTrail
← 研踪 PaperTrail

arXiv:2607.04164 · v2

BrownoutMoE: Learning Structure-Aware Expert Grouping for Adaptive Mixture-of-Experts Model Serving

首次提交 2026-07-05 16:08
版本更新 2026-11-15 09:44
雷达观测 2026-11-17 12:06
arXiv ↗PDF ↗

摘要

Mixture-of-Experts (MoE) large language models (LLMs) are increasingly served on multi-GPU clusters in cloud data centers, where inference must stay accurate and responsive under bursty demand. Expert access in MoE inference is highly imbalanced: a few hot experts process most tokens, whereas many cold experts fragment kernels and leave GPU parallelism underused. Existing serving systems optimize runtime execution but keep the original expert organization. We propose BrownoutMoE, a structure-aware serving framework inspired by the brownout paradigm in cloud computing. It learns layer-wise expert grouping offline with reinforcement learning, using the post-distillation error as the reward. Grouping-consistent distillation trains one united expert per group, and an online controller adjusts how often united experts replace cold experts under latency objectives. On eight A100 GPUs, BrownoutMoE reduces accuracy degradation by up to 71.4% relative to index-order grouping, improves unfused throughput by up to 2.24x over vLLM, and lowers saturated decode latency by 38%.

作者备注:25 pages, 11 figures

AI 论文解读

基于摘要 · 非全文审阅

此版本尚未生成 AI 解读。解读由后台任务生成并按论文版本保存,登录后共享已有结果。

版本差异

本站首次收录为 v2,暂无更早版本可对照。

观测到的版本

相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。

v2 · 2026-11-15 09:44 · 最新观测版本

BrownoutMoE: Learning Structure-Aware Expert Grouping for Adaptive Mixture-of-Experts Model Serving

Mixture-of-Experts (MoE) large language models (LLMs) are increasingly served on multi-GPU clusters in cloud data centers, where inference must stay accurate and responsive under bursty demand. Expert access in MoE inference is highly imbalanced: a few hot experts process most tokens, whereas many cold experts fragment kernels and leave GPU parallelism underused. Existing serving systems optimize runtime execution but keep the original expert organization. We propose BrownoutMoE, a structure-aware serving framework inspired by the brownout paradigm in cloud computing. It learns layer-wise expert grouping offline with reinforcement learning, using the post-distillation error as the reward. Grouping-consistent distillation trains one united expert per group, and an online controller adjusts how often united experts replace cold experts under latency objectives. On eight A100 GPUs, BrownoutMoE reduces accuracy degradation by up to 71.4% relative to index-order grouping, improves unfused throughput by up to 2.24x over vLLM, and lowers saturated decode latency by 38%.

查看此版本解读 →查看此版本原文 ↗