跳到正文
研踪PaperTrail
← 研踪 PaperTrail
正在查看已观测的 v1。查看最新 v2 →

arXiv:2611.19669 · v1

IA-MoE: An Intensity-Aware Mixture of Experts for DNN Inference

首次提交 2026-11-15 07:47
版本更新 2026-11-15 07:47
雷达观测 2026-11-17 12:06
arXiv ↗PDF ↗

摘要

With the ever-increasing parameter scale and structural complexity of deep neural network (DNN) models, traditional cloud-based inference encounters bottlenecks in performance, privacy protection, and energy efficiency. While edge computing and edge intelligence facilitate the migration of inference tasks to edge servers closer to the data source, existing end-edge-cloud collaborative inference still encounters critical challenges, such as sensitivity to heterogeneous resources, complexity in multi-objective optimization, and adaptability to dynamic workloads. To achieve the joint optimization of end-to-end latency and energy consumption under dynamic inference requests with multi-resource constraints, we propose IA-MoE, an intensity-aware Mixture of Experts (MoE) framework. By modeling the model partitioning problem as a bilateral matching problem between layer tasks and virtual machine resources for each inference request, IA-MoE dynamically partitions DNN models in an edge-cloud hybrid computing environment. We evaluate IA-MoE using three representative DNNs (i.e., AlexNet, VGG16, ResNet50) on two public inference datasets in an enhanced CloudSimSDN simulation environment. Experimental results demonstrate that IA-MoE outperforms baseline methods regarding weighted normalized metrics for end-to-end latency and energy consumption. To our knowledge, this is the first study to introduce the MoE paradigm for DNN partitioning and collaborative inference in hybrid computing environments, providing a practical solution for efficient inference in artificial intelligence applications.

作者备注:8 pages, 9 figures. Accepted manuscript of a paper published at IEEE ICNSC 2025

期刊信息:Proceedings of the 2025 IEEE International Conference on Networking, Sensing and Control (ICNSC), 2025

AI 论文解读

基于摘要 · 非全文审阅

此版本尚未生成 AI 解读。解读由后台任务生成并按论文版本保存,登录后共享已有结果。

版本差异

仅对比实际观测的元数据,不代表 PDF 正文或实验变化。版本可能不连续;带删除线为移除,下划线为新增。

摘要变化

With theIn everedge-increasing parametercloud scalecollaborative andinference, structuralthe complexitylayers of a deep neural network (DNN) models,are traditionalplaced cloud-basedon inferenceedge encountersand bottleneckscloud invirtual performance,machines privacy(VMs) protection,for andevery energyrequest, efficiency.while Whilethe edge computingload and edgethe intelligencecross-cloud facilitatebandwidth thechange migrationover oftime. inferenceWe taskspropose toIA-MoE, edgean serversintensity-aware closerMixture toof theExperts data(MoE) source,framework existingthat scores each layer-VM pair by end-edgeto-cloud collaborativeend inferencedelay stilland encountersenergy criticaland challenges,places suchthe aslayers sensitivityone toat heterogeneousa resources,time complexityon inthe multibest-objectivescoring optimization,VM. andSuch adaptabilitygreedy toplacement dynamicis workloads.myopic Towhen achievetrained theon jointper-layer optimizationlabels: ofoffloading end-to-end latencythe andcloud energyslows consumptionthe underfirst dynamicoffloaded inferencelayer requestswhile withthe multi-resourcegain constraints,appears weonly proposein IA-MoE,later anlayers, intensity-awareso Mixturethe ofpolicy Expertskeeps (MoE)too framework.many Byrequests modelingat the modeledge. partitioningTraining problemon asthe aremaining bilateralrequest matchingtime problemafter betweeneach layerdecision tasksremoves andthis virtualmyopia machineand resourcesreduces forthe eachmean inferencedelay request,by IA12-MoE56% dynamicallyon partitionscongested DNNworkloads. modelsIn inCloudSimSDN ansimulations edge-cloudwith hybridthree computingDNNs environment.and Wetwo evaluatepublic request traces, IA-MoE usinghas threethe representativelowest DNNsmean (i.e.,delay AlexNet,of all compared methods on congested VGG16 workloads, ResNet50)across oncross-cloud twobandwidths publicof inference150-600 datasetsMbps, influctuating anbandwidth enhancedand CloudSimSDNsaturated simulationedge environmentservers. ExperimentalA resultsrandom demonstrateforest thattrained IAon the same label is 5-MoE27% outperformsslower, baselineand methodsonly regardinga weightedheuristic normalizedthat metricssends forwhole end-requests to-end latencythe andcloud uses less energy, consumptionat up to 20 times the delay. ToWhen ourthe knowledgeedge is lightly loaded, thisoffloading isdoes thenot firstpay studyand tosimple introduceheuristics theperform as well. IA-MoE paradigmalso forbeats DNNa partitioningDouble andDeep collaborativeQ-Network inference(DQN) agent with the same features in hybridall computing19 environmentstested settings, providingwith a practicalmean solutiondelay forlower efficientby inferencea factor of 1.3 to 10.5, and trains in artificialminutes intelligenceinstead applicationsof hours.

作者备注变化

8Extended pages,version 9of figures.the AcceptedIEEE manuscriptICNSC of2025 paper, with a paperrevised publishedtraining atobjective, IEEEadditional ICNSCbaselines, 2025expanded experimental evaluation, ablation studies, and Roofline analysis

观测到的版本

相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。

v2 · 2026-11-18 01:05 · 最新观测版本

IA-MoE: An Intensity-Aware Mixture of Experts for DNN Inference

In edge-cloud collaborative inference, the layers of a deep neural network (DNN) are placed on edge and cloud virtual machines (VMs) for every request, while the edge load and the cross-cloud bandwidth change over time. We propose IA-MoE, an intensity-aware Mixture of Experts (MoE) framework that scores each layer-VM pair by end-to-end delay and energy and places the layers one at a time on the best-scoring VM. Such greedy placement is myopic when trained on per-layer labels: offloading to the cloud slows the first offloaded layer while the gain appears only in later layers, so the policy keeps too many requests at the edge. Training on the remaining request time after each decision removes this myopia and reduces the mean delay by 12-56% on congested workloads. In CloudSimSDN simulations with three DNNs and two public request traces, IA-MoE has the lowest mean delay of all compared methods on congested VGG16 workloads, across cross-cloud bandwidths of 150-600 Mbps, fluctuating bandwidth and saturated edge servers. A random forest trained on the same label is 5-27% slower, and only a heuristic that sends whole requests to the cloud uses less energy, at up to 20 times the delay. When the edge is lightly loaded, offloading does not pay and simple heuristics perform as well. IA-MoE also beats a Double Deep Q-Network (DQN) agent with the same features in all 19 tested settings, with a mean delay lower by a factor of 1.3 to 10.5, and trains in minutes instead of hours.

查看此版本解读 →查看此版本原文 ↗
v1 · 2026-11-15 07:47

IA-MoE: An Intensity-Aware Mixture of Experts for DNN Inference

With the ever-increasing parameter scale and structural complexity of deep neural network (DNN) models, traditional cloud-based inference encounters bottlenecks in performance, privacy protection, and energy efficiency. While edge computing and edge intelligence facilitate the migration of inference tasks to edge servers closer to the data source, existing end-edge-cloud collaborative inference still encounters critical challenges, such as sensitivity to heterogeneous resources, complexity in multi-objective optimization, and adaptability to dynamic workloads. To achieve the joint optimization of end-to-end latency and energy consumption under dynamic inference requests with multi-resource constraints, we propose IA-MoE, an intensity-aware Mixture of Experts (MoE) framework. By modeling the model partitioning problem as a bilateral matching problem between layer tasks and virtual machine resources for each inference request, IA-MoE dynamically partitions DNN models in an edge-cloud hybrid computing environment. We evaluate IA-MoE using three representative DNNs (i.e., AlexNet, VGG16, ResNet50) on two public inference datasets in an enhanced CloudSimSDN simulation environment. Experimental results demonstrate that IA-MoE outperforms baseline methods regarding weighted normalized metrics for end-to-end latency and energy consumption. To our knowledge, this is the first study to introduce the MoE paradigm for DNN partitioning and collaborative inference in hybrid computing environments, providing a practical solution for efficient inference in artificial intelligence applications.

查看此版本解读 →查看此版本原文 ↗