收藏切换
Collaborative MoE Inference System over Edge Computility Networks
收藏切换
Shengyuan Ye, Han Liang, Xu Chen*
Journal of Computer Research and Development | 2026, 63(8) : 1988 - 2004
Less
收藏切换
Journal of Computer Research and Development | 2026, 63(8): 1988-2004
• Special Issue on AI Computility: Systems and Key Applications •
Collaborative MoE Inference System over Edge Computility Networks
Full
Shengyuan Ye, Han Liang, Xu Chen*
Affiliations
  • School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou 510006
  • born in 1999. PhD candidate. His main research interests include mobile edge computing, distributed machine learning systems, and large language models

    born in 2001. PhD candidate. His main research interests include edge computing, distributed training systems, and multimodal large language model applications

    born in 1986. Professor. His main research interests include edge computing and large language model systems

About Author:

born in 1999. PhD candidate. His main research interests include mobile edge computing, distributed machine learning systems, and large language models

born in 2001. PhD candidate. His main research interests include edge computing, distributed training systems, and multimodal large language model applications

born in 1986. Professor. His main research interests include edge computing and large language model systems

Published: 2026-08-01 doi: 10.7544/issn1000-1239.202660131
Outline
收藏切换

MoE (mixture of expert) architectures have become a widely recognized paradigm for scaling large language models (LLMs), and have been adopted by many frontier models deployed at the network edge. However, deploying MoE-based LLMs on resource-constrained edge devices remains challenging due to their massive parameter sizes, which often exceed the memory capacity of individual platforms. Existing in-situ MoE serving systems adopt expert swapping strategies that offload inactive experts to secondary storage and load them on demand. As LLMs support increasingly longer context windows with correspondingly longer user inputs, the prefilling phase faces a critical bottleneck: long input sequences activate nearly all experts, and the resulting full-scale expert swapping generates substantial I/O traffic, leading to prolonged TTFT (time to first token) latency. Typical edge environments, such as smart homes and smart factories, comprise multiple trusted interconnected devices, including tablets, laptops, and smart home appliances connected via local area networks. These devices collectively form a pervasive edge computility network (ECN), where each node carries underutilize computing power, memory capacity, and I/O bandwidth. This motivates us to harness the aggregated resources of the ECN through coordinated cross-node scheduling. This paper presents Sirius, a collaborative MoE inference system for edge computility networks that pools the collective resources across ECN nodes to accelerate the prefilling phase and reduce TTFT. Specifically, Sirius proposes three key techniques: 1) an interleaved sequence partitioning strategy that resolves workload imbalance in causal attention computation and maximizes communication-computation overlap; 2) an adaptive token scheduling mechanism that jointly considers computation overhead, swap latency, and expert buffer cache states to fully utilize collective I/O bandwidth while achieving load balance; 3) a speculative expert prefetching scheme that exploits the high similarity between adjacent-layer hidden states to overlap expert loading with computation, effectively hiding I/O latency. Experiments on real-world ECN platforming with Mixtral-8x7B and Qwen3-30B-A3B demonstrate that Sirius achieves 1.2–4.0 times inference throughput improvement over state-of-the-art baselines.

MoE  /  edge computility network  /  collaborative inference  /  sequence parallelism  /  expert swapping
叶盛源, 梁涵, 陈旭. 面向边缘算力网络的混合专家模型协同推理系统. 计算机研究与发展, 2026 , 63 (8) : 1988 -2004 . DOI: 10.7544/issn1000-1239.202660131
Shengyuan Ye, Han Liang, Xu Chen. Collaborative MoE Inference System over Edge Computility Networks[J]. Journal of Computer Research and Development, 2026 , 63 (8) : 1988 -2004 . DOI: 10.7544/issn1000-1239.202660131
Year 2026 volume 63 Issue 8
84
37
Cite this Article
BibTeX
Article Info
doi: 10.7544/issn1000-1239.202660131
  • Receive Date:2026-03-12
  • Online Date:2026-11-17
  • Published:2026-08-01
Article Data
Affiliations
History
  • Received:2026-03-12
  • Revised:2026-04-20
Affiliations
    School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou 510006
References
Share
https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/10.7544/issn1000-1239.202660131
Share to
QR

Scan QR to access full text

Cite this article
BibTeX
Citations
表12种不同金属材料的力学参数
科
Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)
属
Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏
  • BibTeX
  • EndNote
  • RefWorks
  • TxT