收藏切换
Bandwidth-Efficient Edge-Cloud Collaborative Inference Method for Evolving Large Language Models
收藏切换
Yuchen Li1, Rui Kong1, Xinran Chen2, Dawei Yin2, Linghe Kong1, *, Guihai Chen1
Journal of Computer Research and Development | 2026, 63(8) : 2024 - 2036
Less
收藏切换
Journal of Computer Research and Development | 2026, 63(8): 2024-2036
• Special Issue on AI Computility: Systems and Key Applications •
Bandwidth-Efficient Edge-Cloud Collaborative Inference Method for Evolving Large Language Models
Full
Yuchen Li1, Rui Kong1, Xinran Chen2, Dawei Yin2, Linghe Kong1, *, Guihai Chen1
Affiliations
  • 1School of Computer Science, Shanghai Jiao Tong University, Shanghai 200240
  • 2Baidu.com Times Technology (Beijing) Co., Ltd., Beijing 100085
  • born in 1997. PhD. His main research interests include data mining and LLMs

    born in 1998. PhD. His main research interests include data mininig and LLMs

    born in 1998. PhD. His main research interests include information retrieval and LLMs

    born in 1984. PhD. His main research interests include data mining and information retrieval

    born in 1983. PhD, professor, PhD supervisor. His main research interests include Internet of things and AI

    born in 1963. PhD, professor, PhD supervisor. His main research interest includes computer networks

About Author:

born in 1997. PhD. His main research interests include data mining and LLMs

born in 1998. PhD. His main research interests include data mininig and LLMs

born in 1998. PhD. His main research interests include information retrieval and LLMs

born in 1984. PhD. His main research interests include data mining and information retrieval

born in 1983. PhD, professor, PhD supervisor. His main research interests include Internet of things and AI

born in 1963. PhD, professor, PhD supervisor. His main research interest includes computer networks

Published: 2026-08-01 doi: 10.7544/issn1000-1239.202660136
Outline
收藏切换

Deploying large language models (LLMs) in mobile and edge computing environments faces multiple challenges, including limited on-device resources, scarce wireless bandwidth, and the continual evolution of cloud-side models. Although speculative decoding based edge-cloud collaborative inference can reduce end-to-end generation latency by using a lightweight draft model at the edge and a target model in the cloud for parallel verification, existing methods usually rely on a tightly coupled relationship between the two models. In practical systems, frequent updates of the cloud model often require repeated synchronization of the edge-side draft model, leading to substantial communication overhead, higher latency, and limited scalability. To address this issue, ECSpec is proposed as a communication-efficient edge-cloud collaborative inference framework for model-evolving scenarios. Its key idea is a shared backbone architecture that enables a single static edge-side draft model to remain compatible with a family of continuously evolving cloud-side target models, thereby avoiding repeated retraining or weight downloading at the edge and significantly reducing communication and maintenance costs. In addition, a channel-aware adaptive speculation mechanism is introduced to dynamically adjust the draft length according to real-time channel conditions and energy budgets, achieving a better tradeoff between inference efficiency and energy consumption. Experimental results demonstrate that ECSpec delivers more stable and efficient collaborative LLM inference under communication-constrained edge environments.

edge computing  /  edge-cloud collaboration  /  large language models  /  speculative decoding  /  collaborative inference
李豫晨, 孔芮, 陈新冉, 殷大伟, 孔令和, 陈贵海. 面向演进式大语言模型的带宽高效边云协同推理方法. 计算机研究与发展, 2026 , 63 (8) : 2024 -2036 . DOI: 10.7544/issn1000-1239.202660136
Yuchen Li, Rui Kong, Xinran Chen, Dawei Yin, Linghe Kong, Guihai Chen. Bandwidth-Efficient Edge-Cloud Collaborative Inference Method for Evolving Large Language Models[J]. Journal of Computer Research and Development, 2026 , 63 (8) : 2024 -2036 . DOI: 10.7544/issn1000-1239.202660136
Year 2026 volume 63 Issue 8
89
44
Cite this Article
BibTeX
Article Info
doi: 10.7544/issn1000-1239.202660136
  • Receive Date:2026-03-17
  • Online Date:2026-11-17
  • Published:2026-08-01
Article Data
Affiliations
History
  • Received:2026-03-17
  • Revised:2026-04-20
Affiliations
    1School of Computer Science, Shanghai Jiao Tong University, Shanghai 200240
    2Baidu.com Times Technology (Beijing) Co., Ltd., Beijing 100085
References
Share
https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/10.7544/issn1000-1239.202660136
Share to
QR

Scan QR to access full text

Cite this article
BibTeX
Citations
表12种不同金属材料的力学参数
科
Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)
属
Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏
  • BibTeX
  • EndNote
  • RefWorks
  • TxT