Article(id=1329420373529047797, tenantId=1146029695717560320, journalId=1324759914146529478, issueId=1329420206000157303, articleNumber=null, orderNo=null, doi=10.7544/issn1000-1239.202660136, pmid=null, cstr=null, oa=null, hot=null, price=null, onlineType=0, articleFormat=0, articleType=null, articleTypeStr=null, receivedDate=1773676800000, receivedDateStr=2026-03-17, revisedDate=1776614400000, revisedDateStr=2026-04-20, acceptedDate=null, acceptedDateStr=null, onlineDate=1794888116485, onlineDateStr=2026-11-17, pubDate=1785513600000, pubDateStr=2026-08-01, doiRegisterDate=null, doiRegisterDateStr=null, onlineIssueDate=1794888116485, onlineIssueDateStr=2026-11-17, onlineJustAcceptDate=null, onlineJustAcceptDateStr=null, onlineFirstDate=null, onlineFirstDateStr=null, sourceXml=null, magXml=null, createTime=1794888116485, creator=13701087609, updateTime=1794888116485, updator=13701087609, issue=Issue{id=1329420206000157303, tenantId=1146029695717560320, journalId=1324759914146529478, year='2026', volume='63', issue='8', pageStart='1905', pageEnd='2181', issueExtLink='null', onlineDate='null', pubDate='1785513600000', pubDateStr='2026-08-01', beforeIssueId=null, nextIssueId=null, price=null, status=1, issueComplete=1, articleOrder=1, issueType=-1, specialIssue=null, createTime=1794888076542, creator='13701087609', updateTime=1794888076542, updator='13701087609', preIssue=null, nextIssue=null, articleTotal=null, ext=null, issueFiles=null, downloadFileDto=null}, startPage=2024, endPage=2036, ext={EN=ArticleExt(id=1329420373726180086, articleId=1329420373529047797, tenantId=1146029695717560320, journalId=1324759914146529478, language=EN, title=Bandwidth-Efficient Edge-Cloud Collaborative Inference Method for Evolving Large Language Models, columnId=1329420214938219240, journalTitle=Journal of Computer Research and Development, columnName=Special Issue on AI Computility: Systems and Key Applications, runingTitle=null, highlight=null, articleAbstract=

Deploying large language models (LLMs) in mobile and edge computing environments faces multiple challenges, including limited on-device resources, scarce wireless bandwidth, and the continual evolution of cloud-side models. Although speculative decoding based edge-cloud collaborative inference can reduce end-to-end generation latency by using a lightweight draft model at the edge and a target model in the cloud for parallel verification, existing methods usually rely on a tightly coupled relationship between the two models. In practical systems, frequent updates of the cloud model often require repeated synchronization of the edge-side draft model, leading to substantial communication overhead, higher latency, and limited scalability. To address this issue, ECSpec is proposed as a communication-efficient edge-cloud collaborative inference framework for model-evolving scenarios. Its key idea is a shared backbone architecture that enables a single static edge-side draft model to remain compatible with a family of continuously evolving cloud-side target models, thereby avoiding repeated retraining or weight downloading at the edge and significantly reducing communication and maintenance costs. In addition, a channel-aware adaptive speculation mechanism is introduced to dynamically adjust the draft length according to real-time channel conditions and energy budgets, achieving a better tradeoff between inference efficiency and energy consumption. Experimental results demonstrate that ECSpec delivers more stable and efficient collaborative LLM inference under communication-constrained edge environments.

, authors=Yuchen Li1, Rui Kong1, Xinran Chen2, Dawei Yin2, Linghe Kong1, *, Guihai Chen1, authorsList=Yuchen Li, Rui Kong, Xinran Chen, Dawei Yin, Linghe Kong, Guihai Chen, authorCompany=null, correspAuthors=Linghe Kong, authorNote=

born in 1997. PhD. His main research interests include data mining and LLMs

born in 1998. PhD. His main research interests include data mininig and LLMs

born in 1998. PhD. His main research interests include information retrieval and LLMs

born in 1984. PhD. His main research interests include data mining and information retrieval

born in 1983. PhD, professor, PhD supervisor. His main research interests include Internet of things and AI

born in 1963. PhD, professor, PhD supervisor. His main research interest includes computer networks

, correspAuthorsNote=null, copyrightStatement=Copyright © 2026 Journal of Computer Research and Development. All rights reserved., copyrightOwner=null, extLink=null, articleAbsUrl=null, sourceXml=null, magXml=null, pdfUrl=null, pdf=null, pdfFileSize=null, pdfExtLink=null, richHtmlUrl=null, mobilePdfUrl=null, reviewReport=null, pdfFirstPage=null, abstractGraph=null, abstractGraphContent=null, abstractVideo=null, citation=null, cebUrl=null, magXmlContent=null, mapNumber=null, fund=null), CN=ArticleExt(id=1329420375433261835, articleId=1329420373529047797, tenantId=1146029695717560320, journalId=1324759914146529478, language=CN, title=面向演进式大语言模型的带宽高效边云协同推理方法, columnId=1329420215122768618, journalTitle=计算机研究与发展, columnName=AI算力网:系统与关键应用专题, runingTitle=null, highlight=null, articleAbstract=

在移动与边缘计算环境中部署大语言模型面临端侧算力与存储受限、无线带宽紧张以及云端模型持续演进等挑战。现有基于投机解码的边云协同推理虽可借助边缘草稿模型与云端目标模型的并行协作降低生成时延,但通常要求二者紧密匹配。当云端模型频繁更新时,往往需要反复同步边缘模型,导致额外通信开销、时延增加,并削弱系统可扩展性。为此,提出通信高效边云协同推理框架ECSpec。其核心在于共享骨干设计,使静态边缘草稿模型可适配多个不断演进的云端目标模型,从而避免端侧重复训练与权重反复下载,显著降低通信与维护成本。进一步结合时变信道与设备异构特性,设计信道感知自适应投机机制,依据实时信道状态和能耗预算动态调整草稿长度,在效率与能耗之间实现平衡。实验结果表明,ECSpec在通信受限的边缘环境中可实现更稳定、高效的大语言模型协同推理。

, authors=李豫晨1, 孔芮1, 陈新冉2, 殷大伟2, 孔令和1, *, 陈贵海1, authorsList=李豫晨, 孔芮, 陈新冉, 殷大伟, 孔令和, 陈贵海, authorCompany=null, correspAuthors=孔令和, authorNote=

作者贡献声明

李豫晨负责方案整体设计并撰写论文;孔芮、陈新冉负责部分算法思路和实验方法;殷大伟、孔令和、陈贵海提出指导意见并修改论文。

1997年生。博士。主要研究方向为数据挖掘、大语言模型

1998年生。博士。主要研究方向为数据挖掘、大语言模型

1998年生。博士。主要研究方向为信息检索、大语言模型

1984年生。博士。主要研究方向为数据挖掘、信息检索

1983年生。博士,教授,博士生导师。主要研究方向为物联网、人工智能

1963年生。博士,教授,博士生导师。主要研究方向为计算机网络

, correspAuthorsNote=
孔令和()
, copyrightStatement=版权所有 © 2026 计算机研究与发展编辑部, copyrightOwner=null, extLink=null, articleAbsUrl=null, sourceXml=bChvMAoMuzmsaTN5h2haRQ==, magXml=iU2aLEGsOB4qM/Hx7IVkJA==, pdfUrl=null, pdf=wLNNuOlbp3bjRiCh0KOEPg==, pdfFileSize=1265323, pdfExtLink=null, richHtmlUrl=null, mobilePdfUrl=null, reviewReport=null, pdfFirstPage=null, abstractGraph=avXqNwFPhk9FYCkzEgYWBA==, abstractGraphContent=null, abstractVideo=null, citation=null, cebUrl=null, magXmlContent=FzHhnDr7Gkd7iVWBglhQ1A==, mapNumber=null, fund=null)}, authors=[Author(id=1329420375810749203, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, orderNo=0, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=yuchenli1230@gmail.com, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329420375898829589, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, authorId=1329420375810749203, language=EN, stringName=Yuchen Li, firstName=Yuchen, middleName=null, lastName=Li, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1School of Computer Science, Shanghai Jiao Tong University, Shanghai 200240, bio={"content":"

born in 1997. PhD. His main research interests include data mining and LLMs

"}, bioImg=null, bioContent=

born in 1997. PhD. His main research interests include data mining and LLMs

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329420375982715670, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, authorId=1329420375810749203, language=CN, stringName=李豫晨, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1上海交通大学计算机学院 上海 200240, bio={"img":"k7lMQow21lX+THpKjOhukg==","content":"

1997年生。博士。主要研究方向为数据挖掘、大语言模型

"}, bioImg=k7lMQow21lX+THpKjOhukg==, bioContent=

1997年生。博士。主要研究方向为数据挖掘、大语言模型

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329420375647171340, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, xref=1, ext=[AuthorCompanyExt(id=1329420375655559949, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, companyId=1329420375647171340, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1School of Computer Science, Shanghai Jiao Tong University, Shanghai 200240), AuthorCompanyExt(id=1329420375663948558, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, companyId=1329420375647171340, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1上海交通大学计算机学院 上海 200240)])]), Author(id=1329420376054018841, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, orderNo=1, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329420376129516315, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, authorId=1329420376054018841, language=EN, stringName=Rui Kong, firstName=Rui, middleName=null, lastName=Kong, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1School of Computer Science, Shanghai Jiao Tong University, Shanghai 200240, bio={"content":"

born in 1998. PhD. His main research interests include data mininig and LLMs

"}, bioImg=null, bioContent=

born in 1998. PhD. His main research interests include data mininig and LLMs

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329420376205013788, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, authorId=1329420376054018841, language=CN, stringName=孔芮, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1上海交通大学计算机学院 上海 200240, bio={"img":"ISl9yA4kNHukbtXimrR0+w==","content":"

1998年生。博士。主要研究方向为数据挖掘、大语言模型

"}, bioImg=ISl9yA4kNHukbtXimrR0+w==, bioContent=

1998年生。博士。主要研究方向为数据挖掘、大语言模型

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329420375647171340, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, xref=1, ext=[AuthorCompanyExt(id=1329420375655559949, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, companyId=1329420375647171340, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1School of Computer Science, Shanghai Jiao Tong University, Shanghai 200240), AuthorCompanyExt(id=1329420375663948558, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, companyId=1329420375647171340, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1上海交通大学计算机学院 上海 200240)])]), Author(id=1329420376276316958, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, orderNo=2, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329420376339231520, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, authorId=1329420376276316958, language=EN, stringName=Xinran Chen, firstName=Xinran, middleName=null, lastName=Chen, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=2, address=2Baidu.com Times Technology (Beijing) Co., Ltd., Beijing 100085, bio={"content":"

born in 1998. PhD. His main research interests include information retrieval and LLMs

"}, bioImg=null, bioContent=

born in 1998. PhD. His main research interests include information retrieval and LLMs

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329420376410534689, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, authorId=1329420376276316958, language=CN, stringName=陈新冉, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=2, address=2百度时代网络技术(北京)有限公司 北京 100085, bio={"img":"s9npwcW/oePmnChnM+bCOA==","content":"

1998年生。博士。主要研究方向为信息检索、大语言模型

"}, bioImg=s9npwcW/oePmnChnM+bCOA==, bioContent=

1998年生。博士。主要研究方向为信息检索、大语言模型

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329420375726863119, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, xref=2, ext=[AuthorCompanyExt(id=1329420375731057424, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, companyId=1329420375726863119, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2Baidu.com Times Technology (Beijing) Co., Ltd., Beijing 100085), AuthorCompanyExt(id=1329420375747834641, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, companyId=1329420375726863119, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2百度时代网络技术(北京)有限公司 北京 100085)])]), Author(id=1329420376481837859, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, orderNo=3, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329420376548946725, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, authorId=1329420376481837859, language=EN, stringName=Dawei Yin, firstName=Dawei, middleName=null, lastName=Yin, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=2, address=2Baidu.com Times Technology (Beijing) Co., Ltd., Beijing 100085, bio={"content":"

born in 1984. PhD. His main research interests include data mining and information retrieval

"}, bioImg=null, bioContent=

born in 1984. PhD. His main research interests include data mining and information retrieval

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329420376624444198, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, authorId=1329420376481837859, language=CN, stringName=殷大伟, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=2, address=2百度时代网络技术(北京)有限公司 北京 100085, bio={"img":"BxzYZBewriJrRYI4RDcH0A==","content":"

1984年生。博士。主要研究方向为数据挖掘、信息检索

"}, bioImg=BxzYZBewriJrRYI4RDcH0A==, bioContent=

1984年生。博士。主要研究方向为数据挖掘、信息检索

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329420375726863119, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, xref=2, ext=[AuthorCompanyExt(id=1329420375731057424, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, companyId=1329420375726863119, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2Baidu.com Times Technology (Beijing) Co., Ltd., Beijing 100085), AuthorCompanyExt(id=1329420375747834641, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, companyId=1329420375726863119, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2百度时代网络技术(北京)有限公司 北京 100085)])]), Author(id=1329420376687358760, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, orderNo=4, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=linghe.kong@sjtu.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=1, authorType=1, ext={EN=AuthorExt(id=1329420376767050538, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, authorId=1329420376687358760, language=EN, stringName=Linghe Kong, firstName=Linghe, middleName=null, lastName=Kong, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, *, address=1School of Computer Science, Shanghai Jiao Tong University, Shanghai 200240, bio={"content":"

born in 1983. PhD, professor, PhD supervisor. His main research interests include Internet of things and AI

"}, bioImg=null, bioContent=

born in 1983. PhD, professor, PhD supervisor. His main research interests include Internet of things and AI

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329420376838353707, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, authorId=1329420376687358760, language=CN, stringName=孔令和, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, *, address=1上海交通大学计算机学院 上海 200240, bio={"img":"7RMrmDucK9trDiRZMmO97Q==","content":"

1983年生。博士,教授,博士生导师。主要研究方向为物联网、人工智能

"}, bioImg=7RMrmDucK9trDiRZMmO97Q==, bioContent=

1983年生。博士,教授,博士生导师。主要研究方向为物联网、人工智能

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329420375647171340, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, xref=1, ext=[AuthorCompanyExt(id=1329420375655559949, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, companyId=1329420375647171340, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1School of Computer Science, Shanghai Jiao Tong University, Shanghai 200240), AuthorCompanyExt(id=1329420375663948558, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, companyId=1329420375647171340, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1上海交通大学计算机学院 上海 200240)])]), Author(id=1329420376909656877, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, orderNo=5, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329420376976765743, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, authorId=1329420376909656877, language=EN, stringName=Guihai Chen, firstName=Guihai, middleName=null, lastName=Chen, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1School of Computer Science, Shanghai Jiao Tong University, Shanghai 200240, bio={"content":"

born in 1963. PhD, professor, PhD supervisor. His main research interest includes computer networks

"}, bioImg=null, bioContent=

born in 1963. PhD, professor, PhD supervisor. His main research interest includes computer networks

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329420377035486000, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, authorId=1329420376909656877, language=CN, stringName=陈贵海, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1上海交通大学计算机学院 上海 200240, bio={"img":"x9mlIl1ntFRuMmQ1fGxwBg==","content":"

1963年生。博士,教授,博士生导师。主要研究方向为计算机网络

"}, bioImg=x9mlIl1ntFRuMmQ1fGxwBg==, bioContent=

1963年生。博士,教授,博士生导师。主要研究方向为计算机网络

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329420375647171340, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, xref=1, ext=[AuthorCompanyExt(id=1329420375655559949, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, companyId=1329420375647171340, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1School of Computer Science, Shanghai Jiao Tong University, Shanghai 200240), AuthorCompanyExt(id=1329420375663948558, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, companyId=1329420375647171340, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1上海交通大学计算机学院 上海 200240)])])], keywords=[Keyword(id=1329420377152926513, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, language=EN, orderNo=1, keyword=edge computing), Keyword(id=1329420377211646770, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, language=EN, orderNo=2, keyword=edge-cloud collaboration), Keyword(id=1329420377278755635, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, language=EN, orderNo=3, keyword=large language models), Keyword(id=1329420377446527797, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, language=EN, orderNo=4, keyword=speculative decoding), Keyword(id=1329420377513636662, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, language=EN, orderNo=5, keyword=collaborative inference), Keyword(id=1329420377576551223, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, language=CN, orderNo=1, keyword=边缘计算), Keyword(id=1329420377643660088, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, language=CN, orderNo=2, keyword=边云协同), Keyword(id=1329420377702380345, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, language=CN, orderNo=3, keyword=大语言模型), Keyword(id=1329420377782072122, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, language=CN, orderNo=4, keyword=投机解码), Keyword(id=1329420377849180987, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, language=CN, orderNo=5, keyword=协同推理)], refs=[Reference(id=1329420379585622867, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=1, rfOrder=0, authorNames=null, journalName=null, refType=null, unstructuredReference=Achiam J, Adler S, Agarwal S, et al. GPT-4 technical report[J]. arXiv preprint, arXiv: 2303.08774, 2023, articleTitle=null, refAbstract=null), Reference(id=1329420379648537428, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=2, rfOrder=1, authorNames=null, journalName=null, refType=null, unstructuredReference=Touvron H, Martin L, Stone K, et al. LLaMA 2: Open foundation and fine-tuned chat models[J]. arXiv preprint, arXiv: 2307.09288, 2023, articleTitle=null, refAbstract=null), Reference(id=1329420379728229205, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=3, rfOrder=2, authorNames=null, journalName=null, refType=null, unstructuredReference=Team G, Anil R, Borgeaud S, et al. Gemini: A family of highly capable multimodal models[J]. arXiv preprint, arXiv: 2312.11805, 2023, articleTitle=null, refAbstract=null), Reference(id=1329420379791143766, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=4, rfOrder=3, authorNames=null, journalName=null, refType=null, unstructuredReference=Li Yuchen, Cai Hengyi, Kong Rui, et al. Towards AI search paradigm[J]. arXiv preprint, arXiv: 2506.17188, 2025, articleTitle=null, refAbstract=null), Reference(id=1329420379858252631, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=5, rfOrder=4, authorNames=null, journalName=null, refType=null, unstructuredReference=Xiong Haoyi, Bian Jiang, Li Yuchen, et al. When search engine services meet large language models: Visions and challenges[J]. IEEE Transactions on Services Computing, 2024, 17(6): 4558−4577, articleTitle=null, refAbstract=null), Reference(id=1329420379950527320, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=6, rfOrder=5, authorNames=null, journalName=null, refType=null, unstructuredReference=Liu Zechun, Zhao Changsheng, Iandola F, et al. MobileLLM: Optimizing sub-billion parameter language models for on-device use cases[C]//Proc of the 41st Int Conf on Machine Learning (ICML). New York: PMLR, 2024: 32431–32454, articleTitle=null, refAbstract=null), Reference(id=1329420380005053273, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=7, rfOrder=6, authorNames=null, journalName=null, refType=null, unstructuredReference=Xu Dianlei, Li Tong, Li Yong, et al. Edge intelligence: Empowering intelligence to the edge of network[J]. Proceedings of the IEEE, 2021, 109(11): 1778−1837, articleTitle=null, refAbstract=null), Reference(id=1329420380067967834, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=8, rfOrder=7, authorNames=null, journalName=null, refType=null, unstructuredReference=Liu Jing, Du Yao, Yang Kun, et al. Edge-cloud collaborative computing on distributed intelligence and model optimization: A survey[J]. arXiv preprint, arXiv: 2505.01821, 2025, articleTitle=null, refAbstract=null), Reference(id=1329420380139271003, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=9, rfOrder=8, authorNames=null, journalName=null, refType=null, unstructuredReference=Lyu Zhonghao, Li Yuchen, Zhu Guangxu, et al. Rethinking resource management in edge learning: A joint pre-training and fine-tuning design paradigm[J]. IEEE Transactions on Wireless Communications, 2025, 24(2): 1584−1601, articleTitle=null, refAbstract=null), Reference(id=1329420380214768476, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=10, rfOrder=9, authorNames=null, journalName=null, refType=null, unstructuredReference=Leviathan Y, Kalman M, Matias Y. Fast inference from Transformers via speculative decoding[C]//Proc of the 40th Int Conf on Machine Learning (ICML). New York: PMLR, 2023: 19274–19286, articleTitle=null, refAbstract=null), Reference(id=1329420380302848861, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=11, rfOrder=10, authorNames=null, journalName=null, refType=null, unstructuredReference=Li Yuhui, Wei Fangyun, Zhang Chao, et al. EAGLE-2: Faster inference of language models with dynamic draft trees[C]//Proc of Empirical Methods in Natural Language Processing (EMNLP). Stroudsburg, PA: ACL, 2024: 7421–7432, articleTitle=null, refAbstract=null), Reference(id=1329420380390929246, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=12, rfOrder=11, authorNames=null, journalName=null, refType=null, unstructuredReference=Cai Tianle, Li Yuhong, Geng Zhengyang, et al. Medusa: Simple LLM inference acceleration framework with multiple decoding heads[J]. arXiv preprint, arXiv: 2401.10774, 2024, articleTitle=null, refAbstract=null), Reference(id=1329420380453843807, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=13, rfOrder=12, authorNames=null, journalName=null, refType=null, unstructuredReference=Chen C, Borgeaud S, Irving G, et al. Accelerating large language model decoding with speculative sampling[J]. arXiv preprint, arXiv: 2302.01318, 2023, articleTitle=null, refAbstract=null), Reference(id=1329420380533535584, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=14, rfOrder=13, authorNames=null, journalName=null, refType=null, unstructuredReference=Li Yuhui, Wei Fangyun, Zhang Chao, et al. EAGLE: Speculative sampling requires rethinking feature uncertainty[C]//Proc of the Int Conf on Machine Learning (ICML). New York: PMLR, 2024: 28935–28948, articleTitle=null, refAbstract=null), Reference(id=1329420380613227361, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=15, rfOrder=14, authorNames=null, journalName=null, refType=null, unstructuredReference=Yang Nan, Ge Tao, Wang Liang, et al. Inference with reference: Lossless acceleration of large language models[J]. arXiv preprint, arXiv: 2304.04487, 2023, articleTitle=null, refAbstract=null), Reference(id=1329420380688724834, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=16, rfOrder=15, authorNames=null, journalName=null, refType=null, unstructuredReference=He Zhenyu, Zhong Zexuan, Cai Tianle, et al. REST: Retrieval-based speculative decoding[C]//Proc of the Conf of the North American Chapter of the Association for Computational Linguistics (NACCL). Stroudsburg, PA: ACL, 2024: 1582–1595, articleTitle=null, refAbstract=null), Reference(id=1329420380768416611, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=17, rfOrder=16, authorNames=null, journalName=null, refType=null, unstructuredReference=Hu E J, Shen Y, Wallis P, et al. LoRA: Low-rank adaptation of large language models[C]//Proc of the Int Conf on Learning Representations (ICLR). Open Access: OpenReview. net, 2022: 1–13, articleTitle=null, refAbstract=null), Reference(id=1329420380831331172, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=18, rfOrder=17, authorNames=null, journalName=null, refType=null, unstructuredReference=Lyu Zhonghao, Zhu Guangxu, Xu Jie, et al. Semantic communications for image recovery and classification via deep joint source and channel coding[J]. IEEE Transactions on Wireless Communications, 2024, 23(8): 8388−8404, articleTitle=null, refAbstract=null), Reference(id=1329420380898440037, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=19, rfOrder=18, authorNames=null, journalName=null, refType=null, unstructuredReference=Kang Yiping, Hauswald J, Gao Cao, et al. Neurosurgeon: Collaborative intelligence between the cloud and mobile edge[C]//Proc of the 22nd Int Conf on Architectural Support for Programming Languages and Operating Systems (ASPLOS). New York: ACM, 2017: 615–629, articleTitle=null, refAbstract=null), Reference(id=1329420380961354598, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=20, rfOrder=19, authorNames=null, journalName=null, refType=null, unstructuredReference=Matsubara Y, Levorato M, Restuccia F. Split computing and early exit for deep learning applications: Survey and research challenges[J]. ACM Computing Surveys, 2024, 55(5): 1−30, articleTitle=null, refAbstract=null), Reference(id=1329420381020074855, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=21, rfOrder=20, authorNames=null, journalName=null, refType=null, unstructuredReference=Borzunov A, Baranchuk D, Dettmers T, et al. Petals: Collaborative inference and fine-tuning of large models[C]//Proc of Workshop on Broadening Research Collaborations. Stroudsburg, PA: ACL, 2022: 558–568, articleTitle=null, refAbstract=null), Reference(id=1329420381082989416, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=22, rfOrder=21, authorNames=null, journalName=null, refType=null, unstructuredReference=Ryabinin M, Dettmers T, Diskin M, et al. SWARM parallelism: Training large models can be surprisingly communication-efficient[C]//Proc of the Int Conf on Machine Learning (ICML). New York: PMLR, 2023: 29416–29440, articleTitle=null, refAbstract=null), Reference(id=1329420381154292585, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=23, rfOrder=22, authorNames=null, journalName=null, refType=null, unstructuredReference=Lyu Zhonghao, Xiao Ming, Xu Jie, et al. The larger the merrier? Efficient large AI model inference in wireless edge networks[J]. IEEE Journal on Selected Areas in Communications, 2025, 44: 2839−2853, articleTitle=null, refAbstract=null), Reference(id=1329420381213012842, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=24, rfOrder=23, authorNames=null, journalName=null, refType=null, unstructuredReference=Frantar E, Ashkboos S, Hoefler T, et al. GPTQ: Accurate post-training compression for generative pretrained transformers[C]//Proc of the Int Conf on Learning Representations (ICLR). (ICLR). Open Access: OpenReview. net, 2023: 1–16, articleTitle=null, refAbstract=null), Reference(id=1329420381271733099, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=25, rfOrder=24, authorNames=null, journalName=null, refType=null, unstructuredReference=Lin Ji, Tang Jiaming, Tang Haotian, et al. AWQ: Activation-aware weight quantization for on-device LLM compression and acceleration[J]. GetMobile: Mobile Computing and Communications, 2025, 28(4): 12−17, articleTitle=null, refAbstract=null), Reference(id=1329420381326259052, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=26, rfOrder=25, authorNames=null, journalName=null, refType=null, unstructuredReference=Li Muyang, Cai Tianle, Cao Jiaxin, et al. DistriFusion: Distributed parallel inference for high-resolution diffusion models[C]//Proc of the IEEE/CVF Conf on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ: IEEE, 2024: 7183–7193, articleTitle=null, refAbstract=null), Reference(id=1329420381384979309, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=27, rfOrder=26, authorNames=null, journalName=null, refType=null, unstructuredReference=过敏意. 大模型时代网络基础设施的机遇与挑战[J]. 计算机研究与发展, 2024, 61(11): 2663−2663, articleTitle=null, refAbstract=null), Reference(id=1329420381447893870, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=27, rfOrder=27, authorNames=null, journalName=null, refType=null, unstructuredReference=Guo Minyi. Opportunities and challenges of network infrastructure in the era of large language model[J]. Journal of Computer Research and Development, 2024, 61(11): 2663−2663 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1329420381510808431, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=28, rfOrder=28, authorNames=null, journalName=null, refType=null, unstructuredReference=Park Jihong, Samarakoon S, Bennis M, et al. Wireless network intelligence at the edge[J]. Proceedings of the IEEE, 2019, 107(11): 2204−2239, articleTitle=null, refAbstract=null), Reference(id=1329420381573722992, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=29, rfOrder=29, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhu Guangxu, Liu Dongzhu, Du Yuqing, et al. Toward an intelligent edge: Wireless communication meets machine learning[J]. IEEE Communications Magazine, 2020, 58(1): 19−25, articleTitle=null, refAbstract=null), Reference(id=1329420381640831857, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=30, rfOrder=30, authorNames=null, journalName=null, refType=null, unstructuredReference=Xin Ji, Tang R, Lee J, et al. DeeBERT: Dynamic early exiting for accelerating BERT inference[C]//Proc of the 58th Annual Meeting of the Association for Computational Linguistics (ACL). Stroudsburg, PA: ACL, 2020: 2246–2251, articleTitle=null, refAbstract=null), Reference(id=1329420381712135026, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=31, rfOrder=31, authorNames=null, journalName=null, refType=null, unstructuredReference=Laskaridis S, Venieris S I, Almeida M, et al. SPINN: Synergistic progressive inference of neural networks over device and cloud[C]//Proc of the Annual Int Conf on Mobile Computing and Networking (MobiCom). New York: ACM, 2020: 1–15, articleTitle=null, refAbstract=null), Reference(id=1329420381775049587, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=32, rfOrder=32, authorNames=null, journalName=null, refType=null, unstructuredReference=Ning Jiahong, Zheng Ce, Yang Tingting. DSSD: Efficient edge-device deployment and collaborative inference via distributed split speculative decoding[C]//Proc of ICML on Machine Learning for Wireless Communication and Networks (ML4Wireless). New York: PMLR, 2025: 1–8, articleTitle=null, refAbstract=null), Reference(id=1329420381837964148, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=33, rfOrder=33, authorNames=null, journalName=null, refType=null, unstructuredReference=Malladi S, Gao T, Nichani E, et al. Fine-tuning language models with just forward passes[C]//Proc of the 37th Int Conf on Neural Information Processing Systems (NIPS). New York: Curran Associates, 2023: 53038–53075, articleTitle=null, refAbstract=null), Reference(id=1329420381938627445, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=34, rfOrder=34, authorNames=null, journalName=null, refType=null, unstructuredReference=Gu Yuxian, Dong Li, Wei Furu, et al. MiniLLM: Knowledge distillation of large language models[C]//Proc of the Int Conf on Learning Representations (ICLR). Open Access: OpenReview. net, 2024: 1–24, articleTitle=null, refAbstract=null), Reference(id=1329420382018319222, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=35, rfOrder=35, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhao Yao, Xie Zhitian, Liang Chen, et al. Lookahead: An inference acceleration framework for large language model with lossless generation accuracy[C]//Proc of the 30th ACM SIGKDD Conf on Knowledge Discovery and Data Mining. New York: ACM, 2024: 6344–6355, articleTitle=null, refAbstract=null)], funds=null, companyList=[AuthorCompany(id=1329420375647171340, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, xref=1, ext=[AuthorCompanyExt(id=1329420375655559949, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, companyId=1329420375647171340, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1School of Computer Science, Shanghai Jiao Tong University, Shanghai 200240), AuthorCompanyExt(id=1329420375663948558, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, companyId=1329420375647171340, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1上海交通大学计算机学院 上海 200240)]), AuthorCompany(id=1329420375726863119, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, xref=2, ext=[AuthorCompanyExt(id=1329420375731057424, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, companyId=1329420375726863119, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2Baidu.com Times Technology (Beijing) Co., Ltd., Beijing 100085), AuthorCompanyExt(id=1329420375747834641, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, companyId=1329420375726863119, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2百度时代网络技术(北京)有限公司 北京 100085)])], figs=[ArticleFig(id=1329420377975010108, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, language=EN, label=Figure 1, caption=System model of ECSpec, figureFileSmall=4n1QA1fcMEUcTIU8gDdsmQ==, figureFileBig=avXqNwFPhk9FYCkzEgYWBA==, tableContent=null), ArticleFig(id=1329420378037924670, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, language=CN, label=图1, caption=ECSpec系统模型, figureFileSmall=4n1QA1fcMEUcTIU8gDdsmQ==, figureFileBig=avXqNwFPhk9FYCkzEgYWBA==, tableContent=null), ArticleFig(id=1329420378222474047, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, language=EN, label=Figure 2, caption=End-to-end latency comparison of four algorithms on GSM8K, figureFileSmall=22RcK8wVX6yL0/AY7wPR6g==, figureFileBig=DMmbLD6CtAO7IUrEeWm/og==, tableContent=null), ArticleFig(id=1329420378285388608, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, language=CN, label=图2, caption=GSM8K 上4种算法的端到端时延对比, figureFileSmall=22RcK8wVX6yL0/AY7wPR6g==, figureFileBig=DMmbLD6CtAO7IUrEeWm/og==, tableContent=null), ArticleFig(id=1329420378344108866, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, language=EN, label=Figure 3, caption=Impact of speculative stride $ K $ on latency, figureFileSmall=bB/OcsxRqTXUZz99rOdraA==, figureFileBig=FkoYTsn5JbABPvm8GaR3hQ==, tableContent=null), ArticleFig(id=1329420378398634819, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, language=CN, label=图3, caption=投机步长$ K $对时延的影响, figureFileSmall=bB/OcsxRqTXUZz99rOdraA==, figureFileBig=FkoYTsn5JbABPvm8GaR3hQ==, tableContent=null), ArticleFig(id=1329420378461549380, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, language=EN, label=Figure 4, caption=Energy consumption breakdown analysis on mobile devices, figureFileSmall=oZ6ugA2duv4Xlt2wqIm5OQ==, figureFileBig=4i7UzKN1IYTo10FbrPjfDw==, tableContent=null), ArticleFig(id=1329420378541241157, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, language=CN, label=图4, caption=移动设备上的能耗构成分析, figureFileSmall=oZ6ugA2duv4Xlt2wqIm5OQ==, figureFileBig=4i7UzKN1IYTo10FbrPjfDw==, tableContent=null), ArticleFig(id=1329420378633515846, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, language=EN, label=Table 1, caption=

Simulated Latency for Synchronizing Draft Models over Wireless Networks

, figureFileSmall=null, figureFileBig=null, tableContent=
网络类型带宽/Mbps每个用户的同
步时间/min
1000个用户同时同步
可扩展性的表现
Public WiFi10约48不可拓展
4G LTE50约9.5高度拥塞
5G Mid-Band300约1.6中度负载
), ArticleFig(id=1329420378692236103, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, language=CN, label=表1, caption=

在无线网络上同步草稿模型的模拟延迟

, figureFileSmall=null, figureFileBig=null, tableContent=
网络类型带宽/Mbps每个用户的同
步时间/min
1000个用户同时同步
可扩展性的表现
Public WiFi10约48不可拓展
4G LTE50约9.5高度拥塞
5G Mid-Band300约1.6中度负载
), ArticleFig(id=1329420378771927880, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, language=EN, label=Table 2, caption=

Impact of Target Model Evolution on Fixed Draft Model Performance

, figureFileSmall=null, figureFileBig=null, tableContent=
目标模型任务类型接受率
LLaMA-2-70B-Base通用0.72
LLaMA-2-70B-Math(LoRA)数学0.45
LLaMA-2-70B-Code(Full)编程0.18
), ArticleFig(id=1329420378843231049, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, language=CN, label=表2, caption=

目标模型演进对固定草稿模型性能的影响

, figureFileSmall=null, figureFileBig=null, tableContent=
目标模型任务类型接受率
LLaMA-2-70B-Base通用0.72
LLaMA-2-70B-Math(LoRA)数学0.45
LLaMA-2-70B-Code(Full)编程0.18
), ArticleFig(id=1329420378922922826, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, language=EN, label=Table 3, caption=

End-to-End Latency for Diverse Speculative Decoding Methods on Six Datasets (T=0)

, figureFileSmall=null, figureFileBig=null, tableContent=
数据集网络类型Cloud-Only/msLookahead/msStandard SD/msMedusa-1/msEAGLE-2/msDSSD/msECSpec/ms
GSM8K5G432.0415.0392.0205.0182.0305.0220.0
4G595.0580.0805.0480.0445.0482.0325.0
WiFi1220.01205.01850.01455.01420.0950.0625.0
Natural Questions5G415.0390.0345.0192.0168.0275.0202.0
4G570.0552.0685.0435.0405.0452.0295.0
WiFi1185.01165.01680.01380.01352.0905.0582.0
Natural Questions5G428.0402.0360.0198.0175.0288.0212.0
4G582.0568.0735.0455.0422.0468.0308.0
WiFi1208.01190.01765.01425.01390.0932.0600.0
MT-Bench5G420.0398.0358.0202.0178.0295.0215.0
4G578.0562.0742.0462.0430.0475.0312.0
WiFi1192.01175.01780.01440.01405.0945.0615.0
WMT14 DE-EN5G418.0395.0350.0195.0172.0282.0208.0
4G575.0560.0720.0445.0415.0460.0302.0
WiFi1188.01168.01725.01405.01372.0918.0592.0
CNN/DailyMail5G425.0400.0355.0198.0175.0285.0210.0
4G582.0565.0728.0452.0420.0465.0306.0
WiFi1205.01180.01750.01420.01382.0924.0598.0
), ArticleFig(id=1329420379002614603, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, language=CN, label=表3, caption=

不同的投机解码方法在6个数据集上的端到端时延(T=0)

, figureFileSmall=null, figureFileBig=null, tableContent=
数据集网络类型Cloud-Only/msLookahead/msStandard SD/msMedusa-1/msEAGLE-2/msDSSD/msECSpec/ms
GSM8K5G432.0415.0392.0205.0182.0305.0220.0
4G595.0580.0805.0480.0445.0482.0325.0
WiFi1220.01205.01850.01455.01420.0950.0625.0
Natural Questions5G415.0390.0345.0192.0168.0275.0202.0
4G570.0552.0685.0435.0405.0452.0295.0
WiFi1185.01165.01680.01380.01352.0905.0582.0
Natural Questions5G428.0402.0360.0198.0175.0288.0212.0
4G582.0568.0735.0455.0422.0468.0308.0
WiFi1208.01190.01765.01425.01390.0932.0600.0
MT-Bench5G420.0398.0358.0202.0178.0295.0215.0
4G578.0562.0742.0462.0430.0475.0312.0
WiFi1192.01175.01780.01440.01405.0945.0615.0
WMT14 DE-EN5G418.0395.0350.0195.0172.0282.0208.0
4G575.0560.0720.0445.0415.0460.0302.0
WiFi1188.01168.01725.01405.01372.0918.0592.0
CNN/DailyMail5G425.0400.0355.0198.0175.0285.0210.0
4G582.0565.0728.0452.0420.0465.0306.0
WiFi1205.01180.01750.01420.01382.0924.0598.0
), ArticleFig(id=1329420379103277901, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, language=EN, label=Table 4, caption=

End-to-End Latency for Diverse Speculative Decoding Methods on Six Datasets (T=1)

, figureFileSmall=null, figureFileBig=null, tableContent=
数据集网络类型Cloud-Only/msLookahead/msStandard SD/msMedusa-1/msEAGLE-2/msDSSD/msECSpec/ms
GSM8K5G435.0420.0405.0272.0245.0345.0202.0
4G598.0585.0825.0665.0592.0540.0295.0
WiFi1225.01210.01880.01852.01680.01150.0595.0
Natural Questions5G418.0395.0355.0290.0258.0332.0194.0
4G575.0560700.0655.0590.0528.0285.0
WiFi1190.01172.01700.01820.01665.01110.0580.0
Natural Questions5G430.0408.0370.0285.0255.0338.0200.0
4G588.0575.0750.0660.0595.0532.0290.0
WiFi1215.01198.01785.01840.01675.01130.0592.0
MT-Bench5G428.0405.0368285.0252.0329.0198.0
4G585.0570.0760.0650.0585522.0286.0
WiFi1210.01195.01795.01805.01652.01100.0588.0
WMT14 DE-EN5G420.0400.0360.0295.0260.0335.0195.0
4G578.0565.0735.0675.0605.0530.0282.0
WiFi1195.01180.01740.01900.01690.01120.0582.0
CNN/DailyMail5G425.0402.0365.0300.0265.0338.0196.0
4G582.0568.0740.0680.0610.0535.0285.0
WiFi1200.01185.01765.01915.01700.01125.0586.0
), ArticleFig(id=1329420379174581070, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, language=CN, label=表4, caption=

不同的投机解码方法在6个数据集上的端到端时延(T=1)

, figureFileSmall=null, figureFileBig=null, tableContent=
数据集网络类型Cloud-Only/msLookahead/msStandard SD/msMedusa-1/msEAGLE-2/msDSSD/msECSpec/ms
GSM8K5G435.0420.0405.0272.0245.0345.0202.0
4G598.0585.0825.0665.0592.0540.0295.0
WiFi1225.01210.01880.01852.01680.01150.0595.0
Natural Questions5G418.0395.0355.0290.0258.0332.0194.0
4G575.0560700.0655.0590.0528.0285.0
WiFi1190.01172.01700.01820.01665.01110.0580.0
Natural Questions5G430.0408.0370.0285.0255.0338.0200.0
4G588.0575.0750.0660.0595.0532.0290.0
WiFi1215.01198.01785.01840.01675.01130.0592.0
MT-Bench5G428.0405.0368285.0252.0329.0198.0
4G585.0570.0760.0650.0585522.0286.0
WiFi1210.01195.01795.01805.01652.01100.0588.0
WMT14 DE-EN5G420.0400.0360.0295.0260.0335.0195.0
4G578.0565.0735.0675.0605.0530.0282.0
WiFi1195.01180.01740.01900.01690.01120.0582.0
CNN/DailyMail5G425.0402.0365.0300.0265.0338.0196.0
4G582.0568.0740.0680.0610.0535.0285.0
WiFi1200.01185.01765.01915.01700.01125.0586.0
), ArticleFig(id=1329420379250078543, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, language=EN, label=Table 5, caption=

Performance on Heterogeneous Edge Devices for ECSpec Under 4G Network Conditions

, figureFileSmall=null, figureFileBig=null, tableContent=
设备处理器草稿
模型
时延/
ms
草稿模型
吞吐率/
(token·s−1)
加速比
GSM8KMT-
Bench
HumanEval
Raspberry Pi 5Cortex-A761456.90.760.850.72
Jetson AGX OrinAmpere GPU8.5117.62.322.452.20
iPhone 15
Pro Max
A17 Pro12.083.32.152.282.08
Snapdragon
8 Gen 3
Hexagon NPU10.595.22.262.362.14
), ArticleFig(id=1329420379312993104, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, language=CN, label=表5, caption=

在4G网络条件下ECSpec在异构边缘设备上的性能表现

, figureFileSmall=null, figureFileBig=null, tableContent=
设备处理器草稿
模型
时延/
ms
草稿模型
吞吐率/
(token·s−1)
加速比
GSM8KMT-
Bench
HumanEval
Raspberry Pi 5Cortex-A761456.90.760.850.72
Jetson AGX OrinAmpere GPU8.5117.62.322.452.20
iPhone 15
Pro Max
A17 Pro12.083.32.152.282.08
Snapdragon
8 Gen 3
Hexagon NPU10.595.22.262.362.14
), ArticleFig(id=1329420379388490577, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, language=EN, label=Table 6, caption=

Scalability Evaluation of ECSpec on Newer Model Architectures

, figureFileSmall=null, figureFileBig=null, tableContent=
目标模型延迟/ms加速比
ECSpec
(5G)
ECSpec
(4G)
ECSpec
(5G)
ECSpec
(4G)
LLaMA-2-70B420.0578.02.232.13
LLaMA-3-70B395.0550.02.562.30
Mixtral 8×7B320.0485.02.051.98
), ArticleFig(id=1329420379463988050, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, language=CN, label=表6, caption=

ECSpec在新一代模型架构上的可扩展性评估

, figureFileSmall=null, figureFileBig=null, tableContent=
目标模型延迟/ms加速比
ECSpec
(5G)
ECSpec
(4G)
ECSpec
(5G)
ECSpec
(4G)
LLaMA-2-70B420.0578.02.232.13
LLaMA-3-70B395.0550.02.562.30
Mixtral 8×7B320.0485.02.051.98
)], attaches=null, journal=Journal(id=1324759606947315908, delFlag=0, nameCn=计算机研究与发展, nameEn=Journal of Computer Research and Development, nameHistory1=null, nameHistory2=null, issn=1000-1239, eissn=null, cn=11-1777/TP, coden=null, periodic=0, language=CN, oaType=null, ccby=null, superviseOffice=null, ownerOffice=null, pubOffice=null, editorOffice=null, officeType=null, aims=null, clcCode=null, officeProv=null, officeCity=null, officeAddr=null, officeZip=null, officeEmail=null, officePhone=null, editDirector=null, officeDirector=null, officeDirectorPhone=null, officeStaffNum=null, officeEmpNum=null, coverPicUrl=k2l8zgqwstuxxevFX2qxVA==, journalPrice=null, startedYear=null, abbrevIsoEn=Journal of Computer Research and Development, journalRemark=null, publicationField=null, createdTime=1793776903141, updatedTime=1793850698612, createdBy=18614031015, updatedBy=13701087609, firstLetterCn=J, firstLetterEn=J, subjectCode=Electronic Information, subjectName=null, subjectCodeEn=Electronic Information, subjectNameEn=null, picCn=k2l8zgqwstuxxevFX2qxVA==, picEn=r6k0Ry+RD+c/VXj5QarmDw==, jcr=null, cjcr=null, exts=[JournalExt(id=1325069127710286688, language=CN, name=计算机研究与发展, nameHistory1=null, nameHistory2=null, managedBy=, sponsoredBy=, publishedBy=, editorOffice=, officeProv=null, officeCity=null, officeAddr=, officeZip=, editDirector=, officeDirector=null, officePhone=null, coverPicUrl=null, journalRemark=, submitArticleUrl=null, websiteUrl=, createdTime=1793850698640, updatedTime=1793850698640, createdBy=13701087609, updatedBy=13701087609, submissionGuidelinesUrl=, submissionAuthorUrl=https://crad.ict.ac.cn/journalx/authorLogOn.action, submissionEditorUrl=https://crad.ict.ac.cn/journalx/editorLogOn.action, submissionReviewUrl=https://crad.ict.ac.cn/journalx/expertLogOn.action, submissionCeEditorUrl=https://crad.ict.ac.cn/journalx/editorInChiefLogOn.action, submissionAeEditorUrl=, option={"copyright":""}), JournalExt(id=1325069127777395553, language=EN, name=Journal of Computer Research and Development, nameHistory1=null, nameHistory2=null, managedBy=, sponsoredBy=, publishedBy=, editorOffice=, officeProv=null, officeCity=null, officeAddr=, officeZip=, editDirector=, officeDirector=null, officePhone=null, coverPicUrl=null, journalRemark=, submitArticleUrl=null, websiteUrl=, createdTime=1793850698656, updatedTime=1793850698656, createdBy=13701087609, updatedBy=13701087609, submissionGuidelinesUrl=, submissionAuthorUrl=https://crad.ict.ac.cn/journalx/authorLogOn.action, submissionEditorUrl=https://crad.ict.ac.cn/journalx/editorLogOn.action, submissionReviewUrl=https://crad.ict.ac.cn/journalx/expertLogOn.action, submissionCeEditorUrl=https://crad.ict.ac.cn/journalx/editorInChiefLogOn.action, submissionAeEditorUrl=, option={"copyright":""})], databaseList=null, tenantJournalId=1324759914146529478, websiteList=[Website(id=1325070300139892771, webName=null, webTitle=null, webDomain=null, webCopyrigh=null, webIpcNo=null, seoTitle=null, seoKeywords=null, seoDescription=null, tenantJournalId=null, journalId=1324759914146529478, journalNameCn=null, journalNameEn=null, grayFlag=null, tenantId=1146029695717560320, platformId=null, journalGroupId=null, journalGroupNameCn=null, journalGroupNameEn=null, type=1, domain=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN, language=CN, createTime=1793850978169, createBy=18614031015, updateTime=1793851040259, updateBy=18614031015, name=计算机研究与发展-中文, tplId=1146099689490845704, title=计算机研究与发展, delFlag=0, indexPage=/home, props=[WebsiteProps(id=1325070893751349360, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=articleTextType, value=kx, createTime=1793851119697, updateTime=1793851119697, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893730377837, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=banner, value=null, createTime=1793851119692, updateTime=1793851119692, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893772320883, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=grayFlag, value=0, createTime=1793851119702, updateTime=1793851119702, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893721989228, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=logo, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/file/pic?fileId=U27tDsy717YF7XSPA+Vwdw==, createTime=1793851119690, updateTime=1793851119690, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893784903797, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=minRunFlag, value=0, createTime=1793851119705, updateTime=1793851119705, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893742960751, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=picServerUrl, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/file/pic, createTime=1793851119695, updateTime=1793851119695, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893780709492, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=silenceFlag, value=0, createTime=1793851119704, updateTime=1793851119704, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893738766446, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=staticResourcePath, value=https://castjournals.cast.org.cn/joweb/cast_kjdb_cn_619/, createTime=1793851119694, updateTime=1793851119694, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893755543665, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=themeColor, value=null, createTime=1793851119698, updateTime=1793851119698, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893763932274, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=themeStyle, value=null, createTime=1793851119700, updateTime=1793851119700, creator=18614031015, updator=18614031015)]), Website(id=1325070300580294698, webName=null, webTitle=null, webDomain=null, webCopyrigh=null, webIpcNo=null, seoTitle=null, seoKeywords=null, seoDescription=null, tenantJournalId=null, journalId=1324759914146529478, journalNameCn=null, journalNameEn=null, grayFlag=null, tenantId=1146029695717560320, platformId=null, journalGroupId=null, journalGroupNameCn=null, journalGroupNameEn=null, type=1, domain=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN, language=EN, createTime=1793850978274, createBy=18614031015, updateTime=1793851036638, updateBy=18614031015, name=计算机研究与发展-英文, tplId=1146101810881728533, title=Journal of Computer Research and Development, delFlag=0, indexPage=/home, props=[WebsiteProps(id=1325070925288321148, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=articleTextType, value=kx, createTime=1793851127216, updateTime=1793851127216, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925263155321, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=banner, value=null, createTime=1793851127210, updateTime=1793851127210, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925309292671, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=grayFlag, value=0, createTime=1793851127221, updateTime=1793851127221, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925254766712, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=logo, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/file/pic?fileId=U27tDsy717YF7XSPA+Vwdw==, createTime=1793851127208, updateTime=1793851127208, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925326069889, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=minRunFlag, value=0, createTime=1793851127225, updateTime=1793851127225, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925279932539, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=picServerUrl, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/file/pic, createTime=1793851127214, updateTime=1793851127214, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925317681280, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=silenceFlag, value=0, createTime=1793851127223, updateTime=1793851127223, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925271543930, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=staticResourcePath, value=https://castjournals.cast.org.cn/joweb/cast_kjdb_en_623/, createTime=1793851127212, updateTime=1793851127212, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925292515453, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=themeColor, value=null, createTime=1793851127217, updateTime=1793851127217, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925300904062, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=themeStyle, value=null, createTime=1793851127219, updateTime=1793851127219, creator=18614031015, updator=18614031015)])], journalTitle=计算机研究与发展, weixinUrl=null, journalUrl=https://crad.ict.ac.cn/, iacademicId=null, status=1, seqNo=null, journalTitleEn=Journal of Computer Research and Development, journalPhotoCn=k2l8zgqwstuxxevFX2qxVA==, journalPhotoEn=r6k0Ry+RD+c/VXj5QarmDw==, journalFirstLetter=J, journalRecommend=null, journalNew=null, journalCollection=null, jcrJf=null, cjcrJf=null, jcrJfStr=null, cjcrJfStr=null, submissionFirstDecision=null, sciSubjectClassification=null, casSubjectClassification=null, citeScore=null, totalCitationFrequency=null, icpCode=null, psCode=null, advertisingLicenseCode=null, copyrightInformation=null, country=null, option=, provinceCode=null, provinceName=null, collectFlag=false, interPubPlatform=, interPubPlatformUrl=null), detailUrlCn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/10.7544/issn1000-1239.202660136, detailUrlEn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/10.7544/issn1000-1239.202660136, pdfUrlCn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/PDF/10.7544/issn1000-1239.202660136, pdfUrlEn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/PDF/10.7544/issn1000-1239.202660136, aliStartDate=0, aliEndDate=0, collectionFlag=false, citedCount=null, citedUrl=null, previewStatus=0, delFlag=0, hasFullText=1, orderTime=1785513600000, fullTextJson=null, articleText=null, reference=null)
收藏切换
面向演进式大语言模型的带宽高效边云协同推理方法
李豫晨 1 , 孔芮 1 , 陈新冉 2 , 殷大伟 2 , 孔令和 1, * , 陈贵海 1
计算机研究与发展 | AI算力网:系统与关键应用专题 2026,63(8): 2024-2036
收起
收藏切换
计算机研究与发展 |AI算力网:系统与关键应用专题 2026 , 63 (8) : 2024 -2036
面向演进式大语言模型的带宽高效边云协同推理方法
全屏
[Author(id=1329420375810749203, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, orderNo=0, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=yuchenli1230@gmail.com, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329420375898829589, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, authorId=1329420375810749203, language=EN, stringName=Yuchen Li, firstName=Yuchen, middleName=null, lastName=Li, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1School of Computer Science, Shanghai Jiao Tong University, Shanghai 200240, bio={"content":"

born in 1997. PhD. His main research interests include data mining and LLMs

"}, bioImg=null, bioContent=

born in 1997. PhD. His main research interests include data mining and LLMs

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329420375982715670, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, authorId=1329420375810749203, language=CN, stringName=李豫晨, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1上海交通大学计算机学院 上海 200240, bio={"img":"k7lMQow21lX+THpKjOhukg==","content":"

1997年生。博士。主要研究方向为数据挖掘、大语言模型

"}, bioImg=k7lMQow21lX+THpKjOhukg==, bioContent=

1997年生。博士。主要研究方向为数据挖掘、大语言模型

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329420375647171340, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, xref=1, ext=[AuthorCompanyExt(id=1329420375655559949, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, companyId=1329420375647171340, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1School of Computer Science, Shanghai Jiao Tong University, Shanghai 200240), AuthorCompanyExt(id=1329420375663948558, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, companyId=1329420375647171340, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1上海交通大学计算机学院 上海 200240)])]), Author(id=1329420376054018841, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, orderNo=1, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329420376129516315, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, authorId=1329420376054018841, language=EN, stringName=Rui Kong, firstName=Rui, middleName=null, lastName=Kong, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1School of Computer Science, Shanghai Jiao Tong University, Shanghai 200240, bio={"content":"

born in 1998. PhD. His main research interests include data mininig and LLMs

"}, bioImg=null, bioContent=

born in 1998. PhD. His main research interests include data mininig and LLMs

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329420376205013788, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, authorId=1329420376054018841, language=CN, stringName=孔芮, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1上海交通大学计算机学院 上海 200240, bio={"img":"ISl9yA4kNHukbtXimrR0+w==","content":"

1998年生。博士。主要研究方向为数据挖掘、大语言模型

"}, bioImg=ISl9yA4kNHukbtXimrR0+w==, bioContent=

1998年生。博士。主要研究方向为数据挖掘、大语言模型

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329420375647171340, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, xref=1, ext=[AuthorCompanyExt(id=1329420375655559949, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, companyId=1329420375647171340, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1School of Computer Science, Shanghai Jiao Tong University, Shanghai 200240), AuthorCompanyExt(id=1329420375663948558, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, companyId=1329420375647171340, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1上海交通大学计算机学院 上海 200240)])]), Author(id=1329420376276316958, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, orderNo=2, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329420376339231520, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, authorId=1329420376276316958, language=EN, stringName=Xinran Chen, firstName=Xinran, middleName=null, lastName=Chen, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=2, address=2Baidu.com Times Technology (Beijing) Co., Ltd., Beijing 100085, bio={"content":"

born in 1998. PhD. His main research interests include information retrieval and LLMs

"}, bioImg=null, bioContent=

born in 1998. PhD. His main research interests include information retrieval and LLMs

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329420376410534689, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, authorId=1329420376276316958, language=CN, stringName=陈新冉, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=2, address=2百度时代网络技术(北京)有限公司 北京 100085, bio={"img":"s9npwcW/oePmnChnM+bCOA==","content":"

1998年生。博士。主要研究方向为信息检索、大语言模型

"}, bioImg=s9npwcW/oePmnChnM+bCOA==, bioContent=

1998年生。博士。主要研究方向为信息检索、大语言模型

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329420375726863119, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, xref=2, ext=[AuthorCompanyExt(id=1329420375731057424, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, companyId=1329420375726863119, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2Baidu.com Times Technology (Beijing) Co., Ltd., Beijing 100085), AuthorCompanyExt(id=1329420375747834641, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, companyId=1329420375726863119, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2百度时代网络技术(北京)有限公司 北京 100085)])]), Author(id=1329420376481837859, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, orderNo=3, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329420376548946725, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, authorId=1329420376481837859, language=EN, stringName=Dawei Yin, firstName=Dawei, middleName=null, lastName=Yin, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=2, address=2Baidu.com Times Technology (Beijing) Co., Ltd., Beijing 100085, bio={"content":"

born in 1984. PhD. His main research interests include data mining and information retrieval

"}, bioImg=null, bioContent=

born in 1984. PhD. His main research interests include data mining and information retrieval

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329420376624444198, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, authorId=1329420376481837859, language=CN, stringName=殷大伟, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=2, address=2百度时代网络技术(北京)有限公司 北京 100085, bio={"img":"BxzYZBewriJrRYI4RDcH0A==","content":"

1984年生。博士。主要研究方向为数据挖掘、信息检索

"}, bioImg=BxzYZBewriJrRYI4RDcH0A==, bioContent=

1984年生。博士。主要研究方向为数据挖掘、信息检索

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329420375726863119, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, xref=2, ext=[AuthorCompanyExt(id=1329420375731057424, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, companyId=1329420375726863119, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2Baidu.com Times Technology (Beijing) Co., Ltd., Beijing 100085), AuthorCompanyExt(id=1329420375747834641, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, companyId=1329420375726863119, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2百度时代网络技术(北京)有限公司 北京 100085)])]), Author(id=1329420376687358760, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, orderNo=4, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=linghe.kong@sjtu.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=1, authorType=1, ext={EN=AuthorExt(id=1329420376767050538, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, authorId=1329420376687358760, language=EN, stringName=Linghe Kong, firstName=Linghe, middleName=null, lastName=Kong, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, *, address=1School of Computer Science, Shanghai Jiao Tong University, Shanghai 200240, bio={"content":"

born in 1983. PhD, professor, PhD supervisor. His main research interests include Internet of things and AI

"}, bioImg=null, bioContent=

born in 1983. PhD, professor, PhD supervisor. His main research interests include Internet of things and AI

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329420376838353707, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, authorId=1329420376687358760, language=CN, stringName=孔令和, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, *, address=1上海交通大学计算机学院 上海 200240, bio={"img":"7RMrmDucK9trDiRZMmO97Q==","content":"

1983年生。博士,教授,博士生导师。主要研究方向为物联网、人工智能

"}, bioImg=7RMrmDucK9trDiRZMmO97Q==, bioContent=

1983年生。博士,教授,博士生导师。主要研究方向为物联网、人工智能

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329420375647171340, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, xref=1, ext=[AuthorCompanyExt(id=1329420375655559949, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, companyId=1329420375647171340, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1School of Computer Science, Shanghai Jiao Tong University, Shanghai 200240), AuthorCompanyExt(id=1329420375663948558, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, companyId=1329420375647171340, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1上海交通大学计算机学院 上海 200240)])]), Author(id=1329420376909656877, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, orderNo=5, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329420376976765743, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, authorId=1329420376909656877, language=EN, stringName=Guihai Chen, firstName=Guihai, middleName=null, lastName=Chen, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1School of Computer Science, Shanghai Jiao Tong University, Shanghai 200240, bio={"content":"

born in 1963. PhD, professor, PhD supervisor. His main research interest includes computer networks

"}, bioImg=null, bioContent=

born in 1963. PhD, professor, PhD supervisor. His main research interest includes computer networks

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329420377035486000, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, authorId=1329420376909656877, language=CN, stringName=陈贵海, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1上海交通大学计算机学院 上海 200240, bio={"img":"x9mlIl1ntFRuMmQ1fGxwBg==","content":"

1963年生。博士,教授,博士生导师。主要研究方向为计算机网络

"}, bioImg=x9mlIl1ntFRuMmQ1fGxwBg==, bioContent=

1963年生。博士,教授,博士生导师。主要研究方向为计算机网络

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329420375647171340, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, xref=1, ext=[AuthorCompanyExt(id=1329420375655559949, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, companyId=1329420375647171340, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1School of Computer Science, Shanghai Jiao Tong University, Shanghai 200240), AuthorCompanyExt(id=1329420375663948558, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329420373529047797, companyId=1329420375647171340, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1上海交通大学计算机学院 上海 200240)])])]
李豫晨1 , 孔芮1, 陈新冉2, 殷大伟2, 孔令和1, * , 陈贵海1
作者信息
  • 1上海交通大学计算机学院 上海 200240
  • 2百度时代网络技术(北京)有限公司 北京 100085
通讯作者:
作者简介:

作者贡献声明

李豫晨负责方案整体设计并撰写论文;孔芮、陈新冉负责部分算法思路和实验方法;殷大伟、孔令和、陈贵海提出指导意见并修改论文。

1997年生。博士。主要研究方向为数据挖掘、大语言模型

1998年生。博士。主要研究方向为数据挖掘、大语言模型

1998年生。博士。主要研究方向为信息检索、大语言模型

1984年生。博士。主要研究方向为数据挖掘、信息检索

1983年生。博士,教授,博士生导师。主要研究方向为物联网、人工智能

1963年生。博士,教授,博士生导师。主要研究方向为计算机网络

Bandwidth-Efficient Edge-Cloud Collaborative Inference Method for Evolving Large Language Models
Yuchen Li1 , Rui Kong1, Xinran Chen2, Dawei Yin2, Linghe Kong1, * , Guihai Chen1
Affiliations
  • 1School of Computer Science, Shanghai Jiao Tong University, Shanghai 200240
  • 2Baidu.com Times Technology (Beijing) Co., Ltd., Beijing 100085
  • born in 1997. PhD. His main research interests include data mining and LLMs

    born in 1998. PhD. His main research interests include data mininig and LLMs

    born in 1998. PhD. His main research interests include information retrieval and LLMs

    born in 1984. PhD. His main research interests include data mining and information retrieval

    born in 1983. PhD, professor, PhD supervisor. His main research interests include Internet of things and AI

    born in 1963. PhD, professor, PhD supervisor. His main research interest includes computer networks

出版时间: 2026-08-01 doi: 10.7544/issn1000-1239.202660136
文章导航
收藏切换

在移动与边缘计算环境中部署大语言模型面临端侧算力与存储受限、无线带宽紧张以及云端模型持续演进等挑战。现有基于投机解码的边云协同推理虽可借助边缘草稿模型与云端目标模型的并行协作降低生成时延,但通常要求二者紧密匹配。当云端模型频繁更新时,往往需要反复同步边缘模型,导致额外通信开销、时延增加,并削弱系统可扩展性。为此,提出通信高效边云协同推理框架ECSpec。其核心在于共享骨干设计,使静态边缘草稿模型可适配多个不断演进的云端目标模型,从而避免端侧重复训练与权重反复下载,显著降低通信与维护成本。进一步结合时变信道与设备异构特性,设计信道感知自适应投机机制,依据实时信道状态和能耗预算动态调整草稿长度,在效率与能耗之间实现平衡。实验结果表明,ECSpec在通信受限的边缘环境中可实现更稳定、高效的大语言模型协同推理。

边缘计算  /  边云协同  /  大语言模型  /  投机解码  /  协同推理

Deploying large language models (LLMs) in mobile and edge computing environments faces multiple challenges, including limited on-device resources, scarce wireless bandwidth, and the continual evolution of cloud-side models. Although speculative decoding based edge-cloud collaborative inference can reduce end-to-end generation latency by using a lightweight draft model at the edge and a target model in the cloud for parallel verification, existing methods usually rely on a tightly coupled relationship between the two models. In practical systems, frequent updates of the cloud model often require repeated synchronization of the edge-side draft model, leading to substantial communication overhead, higher latency, and limited scalability. To address this issue, ECSpec is proposed as a communication-efficient edge-cloud collaborative inference framework for model-evolving scenarios. Its key idea is a shared backbone architecture that enables a single static edge-side draft model to remain compatible with a family of continuously evolving cloud-side target models, thereby avoiding repeated retraining or weight downloading at the edge and significantly reducing communication and maintenance costs. In addition, a channel-aware adaptive speculation mechanism is introduced to dynamically adjust the draft length according to real-time channel conditions and energy budgets, achieving a better tradeoff between inference efficiency and energy consumption. Experimental results demonstrate that ECSpec delivers more stable and efficient collaborative LLM inference under communication-constrained edge environments.

edge computing  /  edge-cloud collaboration  /  large language models  /  speculative decoding  /  collaborative inference
李豫晨, 孔芮, 陈新冉, 殷大伟, 孔令和, 陈贵海. 面向演进式大语言模型的带宽高效边云协同推理方法. 计算机研究与发展, 2026 , 63 (8) : 2024 -2036 . DOI: 10.7544/issn1000-1239.202660136
Yuchen Li, Rui Kong, Xinran Chen, Dawei Yin, Linghe Kong, Guihai Chen. Bandwidth-Efficient Edge-Cloud Collaborative Inference Method for Evolving Large Language Models[J]. Journal of Computer Research and Development, 2026 , 63 (8) : 2024 -2036 . DOI: 10.7544/issn1000-1239.202660136
近年来,以 ChatGPT[1]、LLaMA[2] 和 Gemini[3] 等为代表的大语言模型(large language models,LLMs)已成为新一代智能代理系统的核心技术基础,广泛应用于移动个人助理、实时翻译设备以及复杂推理任务等场景[4-5]。模型能力的提升在很大程度上决定了用户体验质量以及复杂任务执行的成功率。尽管已有大量研究致力于通过量化、剪枝[6]等技术在边缘设备上压缩模型规模,但受限于电池容量、热设计功耗以及内存带宽等因素,单纯依靠端侧设备进行大模型推理仍然面临显著性能瓶颈[7]。
边云协同推理为解决上述问题提供了一种有效途径。该范式通过将计算密集型子任务卸载至云端高性能服务器,同时在边缘侧保留轻量级处理能力,从而在推理能力与响应精度之间实现平衡[8]。然而,在这一协同框架下,如何在保证推理准确性的同时维持实时性能成为关键挑战。尤其是在高度动态的移动网络环境中,用户请求与模型响应之间的时延若过高,将显著影响系统的交互性与可靠性。在自回归生成场景中,该问题尤为突出。由于模型按 token 逐步生成输出,逐 token 的传输机制会带来显著通信开销。此外,边缘设备与云端之间往往需要交换大量信息,从而进一步增加网络负载。因此,亟需设计高效的推理加速机制,以降低通信开销并缩短决策时延[9]。
为缓解自回归生成带来的时延问题,投机解码(speculative decoding,SD)被广泛应用于推理加速。典型的边云投机解码框架中,边缘设备首先利用轻量级草稿模型快速生成一段候选 token 序列(即“草稿”),随后将该序列发送至云端,由高性能目标模型进行并行验证[10]。近年来,基于深度学习的投机解码方法(如EAGLE-2[11]和Medusa[12])在时延降低方面取得了显著进展。然而,将这些方法直接部署于实际边云系统仍面临若干根本性挑战。首先,现有投机解码框架通常依赖草稿模型与目标模型之间的紧耦合关系,而云端目标模型往往通过参数高效微调(parameter-efficient fine-tuning,PEFT)[9]等方式频繁更新。这种更新机制会导致“更新风暴”现象,即为了保持模型一致性而产生过量通信开销,尤其在带宽受限的无线环境下更加严重。其次,若为避免模型同步而保持边缘草稿模型静态,则随着目标模型持续演进,草稿模型与目标模型之间的分布差异将不断扩大,进而显著降低 token 接受率。这种现象可被称为“性能崩塌”,即投机解码效率随着模型分布偏移而急剧下降。最后,投机解码性能高度依赖于无线信道条件。在移动环境中,上行时延随时间变化显著,固定步长的投机策略难以在不同信道状态下保持最优性能。在弱信号场景下,传输固定长度的草稿 token 可能带来高额通信开销;而在强信号环境下,过小的步长又无法充分利用可用带宽。因此,模型频繁演进、分布偏移以及无线时延波动三者叠加,从根本上限制了现有投机解码框架在实际边云部署中的可行性。
针对上述挑战,本文的目标是在边云系统中实现“版本无关”的投机解码机制,即利用单一静态边缘模型为多版本、持续演进的云端目标模型提供加速支持。一旦构建出具有鲁棒性的草稿框架,即可通过1个草稿模型支持多个不同微调版本的目标模型,从而显著降低系统维护成本。此外,该解耦式框架在面对未知目标分布时亦具有良好的泛化能力,避免了传统方法中必须重新训练草稿模型的问题,从而进一步降低部署与通信开销。受基础模型在跨模态处理方面成功经验的启发,本文提出一种面向模型演进场景的边云协同投机解码框架ECSpec。该框架的核心思想在于从结构层面解耦边缘侧草稿模型与云端目标模型的生命周期,从而在带宽受限的无线环境中避免频繁的模型同步。同时,ECSpec 显式建模无线信道的随机性特征,并根据时变网络条件动态调整投机解码策略。通过在统一框架中同时考虑模型演进与无线动态因素,ECSpec能够在边缘系统中实现可扩展、鲁棒且低时延的大语言模型推理。
本文的主要贡献包括3个方面:
1)提出一种新的边云协同投机解码框架ECSpec,实现面向模型演进的版本无关推理机制。通过引入共享冻结锚点骨干架构,从根本上解耦草稿模型与目标模型生命周期,消除无线边缘网络中的“更新风暴”问题。
2)设计了一种适用于边缘环境下推测解码的信道自适应投机策略。通过刻画投机步长、token 接受率与通信速率之间的相互关系,构建具有理论依据的步长选择模型,从而实现对时变信道条件的动态适配,并最大化有效 token 生成吞吐率。
3)在大规模 GPU 集群上进行广泛实验评估,覆盖推理与代码生成等多种任务场景。实验结果表明,ECSpec在生成效率方面持续优于现有投机解码方法;通过动态调整投机步长,在通信受限环境中表现尤为显著。同时,由于边缘草稿模型无需随目标模型更新而反复同步,系统显著降低了更新相关通信的开销。
本节主要概述与本文模型相关的推测解码、边云协同推理和感知无线的移动人工智能。
投机解码已成为缓解大语言模型自回归生成过程中“内存受限型时延瓶颈”的关键技术范式。其核心思想在于通过“草稿—验证”(draft-then-verify)策略,以并行计算能力换取串行内存访问的减少,从而提升生成效率。 Leviathan 等人[10]以及 Chen 等人[13]对投机执行框架进行了形式化建模,证明了小规模草稿模型可以生成候选 token 序列,并由目标模型进行并行验证,在理论上保证最终输出与目标模型逐 token 生成结果完全一致。在此基础上,后续研究重点提升草稿质量与 token 接受率。Medusa[12] 在冻结原始模型参数的基础上引入多个解码头,以同时预测多个未来 token,从而避免单独训练草稿模型。EAGLE-2[11] 与 EAGLE[14]进一步通过层级特征外推机制,在草稿层对输入特征进行加工,生成上下文相关的候选 token,从而提升预测准确性。除模型驱动方法外,亦有非模型化草稿生成策略,例如基于 n-gram 匹配的方法[15]以及基于检索增强的方法[16],通过统计模式或外部数据库生成候选 token,避免复杂神经网络计算。然而,上述投机解码框架普遍建立在“静态环境”假设之上,即草稿模型与目标模型之间保持紧耦合或结构一致。在实际边缘计算场景中,云端目标模型通常通过参数高效微调PEFT[17] 等方式频繁更新。若维持 EAGLE 或 Medusa 等方法所要求的严格结构耦合,则必须通过无线链路频繁同步边缘侧草稿模型参数,从而带来难以承受的通信开销。与现有方法不同,ECSpec 通过结构性解耦草稿模型与目标模型的生命周期,实现单一静态边缘模型长期服务于持续演进的云端目标模型,无需频繁同步更新,从根本上突破了现有投机解码框架在无线边缘环境中的可扩展性限制。
边云协同推理旨在弥合移动设备资源受限与现代深度神经网络(deep neural network,DNN)高计算需求之间的差距[18]。该研究方向通常关注如何在边缘侧与云端之间进行最优计算划分,以在时延、能耗与隐私之间取得平衡。早期工作如 Neurosurgeon [19] 以及 Matsubara 等人[20]提出通过在 DNN 内部寻找最优分割点来最小化端到端时延。随着大语言模型的兴起,去中心化推理框架[21-23]被提出,将 Transformer 模块分布至互联网中的异构设备上执行。此外,为缓解传输高维中间激活张量所带来的通信瓶颈,研究者提出了激进量化方法[24-25]以及激活压缩技术[26]。尽管上述方法在降低通信成本方面取得一定进展,但在生成式任务中仍存在明显局限。由于自回归生成需对每个 token 传输高维激活特征,标准分割式推理往往面临高额传输时延。此外,这些方法通常假设模型结构保持稳定,忽略了实际生产环境中云端模型频繁更新所带来的版本一致性问题。ECSpec 在2个方面推动了该领域发展。首先,其仅传输轻量级 token 索引而非高维激活张量,显著降低通信开销。其次,通过引入基于锚点的对齐机制,使边云协同推理在模型版本不匹配的情况下仍保持稳定有效,从而增强系统在真实部署环境中的鲁棒性。
在移动计算环境中部署人工智能应用,必须充分考虑无线网络的动态特性。与稳定的有线连接不同,移动无线信道存在衰落、干扰与噪声等问题,严重影响实时人工智能应用的服务质量(quality of experience,QoE)[27]。为应对这一挑战,Park 等人[28]与 Zhu 等人[29]在无线边缘智能领域奠定了基础工作,通过优化计算卸载策略与信道质量之间的权衡关系,实现系统性能提升。在自然语言处理场景中,动态推理机制如 DeeBERT[30]与SPINN[31]允许模型基于置信度阈值提前退出,以降低能耗并缩短推理时延。近期工作 DSSD [32] 尝试将投机解码引入无线环境,但其仍采用固定策略,未充分利用实时信道状态信息进行动态优化。ECSpec 在无线网络理论与大语言模型推理系统之间建立了系统化联系。不同于将生成长度视为固定超参数的既有方法,ECSpec 将投机步长建模为依赖信道条件的动态变量,并通过信道感知自适应策略对生成过程进行调节。该机制使系统能够在移动场景下典型的网络波动条件中维持较高吞吐率与稳定性能。
本节首先给出边云协同推理的系统模型。在此基础上,结合真实数据集与实际网络轨迹的初步实验分析,我们识别出阻碍现有投机解码框架在移动环境中实际部署的3项关键挑战:模型同步所带来的高额带宽开销、分布偏移引发的性能崩塌,以及系统对无线网络波动的高度敏感性。
本文考虑一个由边缘设备$ \mathcal{E} $与云端服务器$ C $组成的协同推理系统。云端部署一个高容量的目标模型$ {M}_{\rm t}\left(\cdot ;{\boldsymbol \theta }_{\rm t}\right) $(例如 LLaMA-2-70B),运行于GPU集群之上,其中$ {\boldsymbol \theta }_{\rm t}\in {\mathbb{R}}^{{{N}_{\rm t}}} $表示模型参数。为保持在不断演进的下游任务上的竞争性能,服务提供方会基于不断变化的任务数据集$ {\mathcal{D}}_{\rm task} $,通过参数高效微调或全参数训练方式周期性更新$ {M}_{\rm t}\left(\cdot ;{\boldsymbol \theta }_{\rm t}\right) $。因此,云端目标模型随着时间的推移演变为一个序列:$ \{M_{\rm t}^{(0)},M_{\rm t}^{(1)},… ,M_{\rm t}^{(s)} \} $。 在边缘端,客户端部署一个轻量级的草稿模型$ {M}_{\rm d}\left(\cdot ;{\boldsymbol \theta }_{\rm d}\right) $,其参数量$ {N}_{\rm d}\ll {N}_{\rm t} $。其规模通常仅相当于目标模型的一小部分(例如约为目标模型单层的参数量级)。
由于边缘设备在存储容量、电池能耗及计算能力方面存在严格限制,无法频繁下载大规模模型权重。因此,草稿模型在目标模型多次更新过程中保持静态(冻结)状态。边缘设备与云端通过随机无线链路进行通信。在每一轮投机解码过程中,边缘设备生成一段候选token块$ {x}_{\rm draft} $并发送至云端;云端利用$ {M}_{\rm t} $对该token块进行验证,并将验证结果$ {x}_{\rm verified} $返回至边缘侧。给定输入提示词$ X $,系统生成长度为$ O $的输出序列$ {Y}=\left\{{y}_{1},{y}_{2},… ,{y}_{O}\right\} $。在采用投机解码、草稿长度为$ K$的情况下,第n轮解码的时延模型表示为:
$ {T}_{\rm s}\left(K,{R}_{n}\right)={T}_{\rm e}\left(K\right)+{T}_{\rm u}\left(K,{R}_{n}\right)+{T}_{\rm c}\left(K\right)+{T}_{\rm d} \;,$
其中$ {T}_{\rm e}\left(K\right) $表示边缘端生成$ K $个Token的计算时间;$ {T}_{\rm u}\left(K,{R}_{n}\right) $表示第n轮的上行传输延迟;$ {T}_{\rm c}\left(K\right) $表示云端对草稿块进行验证的计算时间;$ {T}_{\rm d} $表示云端返回验证结果的下行时延。进一步地,上行传输时延可表示为上行数据量$ {B}_{\rm u}\left(K\right) $与当前可达上行速率$ {R}_{n} $的比值:
$ {T}_{\rm u}\left(K,{R}_{n}\right)={B}_{\rm u}\left(K\right)/{R}_{n}\;。$
该模型刻画了投机步长$ K $与无线信道状态$ {R}_{n} $对系统时延的耦合关系,为后续优化问题的形式化奠定基础。
现有的云边协同推测解码在实际部署时存在3个亟待解决的问题:
问题1. 无线网络中的“更新风暴”无线网络中大模型的更新需求与有限无线带宽之间的冲突。第1个核心障碍在于模型新鲜度需求与无线带宽受限之间的矛盾。在标准投机解码框架中,草稿模型需要尽可能逼近目标模型的输出分布,以维持较高token 接受率$ \hat{\gamma } $。因此,当目标模型从旧版本更新至新版本,例如针对医学场景进行微调时,理论上需要重新训练并同步边缘侧草稿模型,以保证投机效率。我们将这种由于模型频繁更新而引发的大规模同步现象称为“更新风暴”。为量化其影响,我们对一个轻量级草稿模型(约3.2 GB)在典型移动网络环境中的同步开销进行了初步评估。表1结果显示,在 4G 网络环境下,即使下载压缩后的草稿模型,也需要接近 10 min。对于每日更新模型的移动应用而言,这将对蜂窝网络基础设施造成流量压力,同时迅速消耗用户流量套餐与电池资源。因此,亟需设计一种无需频繁同步的静态草稿模型机制,以避免更新风暴所带来的不可扩展性问题。
问题2. 分布偏移与性能崩塌。若为节省带宽而放弃模型同步(即保持$ {M}_{\rm d} $的参数固定,而$ {M}_{\rm t} $持续更新),则会面临第2项挑战:分布偏移。随着目标模型微调至新领域,其输出分布将逐渐偏离静态草稿模型的输出分布。为验证这一现象,我们采用通用 LLaMA-2-70B 作为冻结草稿模型,并分别与通用、数学(GSM8K)以及代码(HumanEval)3个版本的 70B 目标模型进行配合测试。表2结果显示,当目标模型演进至编程领域时,草稿模型的 token 接受率从 0.72 降至 0.18,下降幅度高达 75%。在如此低的接受率下,运行草稿模型并传输 token 所产生的额外开销将超过投机带来的收益,甚至导致整体时延高于标准自回归解码。这一现象可被定义为“性能崩塌”。因此,必须构建一种机制弥合静态草稿表示与演进目标表示之间的差距。本文后续提出的基于锚点的对齐机制正是为解决该问题而设计。
问题3. 无线时延敏感性。第3个挑战源于投机步长$ K $与网络条件之间的耦合关系。现有大多数工作采用固定投机步长(例如$ K=5 $)。然而,我们对真实网络轨迹的分析表明,在移动环境中固定步长往往是次优策略。在弱信号场景(例如电梯或地铁环境)下,上行传输时延项 $ {T}_{\rm u}\left(K,{R}_{n}\right) $在总时延中占主导地位。例如,传输 5 个 token 可能带来约 200 ms 的上行延迟,而对应的验证收益仅约 50 ms。相反,在强 5G 信道条件下,过小的固定$ K $无法充分利用可用带宽。这种不匹配构成一个基本矛盾:静态投机策略无法在异构移动场景与动态信道条件下同时满足时延最优需求。因此,需要设计一种信道感知的自适应投机机制,以根据实时网络状态动态调整投机步长。系统优先利用设备侧或操作系统提供的链路测量接口获得近实时的链路速率信息;当底层接口不可用时,则通过短时间窗口内的回路时延、历史上行/下行吞吐率以及实际传输完成时间对有效链路速率进行统计估计。为降低测量噪声对步长决策的干扰,本文对相关链路指标采用滑动平滑处理。
基于上述时延模型,我们的目标是在演进式边云系统中设计一种始终有效的投机解码框架。具体而言,在云端目标模型序列持续更新的前提下,我们希望设计一种模型耦合机制,使边缘草稿模型在保持静态(无需重新训练或重复下载)的情况下,仍能在与任意版本协作时维持较高 token 接受率。同时,在时变无线信道条件下,我们希望联合设计投机步长策略及模型耦合机制,以最大化系统的有效 token 生成速率(effective token generation rate,ETGR)。本文将 ETGR 近似定义为单位时间内系统有效推进 token 数的期望值。为便于分析,正文采用单轮被目标模型接受的草稿 token 数 $ \tau $ 作为主要统计量,即 $ \mathbb{E}\left[\tau |{K}\right] $。需要说明的是,在实际投机解码过程中,当在位置$ {j} $发生拒绝时,目标模型还可能额外生成一个修正 token,因此严格意义上的单轮推进量可写为$ \tau +1 $,其中1表示由验证分支带来的补偿项,为常数阶项,且不随$ {K} $ 呈主导性增长,故在本文的步长优化分析中采用$ \mathbb{E}\left[\tau |{K}\right] $作为近似目标不会改变最优 $ {K} $ 的主要变化趋势。设$ \tau $表示在一次草稿验证轮次中,被接受的token数量,则ETGR定义为:
$ ETGR(K)=\frac{\mathbb{E}[\tau |K]}{\mathbb{E}[{T}_{\rm s}(K,{R}_{n})]} \;。 $
该目标等价于最小化每个被接受 token 的期望时延:$ \mathbb{E}[{T}_{\rm s}(K,{R}_{n})]/\mathbb{E}[\tau |K] $。
综上所述,本文旨在学习一种自适应投机步长策略以及草稿模型与目标模型之间的结构耦合机制,从而在模型持续演进与无线条件动态变化的环境中最大化 ETGR。
本节详细介绍为解决边云协同推理挑战而设计的 ECSpec框架,如图1所示。
为满足“边缘草稿模型参数$ {\boldsymbol \theta }_{\rm d} $保持静态,而云端目标模型参数$ {\boldsymbol \theta }_{\rm t} $持续演进”的约束条件,本文提出一种基于锚点对齐并结合深度嵌入增强的结构化架构。标准投机解码在云端微调后往往失效,其根本原因在于草稿分布与目标分布之间的差异不断扩大。我们基于已有研究观察到:尽管微调过程会改变高层语义决策行为,但在同一模型家族内部,主干网络中的基础 token 表征逻辑通常在较大程度上保持稳定。因此,可以通过假设共享稳定表征空间实现跨版本对齐。需要进一步说明的是,该假设主要适用于以下场景:目标模型与基础模型具有相同或高度兼容的分词器,主干结构与隐藏维度保持一致或可通过轻量映射对齐,且模型更新主要体现在任务适配层或高层语义行为上。当目标模型发生大规模架构升级、词表重构或完全跨家族迁移时,基础表征空间的一致性可能明显下降,此时需重新训练锚点模块与投影头,或退化为仅云端推理模式[33]。
1)结构定义与深度嵌入学习
给定云端在任务微调前的基础模型为$ {M}_{\rm base}\triangleq M_{\rm t}^{\left(0\right)} $,我们将其分解为特征提取器与预测头。预测头包含最终的Transformer模块(称为锚点块$ B_{L}^{\rm base} $)以及语言建模(language modeling,LM)输出头。边缘草稿模型通过复制并冻结该锚点块构建,并在其后附加一个轻量级可训练投影头$ {\mathcal{H}}_{\rm small} $。然而,仅依赖浅层草稿结构通常难以获得较长的接受长度$ \tau $。为在不增加推理时延的前提下增强模型表达能力,本文提出一种深度嵌入增强机制。具体而言,在草稿模型的标准输入嵌入层之后叠加多个多层感知机(multi-layer perceptron,MLP)模块(共$ L $层),称为 assist layers。这些层用于提升草稿表示与目标模型表示之间的对齐程度。草稿模型的整体结构可表示为:
$ {M}_{\rm d}\left(x\right)={\mathcal{H}}_{\rm small}\left(B_{L}^{\rm base}\left(Assist\left(Embed\left(x\right)\right)\right)\right) \;,$
其中$ {B}_{\rm base}\triangleq B_{L}^{\rm base} $且保持冻结状态。需要强调的是,assist layers的设计满足不引入额外位置编码信和不允许跨token的信息交互2个关键约束。因此,对于任意token,在任意位置其输出表示均为确定性且唯一的函数映射。
2)嵌入融合
在离线训练完成后,我们将 assist layers 与原始嵌入层进行融合。由于标准嵌入层与 assist layers 均可视为对单一token索引的确定性映射,因此可将二者折叠为一次查表操作。具体而言,我们预先离线计算新的嵌入表:
$ new\_embedding\left[i\right]=Assist\left(Embed\left(i\right)\right),\forall i\in\left[0,\left|V\right|\right]\; , $
其中$ \left| V\right| $表示词表大小。在实际推理阶段,边缘草稿模型直接使用融合后的$ new\_embedding $层,从而完全消除assist layers的计算开销,同时保留其增强后的表示能力,实现真正的“零额外推理负担”。
3)通用训练目标
为训练静态边缘组件(即 assist layers 与 $ {\mathcal{H}}_{\rm small} $),我们采用特征回归与知识蒸馏相结合的多目标损失函数。该训练过程仅需离线执行1次。首先,将预测目标模型隐藏特征视为回归任务。为保证草稿隐藏状态$ {h}_{\rm d} $与目标隐藏状态$ {h}_{\rm t} $在表示空间中稳定对齐,采用 Smooth L1损失:
$ {\mathcal{L}}_{\rm reg}=\frac{1}{\omega \cdot \pi }\sum\limits_{i=1}^{\omega }\sum\limits_{j=1}^{\pi }{S mooth\;L}1\left({\boldsymbol W}_{\rm p}h_{\rm d}^{\left(i,j\right)},h_{\rm t}^{\left(i,j\right)}\right)\;,$
其中$ {\boldsymbol W}_{\rm p} $为可学习的维度匹配投影矩阵,$ \omega $表示 batch 大小,$ \pi $表示序列长度。此外,为逼近最终生成目标,我们最小化token分布之间的KL散度[34]:
$ {\mathcal{L}}_{\rm cls}={ {T}}^{2}{D}_{\rm KL}\left(\sigma \left(\frac{{z}_{\rm t}}{T}\right)\Big\|\sigma \left(\frac{{z}_{\rm d}}{T}\right)\right)\;,$
其中$ {z}_{\rm t} $和$ {z}_{\rm d} $分别为目标与草稿模型 logits,$ \sigma $表示 Softmax函数,$ {T} $为温度系数。最终训练目标为:
$ \mathcal{L}={\mathcal{L}}_{\rm reg}+{\omega }_{\rm cls}{\mathcal{L}}_{\rm cls} \;。 $
在移动计算场景中,端到端时延与信道动态密切耦合。固定投机步长$ K $无法兼顾传输开销与投机收益之间的权衡。因此,我们将最优步长选择建模为吞吐率最大化问题。
1)精细化时延建模
单次投机解码步长为$ K $时,总时延由4部分构成:边缘侧生成$ {K} $个候选 token 的计算时延、候选 token 及必要元信息的上行传输时延、云端验证与回滚处理时延,以及验证结果的下行反馈时延。进一步地,通信相关项由可变传输项与固定时延项2部分组成。前者主要由待传输数据量与当前链路速率决定,后者主要包括无线传播时延、协议栈处理、调度与服务器请求分发等不可压缩开销。因而,前链路速率主要影响上下行串行化传输部分,而固定处理项不随$ {K} $线性增长:
$ {\hat{T}}_{\rm s}\left(K,{R}_{n}\right)={\hat{T}}_{\rm e}\left(K\right)+{\hat{T}}_{\rm u}\left(K,{R}_{n}\right)+{\hat{T}}_{\rm c}\left(K\right)+{T}_{\rm d}\;。 $
云端验证时延建模为关于$ K $的仿射函数,以刻画加载 $ K $个新token及其 KV cache 所带来的内存带宽开销。将固定开销与边际开销分离,可得:
$ {\hat{T}}_{\rm s}\left(K,{R}_{n}\right)={T}_{\rm fixed}+K\left({\alpha }_{\rm e}+\frac{b}{{R}_{n}}+{\delta }_{\rm c}\right)\;,$
其中$ {T}_{\rm fixed} $表示传播与基础处理时延;$ {R}_{n} $表示当前瞬时通信速率;$ {\delta }_{\rm c} $表示单位 token 的云端边际验证成本。需要特别说明的是,本文中的上行传输时延与下行反馈时延仅表示与数据量及链路速率相关的串行化传输部分;传播时延表示无线链路传播、协议栈转发及往返中的固定延迟,不重复计入上行或下行串行化传输项;云端固定处理时延则表示请求分发、调度与验证准备等服务器侧开销。上述3类时延在模型中相互区分,不存在重复计数。云端固定处理时延项用于表示传播时延以及云端请求分发、调度与基础验证准备等平均开销。该项在本文中被近似视为短时间窗口内相对稳定的参数,以便于刻画通信与验证代价的主导结构。需要指出的是,在真实云系统中,队列积压、资源竞争与负载变化可能导致该项出现额外波动,并进一步影响最优步长估计的稳定性。本文的自适应控制器通过在线更新链路状态与接受统计量,能够在一定程度上缓解此类扰动带来的影响。
2)吞吐率最优策略
为最大化ETGR,我们求解最优投机步长:
$ K_{n}^{*}=\underset{K\in [1,{K}_{{\max }}]}{{\rm arg} \,\max } \frac{\mathbb{E}[\tau |K]}{{T}_{\rm fixed}+K\cdot {T}_{{\rm marginal}\left(n\right)}}\;。 $
该目标函数刻画了核心权衡关系:在弱信号环境下,通信成本主导分母增长,使最优$ K_{n}^{*} $倾向于较小值;而在良好信道条件下,较大步长能够有效摊薄往返时延开销,从而提升吞吐效率。需要说明的是,本文在推导中将单轮接受 token 的期望数量$ \mathbb{E}\left[\tau |{K}\right] $作为$ {K} $的函数进行近似建模,其目的在于刻画扩大步长所带来的潜在收益。严格而言,token 级接受事件并非独立同分布,相邻预测结果在上下文条件下通常具有相关性,因此$ \mathbb{E}\left[\tau |{K}\right] $与$ {K} $的关系在全局上未必严格线性。本文在理论分析中采用局部单调近似,用以揭示通信开销、验证成本与投机收益之间的核心权衡;在较大$ {K} $区域,接受收益通常呈现边际递减,这也构成了自适应步长机制的动机之一。本文在实现中采用有限候选集合上的轻量枚举搜索,即在$ K\in \{1{,}2,… ,{K}_{\max}\} $上根据当前链路速率、接受率估计与验证成本计算目标函数值,并选择使 ETGR 最大的步长。因此,单轮决策复杂度为 O($ {K}_{\max} $)。由于 $ {K}_{\max} $ 通常远小于生成序列长度,且该控制器仅涉及简单代数运算,其运行开销相较于边缘生成、无线传输与云端验证时延可以忽略。
运行时边缘与云端的交互在草稿模型版本上保持“无状态”,但在KV cache上保持“有状态”。本文统一采用“草稿生成—目标验证—失败回滚”的边云交互框架,但在不同生成机制下,候选 token 的产生与接受判定方式略有差异:在贪心解码设置下,草稿模型与目标模型均基于最大概率 token 进行确定性生成与验证;在 T=1、Top-p 采样设置下,草稿模型首先在截断后的候选集合内产生候选 token,云端再依据目标分布执行一致性验证。除候选生成与接受判定方式外,边云交互流程、KV cache 维护与回滚机制保持一致。该机制确保云端计算开销严格随实际验证长度扩展,而非随完整上下文长度增长,从而保留投机解码的核心效率优势。当云端目标模型持续演进时,ECSpec 通过轻量级回退机制优雅处理潜在分布偏移,在无需持续同步嵌入层或边缘参数的前提下保持稳定推理性能。
实验在分布偏移条件下评估ECSpec,覆盖多种目标模型与下游任务。
在任务泛化方面,为覆盖不同推理模式与数据分布,实验选取6类核心下游任务:数学推理(GSM8K)、问答(Natural Questions)、检索增强生成(Natural Questions)、多轮对话(MT-Bench)、机器翻译(WMT14 DE-EN)以及文档摘要(CNN/DailyMail)。
在目标模型方面,我们以 LLaMA-2-70B 作为典型稠密(dense)模型进行细致分析,并进一步在更近期的架构上验证可扩展性,包括 LLaMA-3-70B 以及混合专家模型 Mixtral 8$ \times $7B。
为平衡接受率估计的稳定性与响应速度,本文采用指数滑动平均(EMA)对在线接受率进行跟踪。设第t轮观测到的接受率为$ {\rho }_{\rm t} $,则平滑估计可写为 $ {\rho }_{\rm t}=\alpha {\rho }_{t-1}+(1-\alpha ){\rho }_{\rm t} $,其中$ \alpha \in [0{,}1) $为衰减系数。将ECSpec与覆盖多种解码范式的扩展基线集合进行比较,范围包括标准自回归方法以及最新投机解码框架。
1)Cloud-Only:完全在云端执行的标准自回归解码。该基线作为吞吐与时延的参考,其中每生成一个 token都需要承担完整的网络往返时延以及云端计算开销。
2)Lookahead[35]:基于Jacobi迭代的并行解码算法,无需单独草稿模型,可同时生成多个token,并通过多步验证并行细化候选序列。
3)Standard SD(Std. SD): 传统投机解码设置,使用通用的小型预训练模型(如 LLaMA-2-7B)作为草稿模型,为目标模型(如 LLaMA-2-70B)生成候选 token。该基线不包含本文提出的基于锚点的对齐机制,用于体现当草稿模型未随目标模型演进同步时由分布偏移带来的性能退化。
4)Medusa-1(Synced): 在目标模型上增加多个解码头以预测未来 token 的并行解码框架。与 EAGLE-2 类似,实验假设其头部与目标模型完美同步,用于表征紧耦合架构的理论上限。
5)EAGLE-2(Synced): 当前先进的模型式方法,利用分层特征外推。实验在“理想同步”设置下评估,即假设边缘端扩展层能够与云端目标版本完全一致,并忽略其同步更新所需的通信开销。
7)DSSD: 面向无线边云系统的协同推理框架。与 ECSpec不同,该方法采用固定或启发式的投机长度调度策略,缺乏对信道动态的实时自适应。
为模拟真实的跨设备部署场景,实验显著扩展了硬件环境的多样性。
1)云端服务器。通过NVLink互连的 8$ \times $NVIDIA H800(80 GB);8$ \times $NVIDIA A800(80 GB,主流配置);8$ \times $NVIDIA V100(32 GB)。
2)边缘设备。NVIDIA Jetson AGX Orin(64 GB 内存);Snapdragon 8 Gen 3 参考设备(16 GB 内存);Apple iPhone 15 Pro Max 模拟环境(A17 Pro);Raspberry Pi 5(8 GB 内存)。
为获得更细粒度的性能分析,实验将评估划分为2种生成机制:确定性生成(T=0)与随机采样(T=1)。实验给出了 LLaMA-2-70B 在6个数据集上的完整结果,以验证方法在不同任务类型下的鲁棒性。图2 展示了 GSM8K 上的性能差异,表3 给出了采用 greedy decoding 的详细结果。该设置更有利于依赖严格分布一致性的解码方法。此外,我们选择Cloud-Only为基线计算加速比,即用Cloud-Only的端到端时延除以相应方法的端到端时延。如表3 所示,ECSpec 在推理密集任务(如 GSM8K)与摘要任务(如 CNN/DailyMail)上均能够获得稳定加速。尤其在标准投机解码性能明显退化的场景中,ECSpec 仍可保持显著收益。例如在 WiFi 条件下,相较于Cloud-Only,Standard SD 加速比仅达到 0.66的性能,而 ECSpec 加速比仍可实现 2.30,表明锚点对齐机制能够有效缓解分布偏移导致的接受率下降。同时,尽管 EAGLE-2 等方法在 5G 条件下相较于Cloud-Only可获得最高加速比 2.47,但在 WiFi 场景下其加速比下降至 0.9以下,主要受通信开销限制。相比之下,ECSpec 通过自适应调整投机步长 $ K $,在弱网络条件下仍能够保持稳定加速,并在不同任务与网络环境中持续实现超过加速比为 2.1的整体性能提升。
表4 在 Top-p 采样(p = 0.9,T=1)条件下进一步评估系统性能。该设置对草稿模型与目标模型之间的对齐能力提出更高要求,因为草稿模型分布必须覆盖目标模型分布的主要概率质量。实验结果表明,在相同实验条件下相较于Cloud-Only,ECSpec在6个任务上仍能够保持T超过 2.0的稳定加速比,而 EAGLE-2 在T从 0 提升至 1 后性能明显下降,例如在 5G 条件下加速比从约 2.4 降至约 1.6。与此同时,Standard SD 出现明显的性能崩塌现象,在 GSM8K 弱网络环境下相较于Cloud-Only加速比仅达到 0.65,这是由于未对齐的草稿模型接受率较低,导致频繁验证失败。Lookahead 方法在该场景下收益同样有限(加速比低于 1.06)。相比之下,ECSpec 的锚点对齐能够保证草稿模型生成的 token 在统计意义上与目标模型分布保持一致,从而避免性能退化。
为验证信道感知自适应投机机制的有效性,实验在保持锚点对齐结构不变的条件下,将动态步长策略替换为固定步长策略($ K\in \{1{,}5,7\} $),并在 GSM8K 数据集及不同网络环境中进行测试。图3结果表明,较大的固定步长(如 $ K=5 $)在 5G 环境下具有较低时延(195.0 ms),但在 WiFi 环境中由于通信开销过大,时延增加至 1450.0 ms。相比之下,较小步长(如$ K=1 $)在WiFi环境下更加稳定,但在 5G 环境中时延为 400.0 ms,无法充分利用带宽资源。ECSpec 能够根据信道状态动态调整投机步长,在 5G 条件下实现 195.0 ms 的低时延,在 WiFi 环境下将时延控制在 540.0 ms,从而在不同网络条件下均能够保持较优性能。
表5 给出了 ECSpec 在不同边缘设备上的性能结果。实验表明,ECSpec 的性能取决于边缘草稿模型生成速度与网络通信节省之间的关系。Raspberry Pi 5 由于仅依赖 CPU,其草稿模型生成速度较慢,仅达到 6.9 token/s,导致整体加速比下降至 0.72~0.85。相比之下,具备 GPU 或 NPU 的设备均能够实现显著加速。例如,在 Jetson AGX Orin、iPhone 15 Pro Max 与 Snapdragon 8 Gen 3 上,ECSpec 均可实现超过 2.0 的加速比。在更复杂的 HumanEval 任务中,由于接受率下降,各设备加速比略有降低,例如Jetson AGX Orin加速比从 2.45 降至 2.20,但iPhone 15 Pro Max仍能够保持超过 2.08 的稳定加速比,说明该方法在复杂任务下仍具有良好适用性。
为验证方法的可扩展性,本文进一步在 LLaMA-3-70B 以及一个稀疏 MoE 模型 Mixtral 8×7B进行实验。尽管 LLaMA-3 具有更大的词汇表以及不同的训练数据,ECSpec在 5G 条件下仍实现了最高 2.56的加速比(数据集:MT-Bench,网络环境:5G/4G)。如表6所示,算法能够在不同的模型之间实现良好的迁移。对于 Mixtral 8×7B,由于采用条件计算,其基线云端推理速度本身更快。虽然这在一定程度上减少了执行所带来的潜在增益空间,但 ECSpec 在 4G 条件下仍实现了 1.98的加速比。信道感知策略能够自动向下调整$ K $的取值,以适应更快的云端验证速度,从而避免过度推理。
图4实验结果表明,全本地运行70B模型需要约 42.5 GB 显存,而 ECSpec 仅需约3.5 GB内存即可运行草稿模型,从而能够适配移动设备。能耗分析结果进一步表明,Cloud-Only推理能耗约为4.5J/token,而 ECSpec 通过减少通信时间,将能耗降低至 1.7 J/token,降低约62%。同时,ECSpec将计算负载转移至云端,避免了设备过热问题,从而提高系统稳定性。
本文针对大语言模型在动态边云协同部署中的可扩展性与通信效率问题展开研究。现有基于投机解码的方法依赖模型紧耦合同步与固定步长策略,在云端频繁更新与无线信道时变条件下易产生额外通信开销与性能退化,从而限制了大模型在边云环境中的规模化应用。为解决上述问题,本文提出 ECSpec,一种面向模型演进场景的通信高效边云协同推理框架。ECSpec 通过共享骨干架构设计,使单一且静态的边缘草稿模型能够兼容多个持续演进的云端目标模型,从而消除频繁边缘重训练与模型同步的需求,显著降低通信与维护成本。在此基础上,本文进一步提出信道感知的自适应投机机制,依据实时链路状态动态调整投机步长,在端到端时延、通信开销与设备能耗之间实现有效平衡,增强系统在复杂无线环境中的鲁棒性。
通过全面的实验评估,ECSpec 在多种网络条件、任务类型与模型架构下均表现出稳定且显著的性能优势,为动态边云环境中大语言模型的高效部署提供了一种可行的系统化解决方案。尽管 ECSpec 在通信效率与系统鲁棒性方面取得了良好效果,仍有若干值得进一步探索的方向。未来工作可沿以下5个方向进一步展开:1)面向高频模型更新场景,研究更精细的跨版本 drift 检测、触发式 re-anchor 机制以及低成本再对齐方法,以提升在持续演进环境中的长期稳定性;2)面向高动态无线网络,研究联合链路预测、拥塞控制与投机调度的跨层优化框架,以提升 handoff、弱覆盖与突发抖动条件下的端到端服务质量;3)面向异构终端,构建统一的时延-能耗-温度多目标在线优化方法,以支持不同设备形态下的自适应推理;4)探索ECSpec在稀疏模型、多模态模型与长上下文生成任务中的适用性边界;5)围绕边云协同过程中的隐私保护、缓存安全与推理可信性开展系统化研究。

参考文献 引证文献
排序方式:
1
Achiam J, Adler S, Agarwal S, et al. GPT-4 technical report[J]. arXiv preprint, arXiv: 2303.08774, 2023
2
Touvron H, Martin L, Stone K, et al. LLaMA 2: Open foundation and fine-tuned chat models[J]. arXiv preprint, arXiv: 2307.09288, 2023
3
Team G, Anil R, Borgeaud S, et al. Gemini: A family of highly capable multimodal models[J]. arXiv preprint, arXiv: 2312.11805, 2023
4
Li Yuchen, Cai Hengyi, Kong Rui, et al. Towards AI search paradigm[J]. arXiv preprint, arXiv: 2506.17188, 2025
5
Xiong Haoyi, Bian Jiang, Li Yuchen, et al. When search engine services meet large language models: Visions and challenges[J]. IEEE Transactions on Services Computing, 2024, 17(6): 4558−4577
6
Liu Zechun, Zhao Changsheng, Iandola F, et al. MobileLLM: Optimizing sub-billion parameter language models for on-device use cases[C]//Proc of the 41st Int Conf on Machine Learning (ICML). New York: PMLR, 2024: 32431–32454
7
Xu Dianlei, Li Tong, Li Yong, et al. Edge intelligence: Empowering intelligence to the edge of network[J]. Proceedings of the IEEE, 2021, 109(11): 1778−1837
8
Liu Jing, Du Yao, Yang Kun, et al. Edge-cloud collaborative computing on distributed intelligence and model optimization: A survey[J]. arXiv preprint, arXiv: 2505.01821, 2025
9
Lyu Zhonghao, Li Yuchen, Zhu Guangxu, et al. Rethinking resource management in edge learning: A joint pre-training and fine-tuning design paradigm[J]. IEEE Transactions on Wireless Communications, 2025, 24(2): 1584−1601
10
Leviathan Y, Kalman M, Matias Y. Fast inference from Transformers via speculative decoding[C]//Proc of the 40th Int Conf on Machine Learning (ICML). New York: PMLR, 2023: 19274–19286
11
Li Yuhui, Wei Fangyun, Zhang Chao, et al. EAGLE-2: Faster inference of language models with dynamic draft trees[C]//Proc of Empirical Methods in Natural Language Processing (EMNLP). Stroudsburg, PA: ACL, 2024: 7421–7432
12
Cai Tianle, Li Yuhong, Geng Zhengyang, et al. Medusa: Simple LLM inference acceleration framework with multiple decoding heads[J]. arXiv preprint, arXiv: 2401.10774, 2024
13
Chen C, Borgeaud S, Irving G, et al. Accelerating large language model decoding with speculative sampling[J]. arXiv preprint, arXiv: 2302.01318, 2023
14
Li Yuhui, Wei Fangyun, Zhang Chao, et al. EAGLE: Speculative sampling requires rethinking feature uncertainty[C]//Proc of the Int Conf on Machine Learning (ICML). New York: PMLR, 2024: 28935–28948
15
Yang Nan, Ge Tao, Wang Liang, et al. Inference with reference: Lossless acceleration of large language models[J]. arXiv preprint, arXiv: 2304.04487, 2023
16
He Zhenyu, Zhong Zexuan, Cai Tianle, et al. REST: Retrieval-based speculative decoding[C]//Proc of the Conf of the North American Chapter of the Association for Computational Linguistics (NACCL). Stroudsburg, PA: ACL, 2024: 1582–1595
17
Hu E J, Shen Y, Wallis P, et al. LoRA: Low-rank adaptation of large language models[C]//Proc of the Int Conf on Learning Representations (ICLR). Open Access: OpenReview. net, 2022: 1–13
18
Lyu Zhonghao, Zhu Guangxu, Xu Jie, et al. Semantic communications for image recovery and classification via deep joint source and channel coding[J]. IEEE Transactions on Wireless Communications, 2024, 23(8): 8388−8404
19
Kang Yiping, Hauswald J, Gao Cao, et al. Neurosurgeon: Collaborative intelligence between the cloud and mobile edge[C]//Proc of the 22nd Int Conf on Architectural Support for Programming Languages and Operating Systems (ASPLOS). New York: ACM, 2017: 615–629
20
Matsubara Y, Levorato M, Restuccia F. Split computing and early exit for deep learning applications: Survey and research challenges[J]. ACM Computing Surveys, 2024, 55(5): 1−30
21
Borzunov A, Baranchuk D, Dettmers T, et al. Petals: Collaborative inference and fine-tuning of large models[C]//Proc of Workshop on Broadening Research Collaborations. Stroudsburg, PA: ACL, 2022: 558–568
22
Ryabinin M, Dettmers T, Diskin M, et al. SWARM parallelism: Training large models can be surprisingly communication-efficient[C]//Proc of the Int Conf on Machine Learning (ICML). New York: PMLR, 2023: 29416–29440
23
Lyu Zhonghao, Xiao Ming, Xu Jie, et al. The larger the merrier? Efficient large AI model inference in wireless edge networks[J]. IEEE Journal on Selected Areas in Communications, 2025, 44: 2839−2853
24
Frantar E, Ashkboos S, Hoefler T, et al. GPTQ: Accurate post-training compression for generative pretrained transformers[C]//Proc of the Int Conf on Learning Representations (ICLR). (ICLR). Open Access: OpenReview. net, 2023: 1–16
25
Lin Ji, Tang Jiaming, Tang Haotian, et al. AWQ: Activation-aware weight quantization for on-device LLM compression and acceleration[J]. GetMobile: Mobile Computing and Communications, 2025, 28(4): 12−17
26
Li Muyang, Cai Tianle, Cao Jiaxin, et al. DistriFusion: Distributed parallel inference for high-resolution diffusion models[C]//Proc of the IEEE/CVF Conf on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ: IEEE, 2024: 7183–7193
27
过敏意. 大模型时代网络基础设施的机遇与挑战[J]. 计算机研究与发展, 2024, 61(11): 2663−2663
Guo Minyi. Opportunities and challenges of network infrastructure in the era of large language model[J]. Journal of Computer Research and Development, 2024, 61(11): 2663−2663 (in Chinese)
28
Park Jihong, Samarakoon S, Bennis M, et al. Wireless network intelligence at the edge[J]. Proceedings of the IEEE, 2019, 107(11): 2204−2239
29
Zhu Guangxu, Liu Dongzhu, Du Yuqing, et al. Toward an intelligent edge: Wireless communication meets machine learning[J]. IEEE Communications Magazine, 2020, 58(1): 19−25
30
Xin Ji, Tang R, Lee J, et al. DeeBERT: Dynamic early exiting for accelerating BERT inference[C]//Proc of the 58th Annual Meeting of the Association for Computational Linguistics (ACL). Stroudsburg, PA: ACL, 2020: 2246–2251
31
Laskaridis S, Venieris S I, Almeida M, et al. SPINN: Synergistic progressive inference of neural networks over device and cloud[C]//Proc of the Annual Int Conf on Mobile Computing and Networking (MobiCom). New York: ACM, 2020: 1–15
32
Ning Jiahong, Zheng Ce, Yang Tingting. DSSD: Efficient edge-device deployment and collaborative inference via distributed split speculative decoding[C]//Proc of ICML on Machine Learning for Wireless Communication and Networks (ML4Wireless). New York: PMLR, 2025: 1–8
33
Malladi S, Gao T, Nichani E, et al. Fine-tuning language models with just forward passes[C]//Proc of the 37th Int Conf on Neural Information Processing Systems (NIPS). New York: Curran Associates, 2023: 53038–53075
34
Gu Yuxian, Dong Li, Wei Furu, et al. MiniLLM: Knowledge distillation of large language models[C]//Proc of the Int Conf on Learning Representations (ICLR). Open Access: OpenReview. net, 2024: 1–24
35
Zhao Yao, Xie Zhitian, Liang Chen, et al. Lookahead: An inference acceleration framework for large language model with lossless generation accuracy[C]//Proc of the 30th ACM SIGKDD Conf on Knowledge Discovery and Data Mining. New York: ACM, 2024: 6344–6355
2026年第63卷第8期
84
41
0
引用本文
BibTeX
文章信息
doi: 10.7544/issn1000-1239.202660136
  • 接收时间:2026-03-17
  • 首发时间:2026-11-17
  • 出版时间:2026-08-01
补充材料
相关文章
文章信息
作者
出版历史
  • 收稿日期:2026-03-17
  • 修回日期:2026-04-20
基金
作者信息
    1上海交通大学计算机学院 上海 200240
    2百度时代网络技术(北京)有限公司 北京 100085

通讯作者:

参考文献
分享链接
https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/10.7544/issn1000-1239.202660136
分享至
全文二维码

扫描看全文

引用本文
BibTeX
本文的引用情况
2种不同金属材料的力学参数
科
Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)
属
Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏