Article(id=1325493670253822813, tenantId=1146029695717560320, journalId=1324759914146529478, issueId=1325493569112388051, articleNumber=null, orderNo=null, doi=10.7544/issn1000-1239.202550002, pmid=null, cstr=null, oa=null, hot=null, price=null, onlineType=0, articleFormat=0, articleType=null, articleTypeStr=research-article, receivedDate=1735660800000, receivedDateStr=2025-01-01, revisedDate=1754841600000, revisedDateStr=2025-08-11, acceptedDate=1756742400000, acceptedDateStr=2025-09-02, onlineDate=1793951917470, onlineDateStr=2026-11-06, pubDate=1769875200000, pubDateStr=2026-02-01, doiRegisterDate=null, doiRegisterDateStr=null, onlineIssueDate=1793951917470, onlineIssueDateStr=2026-11-06, onlineJustAcceptDate=null, onlineJustAcceptDateStr=null, onlineFirstDate=null, onlineFirstDateStr=null, sourceXml=null, magXml=null, createTime=1793951917470, creator=13701087609, updateTime=1793951917470, updator=13701087609, issue=Issue{id=1325493569112388051, tenantId=1146029695717560320, journalId=1324759914146529478, year='2026', volume='63', issue='2', pageStart='275', pageEnd='538', issueExtLink='null', onlineDate='null', pubDate='1769875200000', pubDateStr='2026-02-01', beforeIssueId=null, nextIssueId=null, price=null, status=1, issueComplete=1, articleOrder=1, issueType=-1, specialIssue=null, createTime=1793951893355, creator='13701087609', updateTime=1793952511329, updator='13701087609', preIssue=null, nextIssue=null, articleTotal=null, ext={EN=IssueExt(id=1325496161167090054, tenantId=1146029695717560320, journalId=1324759914146529478, issueId=1325493569112388051, language=EN, specialIssueTitle=, coverIllustrator=null, specialIssueEditor=, specialIssueAbout=), CN=IssueExt(id=1325496161167090055, tenantId=1146029695717560320, journalId=1324759914146529478, issueId=1325493569112388051, language=CN, specialIssueTitle=, coverIllustrator=null, specialIssueEditor=, specialIssueAbout=)}, issueFiles=null, downloadFileDto=null}, startPage=505, endPage=524, ext={EN=ArticleExt(id=1325493671914767199, articleId=1325493670253822813, tenantId=1146029695717560320, journalId=1324759914146529478, language=EN, title=Survey on Reinforcement Learning-Based Service Function Chain Deployment, columnId=1325493671541474142, journalTitle=Journal of Computer Research and Development, columnName=Network and Information Security, runingTitle=null, highlight=null, articleAbstract=

With the rapid development of emerging network technologies such as B5G/6G and Internet of things, network services present characteristics of diverse business types, differentiated quality requirements, and dynamic resource demands. Under the software defined networking (SDN) and network function virtualization (NFV) paradigm, service function chaining (SFC) technology enables customized network service deployment through flexible orchestration of virtual network functions (VNF). However, SFC deployment in dynamic network environments faces challenges such as large decision space, and complex and changing environments. Reinforcement learning (RL) demonstrates significant advantages in solving SFC deployment problems due to its capability to adaptively learn complex environmental characteristics and make dynamic decisions. Although research on applying RL to SFC deployment has made certain progress, a systematic and comprehensive review analysis remains lacking. To address this gap, the fundamental concepts and technical architecture of SFC deployment are elaborated, along with a specific introduction to the RL-based SFC deployment framework. Subsequently, from the perspectives of algorithm design, application scenarios, and optimization strategies, the research progress and innovative applications of reinforcement learning in three key phases—SFC placement, scheduling, and reconfiguration—are systematically examined and thoroughly analyzed. Finally, the advantages and limitations of existing research in terms of algorithm design, performance optimization, and practical deployment are summarized, while technical challenges and future development trends in this field are analyzed.

, authors=Yintan Ai1, Hongwei Ruan1, Hanlin Liu1, Hua Li1, 2, 3, *, authorsList=Yintan Ai, Hongwei Ruan, Hanlin Liu, Hua Li, authorCompany=null, correspAuthors=Hua Li, authorNote=

born in 1994. PhD candidate. His main research interests include software-defined networking, service function chaining, and multi-agent reinforcement learning

born in 1981. PhD. His main research interests include software-defined networking and software testing. ()

born in 1996. PhD candidate. Her main research interests include intent-based networking and graph neural networks. ()

born in 1964. PhD, professor, PhD supervisor. Her main research interests include cloud-network integration, big data analysis and evaluation methods, and software service computing and testing

, correspAuthorsNote=null, copyrightStatement=Copyright © 2026 Journal of Computer Research and Development. All rights reserved., copyrightOwner=null, extLink=null, articleAbsUrl=null, sourceXml=null, magXml=null, pdfUrl=null, pdf=null, pdfFileSize=null, pdfExtLink=null, richHtmlUrl=null, mobilePdfUrl=null, reviewReport=null, pdfFirstPage=null, abstractGraph=null, abstractGraphContent=null, abstractVideo=null, citation=null, cebUrl=null, magXmlContent=null, mapNumber=null, fund=null), CN=ArticleExt(id=1325493681926570873, articleId=1325493670253822813, tenantId=1146029695717560320, journalId=1324759914146529478, language=CN, title=基于强化学习的服务功能链部署综述, columnId=1325493672690713440, journalTitle=计算机研究与发展, columnName=网络与信息安全, runingTitle=null, highlight=null, articleAbstract=

在软件定义网络和网络功能虚拟化范式下,服务功能链(service function chaining,SFC)技术通过灵活编排虚拟网络功能实现了网络服务的定制化部署。然而,动态网络环境下SFC部署面临着决策空间大、环境复杂多变等挑战。强化学习因其自适应学习复杂环境特征并动态决策的能力,在解决SFC部署问题上展现出显著优势。该研究首先阐述了SFC部署的基本概念与技术架构,并具体介绍了基于强化学习的SFC部署框架。随后,从算法设计、应用场景和优化策略等角度,梳理并深入分析了强化学习在SFC放置、调度和重配置3个关键阶段中的研究进展与创新应用。最后,总结了现有研究在算法设计、性能优化和实际部署方面的优势与局限,并分析了该领域的技术挑战与未来发展趋势。

, authors=爱因坦1, 阮宏玮1, 刘翰林1, 李华1, 2, 3, *, authorsList=爱因坦, 阮宏玮, 刘翰林, 李华, authorCompany=null, correspAuthors=李华, authorNote=

1994年生。博士研究生。主要研究方向为软件定义网络、服务功能链、多智能体强化学习

1981年生。博士。主要研究方向为软件定义网络、软件测试

1996年生。博士研究生。主要研究方向为意图驱动网络、图神经网络

1964年生。博士,教授,博士生导师。主要研究方向为云网融合、大数据分析与评估方法、软件服务计算与测试

, correspAuthorsNote=
李华()
, copyrightStatement=版权所有 © 2026 计算机研究与发展编辑部, copyrightOwner=null, extLink=null, articleAbsUrl=null, sourceXml=p2ZK6cjVYg8he6UGnwuoSg==, magXml=qwHb7ytJ3rWon9g30y9Ebw==, pdfUrl=null, pdf=4wqAvV21WhWNx9Mi+bpQBQ==, pdfFileSize=1134713, pdfExtLink=null, richHtmlUrl=null, mobilePdfUrl=null, reviewReport=null, pdfFirstPage=null, abstractGraph=OT331tEXFjwTuusXEoYSXQ==, abstractGraphContent=null, abstractVideo=null, citation=null, cebUrl=null, magXmlContent=oU7CiY4A41StkSsYI+xmmw==, mapNumber=null, fund=null)}, authors=[Author(id=1325493689522455429, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, orderNo=0, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=aiyintan@mail.imu.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1325493689904137095, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, authorId=1325493689522455429, language=EN, stringName=Yintan Ai, firstName=Yintan, middleName=null, lastName=Ai, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1College of Computer Science, Inner Mongolia University, Hohhot 010021, bio={"content":"

born in 1994. PhD candidate. His main research interests include software-defined networking, service function chaining, and multi-agent reinforcement learning

"}, bioImg=null, bioContent=

born in 1994. PhD candidate. His main research interests include software-defined networking, service function chaining, and multi-agent reinforcement learning

, aboutCorrespAuthor=null), CN=AuthorExt(id=1325493690390676360, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, authorId=1325493689522455429, language=CN, stringName=爱因坦, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1内蒙古大学计算机学院 呼和浩特 010021, bio={"img":"9hXPvmADSjdXt/3DBe4MvA==","content":"

1994年生。博士研究生。主要研究方向为软件定义网络、服务功能链、多智能体强化学习

"}, bioImg=9hXPvmADSjdXt/3DBe4MvA==, bioContent=

1994年生。博士研究生。主要研究方向为软件定义网络、服务功能链、多智能体强化学习

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1325493684887749498, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, xref=1, ext=[AuthorCompanyExt(id=1325493685646918523, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, companyId=1325493684887749498, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1College of Computer Science, Inner Mongolia University, Hohhot 010021), AuthorCompanyExt(id=1325493685734998908, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, companyId=1325493684887749498, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1内蒙古大学计算机学院 呼和浩特 010021)])]), Author(id=1325493691191788426, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, orderNo=1, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=csrhw@imu.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1325493691984511884, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, authorId=1325493691191788426, language=EN, stringName=Hongwei Ruan, firstName=Hongwei, middleName=null, lastName=Ruan, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1College of Computer Science, Inner Mongolia University, Hohhot 010021, bio={"content":"

born in 1981. PhD. His main research interests include software-defined networking and software testing. ()

"}, bioImg=null, bioContent=

born in 1981. PhD. His main research interests include software-defined networking and software testing. ()

, aboutCorrespAuthor=null), CN=AuthorExt(id=1325493693632873357, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, authorId=1325493691191788426, language=CN, stringName=阮宏玮, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1内蒙古大学计算机学院 呼和浩特 010021, bio={"img":"VJ2TOW7EPUJtSWrVbLpJpA==","content":"

1981年生。博士。主要研究方向为软件定义网络、软件测试

"}, bioImg=VJ2TOW7EPUJtSWrVbLpJpA==, bioContent=

1981年生。博士。主要研究方向为软件定义网络、软件测试

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1325493684887749498, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, xref=1, ext=[AuthorCompanyExt(id=1325493685646918523, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, companyId=1325493684887749498, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1College of Computer Science, Inner Mongolia University, Hohhot 010021), AuthorCompanyExt(id=1325493685734998908, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, companyId=1325493684887749498, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1内蒙古大学计算机学院 呼和浩特 010021)])]), Author(id=1325493693775479695, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, orderNo=2, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=imucslhl@mail.imu.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1325493694870193041, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, authorId=1325493693775479695, language=EN, stringName=Hanlin Liu, firstName=Hanlin, middleName=null, lastName=Liu, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1College of Computer Science, Inner Mongolia University, Hohhot 010021, bio={"content":"

born in 1996. PhD candidate. Her main research interests include intent-based networking and graph neural networks. ()

"}, bioImg=null, bioContent=

born in 1996. PhD candidate. Her main research interests include intent-based networking and graph neural networks. ()

, aboutCorrespAuthor=null), CN=AuthorExt(id=1325493694966662034, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, authorId=1325493693775479695, language=CN, stringName=刘翰林, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1内蒙古大学计算机学院 呼和浩特 010021, bio={"img":"D33AmwLpNXR1CyGpMxJd+Q==","content":"

1996年生。博士研究生。主要研究方向为意图驱动网络、图神经网络

"}, bioImg=D33AmwLpNXR1CyGpMxJd+Q==, bioContent=

1996年生。博士研究生。主要研究方向为意图驱动网络、图神经网络

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1325493684887749498, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, xref=1, ext=[AuthorCompanyExt(id=1325493685646918523, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, companyId=1325493684887749498, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1College of Computer Science, Inner Mongolia University, Hohhot 010021), AuthorCompanyExt(id=1325493685734998908, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, companyId=1325493684887749498, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1内蒙古大学计算机学院 呼和浩特 010021)])]), Author(id=1325493695398675348, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, orderNo=3, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=cslihua@imu.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=1, authorType=1, ext={EN=AuthorExt(id=1325493696673743769, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, authorId=1325493695398675348, language=EN, stringName=Hua Li, firstName=Hua, middleName=null, lastName=Li, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, 3, *, address=1College of Computer Science, Inner Mongolia University, Hohhot 010021
2Engineering Research Center of Ecological Big Data (Inner Mongolia University), Ministry of Education, Hohhot 010021
3Inner Mongolia Engineering Laboratory for Cloud Computing and Service Software (Inner Mongolia University), Hohhot 010021, bio={"content":"

born in 1964. PhD, professor, PhD supervisor. Her main research interests include cloud-network integration, big data analysis and evaluation methods, and software service computing and testing

"}, bioImg=null, bioContent=

born in 1964. PhD, professor, PhD supervisor. Her main research interests include cloud-network integration, big data analysis and evaluation methods, and software service computing and testing

, aboutCorrespAuthor=null), CN=AuthorExt(id=1325493698351465370, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, authorId=1325493695398675348, language=CN, stringName=李华, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, 3, *, address=1内蒙古大学计算机学院 呼和浩特 010021
2生态大数据教育部工程研究中心(内蒙古大学) 呼和浩特 010021
3内蒙古自治区云计算与服务软件工程实验室(内蒙古大学) 呼和浩特 010021, bio={"img":"spUYtazfBuMiefNWiI0mDg==","content":"

1964年生。博士,教授,博士生导师。主要研究方向为云网融合、大数据分析与评估方法、软件服务计算与测试

"}, bioImg=spUYtazfBuMiefNWiI0mDg==, bioContent=

1964年生。博士,教授,博士生导师。主要研究方向为云网融合、大数据分析与评估方法、软件服务计算与测试

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1325493684887749498, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, xref=1, ext=[AuthorCompanyExt(id=1325493685646918523, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, companyId=1325493684887749498, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1College of Computer Science, Inner Mongolia University, Hohhot 010021), AuthorCompanyExt(id=1325493685734998908, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, companyId=1325493684887749498, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1内蒙古大学计算机学院 呼和浩特 010021)]), AuthorCompany(id=1325493686091514749, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, xref=2, ext=[AuthorCompanyExt(id=1325493686494167934, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, companyId=1325493686091514749, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2Engineering Research Center of Ecological Big Data (Inner Mongolia University), Ministry of Education, Hohhot 010021), AuthorCompanyExt(id=1325493686557082495, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, companyId=1325493686091514749, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2生态大数据教育部工程研究中心(内蒙古大学) 呼和浩特 010021)]), AuthorCompany(id=1325493687366583168, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, xref=3, ext=[AuthorCompanyExt(id=1325493687404331905, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, companyId=1325493687366583168, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=3Inner Mongolia Engineering Laboratory for Cloud Computing and Service Software (Inner Mongolia University), Hohhot 010021), AuthorCompanyExt(id=1325493687416914818, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, companyId=1325493687366583168, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=3内蒙古自治区云计算与服务软件工程实验室(内蒙古大学) 呼和浩特 010021)])])], keywords=[Keyword(id=1325493699479733147, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, language=EN, orderNo=1, keyword=service function chaining (SFC)), Keyword(id=1325493699928523678, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, language=EN, orderNo=2, keyword=reinforcement learning), Keyword(id=1325493700343759775, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, language=EN, orderNo=3, keyword=resource optimization), Keyword(id=1325493700800938912, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, language=EN, orderNo=4, keyword=virtual network function), Keyword(id=1325493702843564962, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, language=EN, orderNo=5, keyword=intelligent deployment), Keyword(id=1325493703258801059, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, language=CN, orderNo=1, keyword=服务功能链), Keyword(id=1325493703695008676, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, language=CN, orderNo=2, keyword=强化学习), Keyword(id=1325493704114439077, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, language=CN, orderNo=3, keyword=资源优化), Keyword(id=1325493704940716966, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, language=CN, orderNo=4, keyword=虚拟网络功能), Keyword(id=1325493705041380264, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, language=CN, orderNo=5, keyword=智能部署)], refs=[Reference(id=1325493716051428288, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=1, rfOrder=0, authorNames=null, journalName=null, refType=null, unstructuredReference=International Telecommunication Union (ITU). ICT facts and figures 2024[R]. Geneva: International Telecommunication Union, 2024, articleTitle=null, refAbstract=null), Reference(id=1325493716135314369, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=2, rfOrder=1, authorNames=null, journalName=null, refType=null, unstructuredReference=CISCO. 2024 global networking trends report[R]. San Jose: Cisco Systems Inc, 2024, articleTitle=null, refAbstract=null), Reference(id=1325493716277920706, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=3, rfOrder=2, authorNames=null, journalName=null, refType=null, unstructuredReference=Mijumbi R, Serrat J, Gorricho J L, et al. Network function virtualization: State-of-the-art and research challenges[J]. IEEE Communications Surveys & Tutorials, 2015, 18(1): 236−262, articleTitle=null, refAbstract=null), Reference(id=1325493716366001091, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=4, rfOrder=3, authorNames=null, journalName=null, refType=null, unstructuredReference=Kreutz D, Ramos F M V, Verissimo P E, et al. Software-defined networking: A comprehensive survey[J]. Proceedings of the IEEE, 2014, 103(1): 14−76, articleTitle=null, refAbstract=null), Reference(id=1325493716449887172, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=5, rfOrder=4, authorNames=null, journalName=null, refType=null, unstructuredReference=Halpern J, Pignataro C. Service function chaining (SFC) architecture[R]. Fremont, CA: Internet Engineering Task Force, 2015, articleTitle=null, refAbstract=null), Reference(id=1325493716735099845, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=6, rfOrder=5, authorNames=null, journalName=null, refType=null, unstructuredReference=Bhamare D, Jain R, Samaka M, et al. A survey on service function chaining[J]. Journal of Network and Computer Applications, 2016, 75: 138−155, articleTitle=null, refAbstract=null), Reference(id=1325493716810597318, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=7, rfOrder=6, authorNames=null, journalName=null, refType=null, unstructuredReference=Xiao Yikai, Zhang Qixia, Liu Fangming, et al. NFVdeep: Adaptive online service function chain deployment with deep reinforcement learning[C]//Proc of the 27th Int Symp on Quality of Service. New York: ACM, 2019: 1−10, articleTitle=null, refAbstract=null), Reference(id=1325493716898677703, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=8, rfOrder=7, authorNames=null, journalName=null, refType=null, unstructuredReference=Kaelbling L P, Littman M L, Moore A W. Reinforcement learning: A survey[J]. Journal of Artificial Intelligence Research, 1996, 4: 237−285, articleTitle=null, refAbstract=null), Reference(id=1325493717167113160, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=9, rfOrder=8, authorNames=null, journalName=null, refType=null, unstructuredReference=Heo D N, Lange S, Kim H G, et al. Graph neural network based service function chaining for automatic network control[C]//Proc of the 21st Asia-Pacific Network Operations and Management Symp (APNOMS). Piscataway, NJ: IEEE, 2020: 7−12, articleTitle=null, refAbstract=null), Reference(id=1325493717280359369, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=10, rfOrder=9, authorNames=null, journalName=null, refType=null, unstructuredReference=Shah H A, Zhao Lian. Multiagent deep-reinforcement-learning-based virtual resource allocation through network function virtualization in Internet of things[J]. IEEE Internet of Things Journal, 2020, 8(5): 3410−3421, articleTitle=null, refAbstract=null), Reference(id=1325493717334885322, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=11, rfOrder=10, authorNames=null, journalName=null, refType=null, unstructuredReference=Cai Xuhui, Deng Hui, Deng Lingli, et al. Evolving NFV towards the next decade[R]. Sophia Antipolis, France: European Telecommunications Standards Institute, 2023, articleTitle=null, refAbstract=null), Reference(id=1325493717410382795, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=12, rfOrder=11, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhang Biao, Fan Qilin, Zhang Xu, et al. A survey of VNF forwarding graph embedding in B5G/6G networks[J]. Wireless Networks, 2024, 30(5): 3735−3758, articleTitle=null, refAbstract=null), Reference(id=1325493717477491660, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=13, rfOrder=12, authorNames=null, journalName=null, refType=null, unstructuredReference=周伟林, 杨芫, 徐明伟. 网络功能虚拟化技术研究综述[J]. 计算机研究与发展, 2018, 55(4): 675−688, articleTitle=null, refAbstract=null), Reference(id=1325493717582349261, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=13, rfOrder=13, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhou Weilin, Yang Yuan, Xu Mingwei. Network function virtualization technology research[J]. Journal of Computer Research and Development, 2018, 55(4): 675−688 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1325493717657846734, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=14, rfOrder=14, authorNames=null, journalName=null, refType=null, unstructuredReference=Kaur K, Mangat V, Kumar K. A comprehensive survey of service function chain provisioning approaches in SDN and NFV architecture[J]. Computer Science Review, 2020, 38: 100298, articleTitle=null, refAbstract=null), Reference(id=1325493717737538511, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=15, rfOrder=15, authorNames=null, journalName=null, refType=null, unstructuredReference=Schardong F, Nunes I, Schaeffer-Filho A. NFV resource allocation: A systematic review and taxonomy of VNF forwarding graph embedding[J]. Computer Networks, 2021, 185: 107726, articleTitle=null, refAbstract=null), Reference(id=1325493717959836624, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=16, rfOrder=16, authorNames=null, journalName=null, refType=null, unstructuredReference=祖家琛, 胡谷雨, 严佳洁, 等. 网络功能虚拟化下服务功能链的资源管理研究综述[J]. 计算机研究与发展, 2021, 58(1): 137−152, articleTitle=null, refAbstract=null), Reference(id=1325493718047917009, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=16, rfOrder=17, authorNames=null, journalName=null, refType=null, unstructuredReference=Zu Jiachen, Hu Guyu, Yan Jiajie, et al. Resource management of service function chain in NFV enabled network: A survey[J]. Journal of Computer Research and Development, 2021, 58(1): 137−152 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1325493718106637266, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=17, rfOrder=18, authorNames=null, journalName=null, refType=null, unstructuredReference=Yang Song, Li Fan, Trajanovski S, et al. Recent advances of resource allocation in network function virtualization[J]. IEEE Transactions on Parallel and Distributed Systems, 2021, 32(2): 295−314, articleTitle=null, refAbstract=null), Reference(id=1325493718190523347, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=18, rfOrder=19, authorNames=null, journalName=null, refType=null, unstructuredReference=Santos G L, Bezerra D F, Rocha E S, et al. Service function chain placement in distributed scenarios: A systematic review[J]. Journal of Network and Systems Management, 2022, 30(1): 4, articleTitle=null, refAbstract=null), Reference(id=1325493718257632212, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=19, rfOrder=20, authorNames=null, journalName=null, refType=null, unstructuredReference=Pattaranantakul M, Vorakulpipat C, Takahashi T. Service function chaining security survey: Addressing security challenges and threats[J]. Computer Networks, 2023, 221: 109484, articleTitle=null, refAbstract=null), Reference(id=1325493718328935381, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=20, rfOrder=21, authorNames=null, journalName=null, refType=null, unstructuredReference=ETSI ISG NFV. Network functions virtualisation (NFV): Architectural framework[R]. Sophia Antipolis, France: European Telecommunications Standards Institute, 2013, articleTitle=null, refAbstract=null), Reference(id=1325493718391849942, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=21, rfOrder=22, authorNames=null, journalName=null, refType=null, unstructuredReference=John W, Pentikousis K, Agapiou G, et al. Research directions in network service chaining[C]//Proc of the 2013 IEEE SDN for Future Networks and Services (SDN4FNS). Piscataway, NJ: IEEE, 2013: 1−7, articleTitle=null, refAbstract=null), Reference(id=1325493718458958807, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=22, rfOrder=23, authorNames=null, journalName=null, refType=null, unstructuredReference=Bjorklund M. YANG—A data modeling language for the network configuration protocol (NETCONF)[R]. Fremont, CA: Internet Engineering Task Force, 2010, articleTitle=null, refAbstract=null), Reference(id=1325493718521873368, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=23, rfOrder=24, authorNames=null, journalName=null, refType=null, unstructuredReference=Mehraghdam S, Keller M, Karl H. Specifying and placing chains of virtual network functions[C]//Proc of the 3rd IEEE Int Conf on Cloud Networking (CloudNet). Piscataway, NJ: IEEE, 2014: 7−13, articleTitle=null, refAbstract=null), Reference(id=1325493718593176537, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=24, rfOrder=25, authorNames=null, journalName=null, refType=null, unstructuredReference=Binz T, Breitenbücher U, Kopp O, et al. TOSCA: Portable Automated Deployment and Management of Cloud Applications[M]//Advanced Web Services. Berlin: Springer, 2013: 527−549, articleTitle=null, refAbstract=null), Reference(id=1325493718668674010, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=25, rfOrder=26, authorNames=null, journalName=null, refType=null, unstructuredReference=Sun Chen, Bi Jun, Zheng Zhilong, et al. NFP: Enabling network function parallelism in NFV[C]//Proc of the Conf of the ACM Special Interest Group on Data Communication. New York: ACM, 2017: 43−56, articleTitle=null, refAbstract=null), Reference(id=1325493718739977179, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=25, rfOrder=27, authorNames=null, journalName=null, refType=null, unstructuredReference=Sun Chen, Bi Jun, Zheng Zhilong, et al. NFP: Enabling network function parallelism in NFV[C]//Proc of the Conf of the ACM Special Interest Group on Data Communication. New York: ACM, 2017: 43−56, articleTitle=null, refAbstract=null), Reference(id=1325493720438670300, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=26, rfOrder=28, authorNames=null, journalName=null, refType=null, unstructuredReference=Lin I C, Yeh Y H, Lin K C J. Toward optimal partial parallelization for service function chaining[J]. IEEE/ACM Transactions on Networking, 2021, 29(5): 2033−2044, articleTitle=null, refAbstract=null), Reference(id=1325493720518362077, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=27, rfOrder=29, authorNames=null, journalName=null, refType=null, unstructuredReference=Bari F, Chowdhury S R, Ahmed R, et al. Orchestrating virtualized network functions[J]. IEEE Transactions on Network and Service Management, 2016, 13(4): 725−739, articleTitle=null, refAbstract=null), Reference(id=1325493720589665246, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=28, rfOrder=30, authorNames=null, journalName=null, refType=null, unstructuredReference=Addis B, Belabed D, Bouet M, et al. Virtual network functions placement and routing optimization[C]//Proc of the 4th IEEE Int Conf on Cloud Networking (CloudNet). Piscataway, NJ: IEEE, 2015: 171−177, articleTitle=null, refAbstract=null), Reference(id=1325493720816157663, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=29, rfOrder=31, authorNames=null, journalName=null, refType=null, unstructuredReference=刘艺, 张红旗, 杨英杰, 等. 一种区分等级的可生存服务功能链映射方法[J]. 计算机研究与发展, 2018, 55(4): 748−767, articleTitle=null, refAbstract=null), Reference(id=1325493720887460832, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=29, rfOrder=32, authorNames=null, journalName=null, refType=null, unstructuredReference=Liu Yi, Zhang Hongqi, Yang Yingjie, et al. A hierarchical method for survivable service function chain embedding[J]. Journal of Computer Research and Development, 2018, 55(4): 748−767 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1325493720958764001, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=30, rfOrder=33, authorNames=null, journalName=null, refType=null, unstructuredReference=Riera J F, Hesselbach X, Escalona E, et al. On the complex scheduling formulation of virtual network functions over optical networks[C]//Proc of the 16th Int Conf on Transparent Optical Networks (ICTON). Piscataway, NJ: IEEE, 2014: 1−5, articleTitle=null, refAbstract=null), Reference(id=1325493721051038690, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=31, rfOrder=34, authorNames=null, journalName=null, refType=null, unstructuredReference=Qu Long, Assi C, Shaban K. Delay-aware scheduling and resource optimization with network function virtualization[J]. IEEE Transactions on Communications, 2016, 64(9): 3746−3758, articleTitle=null, refAbstract=null), Reference(id=1325493721306891235, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=32, rfOrder=35, authorNames=null, journalName=null, refType=null, unstructuredReference=阳勇, 孟相如, 康巧燕, 等. 基于资源需求预测的动态服务功能链迁移方法[J]. 计算机研究与发展, 2023, 60(5): 1151−1163, articleTitle=null, refAbstract=null), Reference(id=1325493721646629860, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=32, rfOrder=36, authorNames=null, journalName=null, refType=null, unstructuredReference=Yang Yong, Meng Xiangru, Kang Qiaoyan, et al. Dynamic service function chain migration method based on resource requirements prediction[J]. Journal of Computer Research and Development, 2023, 60(5): 1151−1163 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1325493721747293157, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=33, rfOrder=37, authorNames=null, journalName=null, refType=null, unstructuredReference=Eramo V, Miucci E, Ammar M, et al. An approach for service function chain routing and virtual function network instance migration in network function virtualization architectures[J]. IEEE/ACM Transactions on Networking, 2017, 25(4): 2008−2025, articleTitle=null, refAbstract=null), Reference(id=1325493721814402022, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=34, rfOrder=38, authorNames=null, journalName=null, refType=null, unstructuredReference=Liu Junjie, Lu Wei, Zhou Fen, et al. On dynamic service function chain deployment and readjustment[J]. IEEE Transactions on Network and Service Management, 2017, 14(3): 543−553, articleTitle=null, refAbstract=null), Reference(id=1325493721894093799, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=35, rfOrder=39, authorNames=null, journalName=null, refType=null, unstructuredReference=Bellman R. A Markovian decision process[J]. Journal of Mathematics and Mechanics, 1957, 6(5): 679−684, articleTitle=null, refAbstract=null), Reference(id=1325493722011534312, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=36, rfOrder=40, authorNames=null, journalName=null, refType=null, unstructuredReference=Mnih V, Kavukcuoglu K, Silver D, et al. Human-level control through deep reinforcement learning[J]. Nature, 2015, 518(7540): 529−533, articleTitle=null, refAbstract=null), Reference(id=1325493722082837481, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=37, rfOrder=41, authorNames=null, journalName=null, refType=null, unstructuredReference=Silver D, Huang A, Maddison C J, et al. Mastering the game of Go with deep neural networks and tree search[J]. Nature, 2016, 529(7587): 484−489, articleTitle=null, refAbstract=null), Reference(id=1325493722149946346, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=38, rfOrder=42, authorNames=null, journalName=null, refType=null, unstructuredReference=刘全, 翟建伟, 章宗长, 等. 深度强化学习综述[J]. 计算机学报, 2018, 41(1): 1−27, articleTitle=null, refAbstract=null), Reference(id=1325493722342884331, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=38, rfOrder=43, authorNames=null, journalName=null, refType=null, unstructuredReference=Liu Quan, Zhai Jianwei, Zhang Zongzhang, et al. A survey on deep reinforcement learning[J]. Chinese Journal of Computers, 2018, 41(1): 1−27 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1325493722510656492, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=39, rfOrder=44, authorNames=null, journalName=null, refType=null, unstructuredReference=丁世飞, 杜威, 张健, 等. 多智能体深度强化学习研究进展[J]. 计算机学报, 2024, 47(7): 1547−1567, articleTitle=null, refAbstract=null), Reference(id=1325493722602931181, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=39, rfOrder=45, authorNames=null, journalName=null, refType=null, unstructuredReference=Ding Shifei, Du Wei, Zhang Jian, et al. Research progress of multi-agent deep reinforcement learning[J]. Chinese Journal of Computers, 2024, 47(7): 1547−1567 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1325493722670040046, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=40, rfOrder=46, authorNames=null, journalName=null, refType=null, unstructuredReference=Kim S I, Kim H S. A research on dynamic service function chaining based on reinforcement learning using resource usage[C]//Proc of the 9th Int Conf on Ubiquitous and Future Networks (ICUFN). Piscataway, NJ: IEEE, 2017: 582−586, articleTitle=null, refAbstract=null), Reference(id=1325493722745537519, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=41, rfOrder=47, authorNames=null, journalName=null, refType=null, unstructuredReference=Sun Jian, Huang Guanhua, Sun Gang, et al. A Q-learning-based approach for deploying dynamic service function chains[J]. Symmetry, 2018, 10(11): 646, articleTitle=null, refAbstract=null), Reference(id=1325493722812646384, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=42, rfOrder=48, authorNames=null, journalName=null, refType=null, unstructuredReference=Watkins C J C H, Dayan P. Q-learning[J]. Machine Learning, 1992, 8(3): 279−292, articleTitle=null, refAbstract=null), Reference(id=1325493722879755249, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=43, rfOrder=49, authorNames=null, journalName=null, refType=null, unstructuredReference=Chen Jing, Chen Jia, Zhang Hongke. DRL-QOR: Deep reinforcement learning-based QoS/QoE-aware adaptive online orchestration in NFV-enabled networks[J]. IEEE Transactions on Network and Service Management, 2021, 18(2): 1758−1774, articleTitle=null, refAbstract=null), Reference(id=1325493722951058418, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=44, rfOrder=50, authorNames=null, journalName=null, refType=null, unstructuredReference=Nouruzi A, Zakeri A, Javan M R, et al. Online service provisioning in NFV-enabled networks using deep reinforcement learning[J]. IEEE Transactions on Network and Service Management, 2022, 19(3): 3276−3289, articleTitle=null, refAbstract=null), Reference(id=1325493723026555891, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=45, rfOrder=51, authorNames=null, journalName=null, refType=null, unstructuredReference=Li Guanglei, Feng Bohao, Zhou Huachun, et al. Adaptive service function chaining mappings in 5G using deep Q-learning[J]. Computer Communications, 2020, 152: 305−315, articleTitle=null, refAbstract=null), Reference(id=1325493723110441972, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=46, rfOrder=52, authorNames=null, journalName=null, refType=null, unstructuredReference=刘光远, 陈世莹, 庞紫园. 边缘云环境中收益最大化的在线服务功能链部署机制[J]. 清华大学学报: 自然科学版, 2025, 65(8): 1516−1529, articleTitle=null, refAbstract=null), Reference(id=1325493723185939445, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=46, rfOrder=53, authorNames=null, journalName=null, refType=null, unstructuredReference=Liu Guangyuan, Chen Shiying, Pang Ziyuan. Deployment mechanism for maximizing revenue from online service function chains in edge cloud environments[J]. Journal of Tsinghua University: Science and Technology, 2025, 65(8): 1516−1529, articleTitle=null, refAbstract=null), Reference(id=1325493724893021174, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=47, rfOrder=54, authorNames=null, journalName=null, refType=null, unstructuredReference=Quang P T A, Hadjadj-Aoul Y, Outtagarts A. A deep reinforcement learning approach for VNF forwarding graph embedding[J]. IEEE Transactions on Network and Service Management, 2019, 16(4): 1318−1331, articleTitle=null, refAbstract=null), Reference(id=1325493724960130039, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=48, rfOrder=55, authorNames=null, journalName=null, refType=null, unstructuredReference=唐伦, 贺兰钦, 连沁怡, 等. 基于改进深度强化学习的虚拟网络功能部署优化算法[J]. 电子与信息学报, 2021, 43(6): 1724−1732, articleTitle=null, refAbstract=null), Reference(id=1325493725073376248, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=48, rfOrder=56, authorNames=null, journalName=null, refType=null, unstructuredReference=Tang Lun, He Lanqin, Lian Qinyi, et al. Virtual network function placement optimization algorithm based on improve deep reinforcement learning[J]. Journal of Electronics & Information Technology, 2021, 43(6): 1724−1732 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1325493725153068025, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=49, rfOrder=57, authorNames=null, journalName=null, refType=null, unstructuredReference=He Nan, Yang Song, Li Fan, et al. A-DDPG: Attention mechanism-based deep reinforcement learning for NFV[C]//Proc of the 29th IEEE/ACM Int Symp on Quality of Service (IWQOS). Piscataway, NJ: IEEE, 2021: 1−10, articleTitle=null, refAbstract=null), Reference(id=1325493725236954106, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=50, rfOrder=58, authorNames=null, journalName=null, refType=null, unstructuredReference=He Nan, Yang Song, Li Fan, et al. Leveraging deep reinforcement learning with attention mechanism for virtual network function placement and routing[J]. IEEE Transactions on Parallel and Distributed Systems, 2023, 34(4): 1186−1201, articleTitle=null, refAbstract=null), Reference(id=1325493725346006011, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=51, rfOrder=59, authorNames=null, journalName=null, refType=null, unstructuredReference=Liu Yicen, Zhang Junning. Service function chain embedding meets machine learning: Deep reinforcement learning approach[J]. IEEE Transactions on Network and Service Management, 2024, 21(3): 3465−3481, articleTitle=null, refAbstract=null), Reference(id=1325493725413114876, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=52, rfOrder=60, authorNames=null, journalName=null, refType=null, unstructuredReference=Huang Zhongwei, Zhong Weijian, Li Dagang, et al. Delay constrained SFC orchestration for edge intelligence-enabled IIoT: A DRL approach[J]. Journal of Network and Systems Management, 2023, 31(3): 53, articleTitle=null, refAbstract=null), Reference(id=1325493725513778173, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=53, rfOrder=61, authorNames=null, journalName=null, refType=null, unstructuredReference=Wang Lei, Mao Weixi, Zhao Jin, et al. DDQP: A double deep Q-learning approach to online fault-tolerant SFC placement[J]. IEEE Transactions on Network and Service Management, 2021, 18(1): 118−132, articleTitle=null, refAbstract=null), Reference(id=1325493725610247166, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=54, rfOrder=62, authorNames=null, journalName=null, refType=null, unstructuredReference=Qu Hua, Wang Ke, Zhao Jihong. Reliable service function chain deployment method based on deep reinforcement learning[J]. Sensors, 2021, 21(8): 2733, articleTitle=null, refAbstract=null), Reference(id=1325493725694133247, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=55, rfOrder=63, authorNames=null, journalName=null, refType=null, unstructuredReference=Santos G L, Lynn T, Kelner J, et al. Availability-aware and energy-aware dynamic SFC placement using reinforcement learning[J]. The Journal of Supercomputing, 2021, 77(11): 12711−12740, articleTitle=null, refAbstract=null), Reference(id=1325493725798990848, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=56, rfOrder=64, authorNames=null, journalName=null, refType=null, unstructuredReference=Santos G L, Endo P T, Lynn T, et al. A reinforcement learning-based approach for availability-aware service function chain placement in large-scale networks[J]. Future Generation Computer Systems, 2022, 136: 93−109, articleTitle=null, refAbstract=null), Reference(id=1325493725874487296, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=57, rfOrder=65, authorNames=null, journalName=null, refType=null, unstructuredReference=Marler R T, Arora J S. The weighted sum method for multi-objective optimization: new insights[J]. Structural and Multidisciplinary Optimization, 2010, 41(6): 853−862, articleTitle=null, refAbstract=null), Reference(id=1325493725987733505, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=58, rfOrder=66, authorNames=null, journalName=null, refType=null, unstructuredReference=Konak A, Coit D W, Smith A E. Multi-objective optimization using genetic algorithms: A tutorial[J]. Reliability Engineering & System Safety, 2006, 91(9): 992−1007, articleTitle=null, refAbstract=null), Reference(id=1325493726080008194, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=59, rfOrder=67, authorNames=null, journalName=null, refType=null, unstructuredReference=Bi Yu, Meixner C C, Bunyakitanon M, et al. Multi-objective deep reinforcement learning assisted service function chains placement[J]. IEEE Transactions on Network and Service Management, 2021, 18(4): 4134−4150, articleTitle=null, refAbstract=null), Reference(id=1325493726201643011, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=60, rfOrder=68, authorNames=null, journalName=null, refType=null, unstructuredReference=Li Huaping, Kordi M E. DSPPV: Dynamic service function chains placement with parallelized virtual network functions in mobile edge computing[J]. Internet of Things, 2023, 22: 100733, articleTitle=null, refAbstract=null), Reference(id=1325493726277140484, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=61, rfOrder=69, authorNames=null, journalName=null, refType=null, unstructuredReference=Jia Jing, Hua Jing. Dynamic SFC placement with parallelized VNFs in data center networks: A DRL-based approach[J]. ICT Express, 2024, 10(1): 104−110, articleTitle=null, refAbstract=null), Reference(id=1325493726486855685, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=62, rfOrder=70, authorNames=null, journalName=null, refType=null, unstructuredReference=Sun Penghao, Lan Julong, Li Junfei, et al. Combining deep reinforcement learning with graph neural networks for optimal VNF placement[J]. IEEE Communications Letters, 2020, 25(1): 176−180, articleTitle=null, refAbstract=null), Reference(id=1325493726562353158, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=63, rfOrder=71, authorNames=null, journalName=null, refType=null, unstructuredReference=Battaglia P W, Hamrick J B, Bapst V, et al. Relational inductive biases, deep learning, and graph networks[J]. arXiv preprint, arXiv: 1806.01261, 2018, articleTitle=null, refAbstract=null), Reference(id=1325493726642044935, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=64, rfOrder=72, authorNames=null, journalName=null, refType=null, unstructuredReference=Xie Yanghao, Huang Lin, Kong Yuyang, et al. Virtualized network function forwarding graph placing in SDN and NFV-enabled IoT networks: A graph neural network assisted deep reinforcement learning method[J]. IEEE Transactions on Network and Service Management, 2021, 19(1): 524−537, articleTitle=null, refAbstract=null), Reference(id=1325493726738513928, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=65, rfOrder=73, authorNames=null, journalName=null, refType=null, unstructuredReference=Qiu Rixuan, Bao Jiawen, Li Yuancheng, et al. Virtual network function deployment algorithm based on graph convolution deep reinforcement learning[J]. The Journal of Supercomputing, 2023, 79(6): 6849−6870, articleTitle=null, refAbstract=null), Reference(id=1325493726818205705, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=66, rfOrder=74, authorNames=null, journalName=null, refType=null, unstructuredReference=Fan Qilin, Pan Pan, Li Xiuhua, et al. DRL-D: Revenue-aware online service function chain deployment via deep reinforcement learning[J]. IEEE Transactions on Network and Service Management, 2022, 19(4): 4531−4545, articleTitle=null, refAbstract=null), Reference(id=1325493726881120266, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=67, rfOrder=75, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhu Yuchao, Yao Haipeng, Mai Tianle, et al. Multiagent reinforcement-learning-aided service function chain deployment for Internet of things[J]. IEEE Internet of Things Journal, 2022, 9(17): 15674−15684, articleTitle=null, refAbstract=null), Reference(id=1325493727061475339, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=68, rfOrder=76, authorNames=null, journalName=null, refType=null, unstructuredReference=Xiao Da, Zhang J A, Liu Xin, et al. A two-stage GCN-based deep reinforcement learning framework for SFC embedding in multi-datacenter networks[J]. IEEE Transactions on Network and Service Management, 2023, 20(4): 4297−4312, articleTitle=null, refAbstract=null), Reference(id=1325493727174721548, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=69, rfOrder=77, authorNames=null, journalName=null, refType=null, unstructuredReference=Wang Shaoyang, Yuen C, Ni Wei, et al. Multiagent deep reinforcement learning for cost-and delay-sensitive virtual network function placement and routing[J]. IEEE Transactions on Communications, 2022, 70(8): 5208−5224, articleTitle=null, refAbstract=null), Reference(id=1325493727338299405, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=70, rfOrder=78, authorNames=null, journalName=null, refType=null, unstructuredReference=Wang Kan, Yuan Peng, Jan M A, et al. Digital twin-assisted service function chaining in multi-domain computing power networks with multi-agent reinforcement learning[J]. Future Generation Computer Systems, 2024, 158: 294−307, articleTitle=null, refAbstract=null), Reference(id=1325493727426379790, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=71, rfOrder=79, authorNames=null, journalName=null, refType=null, unstructuredReference=Wang Shuyi, Cao Haotong, Yang Longxiang, et al. GCN-based multi-agent deep reinforcement learning for dynamic service function chain deployment in IoT[J]. IEEE Transactions on Consumer Electronics, 2024, 70(3): 6105−6118, articleTitle=null, refAbstract=null), Reference(id=1325493727518654479, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=72, rfOrder=80, authorNames=null, journalName=null, refType=null, unstructuredReference=Wang Tianfeng, Zu Jiachen, Hu Guyu, et al. Adaptive service function chain scheduling in mobile edge computing via deep reinforcement learning[J]. IEEE Access, 2020, 8: 164922−164935, articleTitle=null, refAbstract=null), Reference(id=1325493727581569040, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=73, rfOrder=81, authorNames=null, journalName=null, refType=null, unstructuredReference=He Rui, Ren Bangbang, Xie Junjie, et al. A reinforcement learning method for scheduling service function chains with multi-resource constraints[J]. Computer Networks, 2023, 235: 109985, articleTitle=null, refAbstract=null), Reference(id=1325493727657066513, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=74, rfOrder=82, authorNames=null, journalName=null, refType=null, unstructuredReference=Yao Jiamin, Wang Junli, Wang Cheng, et al. DRL-based VNF cooperative scheduling framework with priority-weighted delay[J]. IEEE Transactions on Mobile Computing, 2024, 23(12): 11375−11388, articleTitle=null, refAbstract=null), Reference(id=1325493729318010898, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=75, rfOrder=83, authorNames=null, journalName=null, refType=null, unstructuredReference=Jia Ziye, Cao Yilu, He Lijun, et al. Service function chain dynamic scheduling in space-air-ground integrated networks[J]. IEEE Transactions on Vehicular Technology, 2025, 74(7): 11235−11248, articleTitle=null, refAbstract=null), Reference(id=1325493729473200147, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=76, rfOrder=84, authorNames=null, journalName=null, refType=null, unstructuredReference=Li Junling, Shi Weisen, Zhang Ning, et al. Delay-aware VNF scheduling: A reinforcement learning approach with variable action set[J]. IEEE Transactions on Cognitive Communications and Networking, 2020, 7(1): 304−318, articleTitle=null, refAbstract=null), Reference(id=1325493729829715988, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=77, rfOrder=85, authorNames=null, journalName=null, refType=null, unstructuredReference=Akbari M, Abedi M R, Joda R, et al. Age of information aware VNF scheduling in industrial IoT using deep reinforcement learning[J]. IEEE Journal on Selected Areas in Communications, 2021, 39(8): 2487−2500, articleTitle=null, refAbstract=null), Reference(id=1325493729921990677, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=78, rfOrder=86, authorNames=null, journalName=null, refType=null, unstructuredReference=Kuai Zhenran, Wang Tianyu, Wang Shaowei. Fair virtual network function mapping and scheduling using proximal policy optimization[J]. IEEE Transactions on Communications, 2022, 70(11): 7434−7445, articleTitle=null, refAbstract=null), Reference(id=1325493730110734358, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=79, rfOrder=87, authorNames=null, journalName=null, refType=null, unstructuredReference=张庆华, 张先超, 王寅昊, 等. 面向医疗急救的信息网络服务功能链调度方法[J]. 电子学报, 2023, 51(11): 3128−3136, articleTitle=null, refAbstract=null), Reference(id=1325493730198814743, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=79, rfOrder=88, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhang Qinghua, Zhang Xianchao, Wang Yinhao, et al. Service function chains scheduling of information networks used for emergency treatment[J]. Acta Electronica Sinica, 2023, 51(11): 3128−3136 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1325493730282700824, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=80, rfOrder=89, authorNames=null, journalName=null, refType=null, unstructuredReference=Lu Yuxi, Zhang Peiying, Duan Youxiang, et al. Dynamic scheduling of iov edge cloud service functions under NFV: A multi-agent reinforcement learning approach[J]. IEEE Transactions on Vehicular Technology, 2023, 73(4): 5730−5741, articleTitle=null, refAbstract=null), Reference(id=1325493730584690713, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=81, rfOrder=90, authorNames=null, journalName=null, refType=null, unstructuredReference=Liu Zhiwei, Shu Zhaogang, Chen Shuwu, et al. Low-latency virtual network function scheduling algorithm based on deep reinforcement learning[J]. Computer Networks, 2024, 246: 110418, articleTitle=null, refAbstract=null), Reference(id=1325493730660188186, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=82, rfOrder=91, authorNames=null, journalName=null, refType=null, unstructuredReference=唐伦, 周钰, 谭颀, 等. 基于强化学习的5G网络切片虚拟网络功能迁移算法[J]. 电子与信息学报, 2020, 42(3): 669−677, articleTitle=null, refAbstract=null), Reference(id=1325493730744074267, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=82, rfOrder=92, authorNames=null, journalName=null, refType=null, unstructuredReference=Tang Lun, Zhou Yu, Tan Qi, et al. Virtual network function migration algorithm based on reinforcement learning for 5G network slicing[J]. Journal of Electronics & Information Technology, 2020, 42(3): 669−677 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1325493730836348956, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=83, rfOrder=93, authorNames=null, journalName=null, refType=null, unstructuredReference=Wang Yuantao, Shu Zhaogang, Chen Shuwu, et al. A cost and demand sensitive adjustment algorithm for service function chain in data center network[J]. Computer Networks, 2024, 242: 110254, articleTitle=null, refAbstract=null), Reference(id=1325493730916040733, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=84, rfOrder=94, authorNames=null, journalName=null, refType=null, unstructuredReference=Pei Jianing, Hong Peilin, Pan Miao, et al. Optimal VNF placement via deep reinforcement learning in SDN/NFV-enabled networks[J]. IEEE Journal on Selected Areas in Communications, 2019, 38(2): 263−278, articleTitle=null, refAbstract=null), Reference(id=1325493730999926814, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=85, rfOrder=95, authorNames=null, journalName=null, refType=null, unstructuredReference=Ran Jing, Wang Wenkai, Hu Hefei. Dynamic service function chain deployment and readjustment method based on deep reinforcement learning[J]. Sensors, 2023, 23(6): 3054, articleTitle=null, refAbstract=null), Reference(id=1325493731071229983, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=86, rfOrder=96, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhang Peiying, Li Yilin, Tan Lizhuang, et al. Latency-sensitive service function chains intelligent migration in satellite communication driven by deep reinforcement learning[J]. Transactions on Emerging Telecommunications Technologies, 2024, 35(11): e70006, articleTitle=null, refAbstract=null), Reference(id=1325493731134144544, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=87, rfOrder=97, authorNames=null, journalName=null, refType=null, unstructuredReference=Hu Hefei, Zhang Wei, Xu Lingyi, et al. A mobility-aware service function chain migration strategy based on deep reinforcement learning[J]. Journal of Network and Systems Management, 2023, 31(1): 21, articleTitle=null, refAbstract=null), Reference(id=1325493731201253409, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=88, rfOrder=98, authorNames=null, journalName=null, refType=null, unstructuredReference=Xu Lingyi, Liu Wenbin, Wang Zhiwei, et al. Mobile-aware service function chain intelligent seamless migration in multi-access edge computing[J]. Journal of Network and Systems Management, 2024, 32(3): 49, articleTitle=null, refAbstract=null), Reference(id=1325493731268362274, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=89, rfOrder=99, authorNames=null, journalName=null, refType=null, unstructuredReference=Liu Yicen, Lu Yu, Li Xi, et al. On dynamic service function chain reconfiguration in IoT networks[J]. IEEE Internet of Things Journal, 2020, 7(11): 10969−10984, articleTitle=null, refAbstract=null), Reference(id=1325493731352248355, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=90, rfOrder=100, authorNames=null, journalName=null, refType=null, unstructuredReference=Cai Jun, Qian Kaili, Luo Jianzhen, et al. SARM: Service function chain active reconfiguration mechanism based on load and demand prediction[J]. International Journal of Intelligent Systems, 2022, 37(9): 6388−6414, articleTitle=null, refAbstract=null), Reference(id=1325493731436134436, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=91, rfOrder=101, authorNames=null, journalName=null, refType=null, unstructuredReference=Qu Hua, Wang Ke, Zhao Jihong. Priority-awareness VNF migration method based on deep reinforcement learning[J]. Computer Networks, 2022, 208: 108866, articleTitle=null, refAbstract=null), Reference(id=1325493731532603429, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=92, rfOrder=102, authorNames=null, journalName=null, refType=null, unstructuredReference=Yu Xue, Wang Ran, Hao Jie, et al. Priority-aware deployment of autoscaling service function chains based on deep reinforcement learning[J]. IEEE Transactions on Cognitive Communications and Networking, 2024, 10(3): 1050−1062, articleTitle=null, refAbstract=null), Reference(id=1325493731608100902, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=93, rfOrder=103, authorNames=null, journalName=null, refType=null, unstructuredReference=Chen Jing, Chen Jia, Guo Kuo, et al. Fault tolerance oriented SFC optimization in SDN/NFV-enabled cloud environment based on deep reinforcement learning[J]. IEEE Transactions on Cloud Computing, 2024, 12(1): 200−218, articleTitle=null, refAbstract=null), Reference(id=1325493731683598375, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=94, rfOrder=104, authorNames=null, journalName=null, refType=null, unstructuredReference=Li Yilin, Zhang Peiying, Kumar N, et al. Reliability-assured service function chain migration strategy in edge networks using deep reinforcement learning[J]. Journal of Network and Computer Applications, 2024, 231: 103999, articleTitle=null, refAbstract=null)], funds=null, companyList=[AuthorCompany(id=1325493684887749498, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, xref=1, ext=[AuthorCompanyExt(id=1325493685646918523, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, companyId=1325493684887749498, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1College of Computer Science, Inner Mongolia University, Hohhot 010021), AuthorCompanyExt(id=1325493685734998908, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, companyId=1325493684887749498, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1内蒙古大学计算机学院 呼和浩特 010021)]), AuthorCompany(id=1325493686091514749, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, xref=2, ext=[AuthorCompanyExt(id=1325493686494167934, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, companyId=1325493686091514749, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2Engineering Research Center of Ecological Big Data (Inner Mongolia University), Ministry of Education, Hohhot 010021), AuthorCompanyExt(id=1325493686557082495, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, companyId=1325493686091514749, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2生态大数据教育部工程研究中心(内蒙古大学) 呼和浩特 010021)]), AuthorCompany(id=1325493687366583168, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, xref=3, ext=[AuthorCompanyExt(id=1325493687404331905, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, companyId=1325493687366583168, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=3Inner Mongolia Engineering Laboratory for Cloud Computing and Service Software (Inner Mongolia University), Hohhot 010021), AuthorCompanyExt(id=1325493687416914818, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, companyId=1325493687366583168, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=3内蒙古自治区云计算与服务软件工程实验室(内蒙古大学) 呼和浩特 010021)])], figs=[ArticleFig(id=1325493707520213929, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, language=EN, label=Figure 1, caption=Framework of NFV architecture, figureFileSmall=zZnBPhE6ec6d20kcj7gilQ==, figureFileBig=OT331tEXFjwTuusXEoYSXQ==, tableContent=null), ArticleFig(id=1325493707897701290, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, language=CN, label=图1, caption=NFV架构框架, figureFileSmall=zZnBPhE6ec6d20kcj7gilQ==, figureFileBig=OT331tEXFjwTuusXEoYSXQ==, tableContent=null), ArticleFig(id=1325493708447155115, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, language=EN, label=Figure 2, caption=SFC architecture in NFV/SDN architecture, figureFileSmall=Es/dM2Zq4h1CF2N3fserAQ==, figureFileBig=jCyloZsmVLhT6x/utKT9pA==, tableContent=null), ArticleFig(id=1325493708803670957, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, language=CN, label=图2, caption=NFV/SDN架构下SFC体系结构, figureFileSmall=Es/dM2Zq4h1CF2N3fserAQ==, figureFileBig=jCyloZsmVLhT6x/utKT9pA==, tableContent=null), ArticleFig(id=1325493708895945646, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, language=EN, label=Figure 3, caption=Deployment workflow of SFC, figureFileSmall=A34oHszd+4gPjnVYe7xBYQ==, figureFileBig=Eh5gFEs4c/AzoEW9ltFrAw==, tableContent=null), ArticleFig(id=1325493709277627311, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, language=CN, label=图3, caption=SFC部署流程, figureFileSmall=A34oHszd+4gPjnVYe7xBYQ==, figureFileBig=Eh5gFEs4c/AzoEW9ltFrAw==, tableContent=null), ArticleFig(id=1325493709567034288, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, language=EN, label=Figure 4, caption=MDP modeling of SFC deployment process, figureFileSmall=a8IAeadZU4zkcXGCGdxEIw==, figureFileBig=Zrj+FkRapYN9RgIYUAOjWQ==, tableContent=null), ArticleFig(id=1325493709642531761, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, language=CN, label=图4, caption=SFC部署过程的MDP建模, figureFileSmall=a8IAeadZU4zkcXGCGdxEIw==, figureFileBig=Zrj+FkRapYN9RgIYUAOjWQ==, tableContent=null), ArticleFig(id=1325493709965493170, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, language=EN, label=Figure 5, caption=SFC deployment framework based on reinforcement learning, figureFileSmall=BOE64XDDXYMJjY7RX/2+rQ==, figureFileBig=7rZvG42gNVlmR6qeTrsUFQ==, tableContent=null), ArticleFig(id=1325493711492219827, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, language=CN, label=图5, caption=基于强化学习的SFC部署框架, figureFileSmall=BOE64XDDXYMJjY7RX/2+rQ==, figureFileBig=7rZvG42gNVlmR6qeTrsUFQ==, tableContent=null), ArticleFig(id=1325493711571911604, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, language=EN, label=Table 1, caption=Comparison of Our Paper and Other Service Function Chaining Relative Surveys, figureFileSmall=null, figureFileBig=null, tableContent=
相关综述年份综述范围基于强化学习的SFC分析情况
放置调度重配置分析深度
注:“√”表示详细分析;“△”表示简要提及;“×”表示未涉及。“无”表示未涉及;“浅”表示仅做简单介绍;“中”表示有专门章节但不系统;“深”表示系统全面分析。
文献[6]2016SFC定义、架构、用例、优化策略分类×××无
文献[13]2018NFV架构、NFV环境下SFC部署方法×××无
文献[14]2020NFV/SDN架构下的SFC定义、架构、优化策略分类△××无
文献[15]2020NFV架构下资源分配、VNF-FG嵌入问题(VNF放置、流量路由)△××无
文献[16]2021NFV架构下资源管理(组链、部署、调度和按需缩放)×△×无
文献[17]2021NFV资源分配问题定义及建模、VNF-FG嵌入策略分类△××浅
文献[18]2022分布式场景下SFC放置、优化策略分类△××浅
文献[19]2023SFC用例、安全机制(架构安全、解决方案、安全管理)×××无
文献[12]2024B5G/6G网络中VNF-FG嵌入、优化方法分类△△×中
本文2024基于强化学习的SFC部署架构以及(放置、调度、重配置)√√√深
), ArticleFig(id=1325493711718712245, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, language=CN, label=表1, caption=

本文与其他服务功能链相关综述的对比

, figureFileSmall=null, figureFileBig=null, tableContent=
相关综述年份综述范围基于强化学习的SFC分析情况
放置调度重配置分析深度
注:“√”表示详细分析;“△”表示简要提及;“×”表示未涉及。“无”表示未涉及;“浅”表示仅做简单介绍;“中”表示有专门章节但不系统;“深”表示系统全面分析。
文献[6]2016SFC定义、架构、用例、优化策略分类×××无
文献[13]2018NFV架构、NFV环境下SFC部署方法×××无
文献[14]2020NFV/SDN架构下的SFC定义、架构、优化策略分类△××无
文献[15]2020NFV架构下资源分配、VNF-FG嵌入问题(VNF放置、流量路由)△××无
文献[16]2021NFV架构下资源管理(组链、部署、调度和按需缩放)×△×无
文献[17]2021NFV资源分配问题定义及建模、VNF-FG嵌入策略分类△××浅
文献[18]2022分布式场景下SFC放置、优化策略分类△××浅
文献[19]2023SFC用例、安全机制(架构安全、解决方案、安全管理)×××无
文献[12]2024B5G/6G网络中VNF-FG嵌入、优化方法分类△△×中
本文2024基于强化学习的SFC部署架构以及(放置、调度、重配置)√√√深
), ArticleFig(id=1325493711840347062, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, language=EN, label=Table 2, caption=Comparison of SFC Optimization Methods, figureFileSmall=null, figureFileBig=null, tableContent=
方法计算复杂度求解质量适应性适用场景
ILP/MILP高(指数级时间复杂度)精确解低(环境变化需重新建模求解)小规模静态网络
启发式算法中等(多项式时间复杂度)近似最优解中等(依赖预设规则,难以自适应调整)中等规模静态网络
强化学习训练阶段高,决策阶段低随经验积累逐渐接近最优高(能通过交互学习适应环境变化)动态大规模网络
), ArticleFig(id=1325493712079422391, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, language=CN, label=表2, caption=

SFC优化方法对比

, figureFileSmall=null, figureFileBig=null, tableContent=
方法计算复杂度求解质量适应性适用场景
ILP/MILP高(指数级时间复杂度)精确解低(环境变化需重新建模求解)小规模静态网络
启发式算法中等(多项式时间复杂度)近似最优解中等(依赖预设规则,难以自适应调整)中等规模静态网络
强化学习训练阶段高,决策阶段低随经验积累逐渐接近最优高(能通过交互学习适应环境变化)动态大规模网络
), ArticleFig(id=1325493712180085688, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, language=EN, label=Table 3, caption=Comparison of Reinforcement Learning Algorithms, figureFileSmall=null, figureFileBig=null, tableContent=
类别算法特点及优势劣势
基于值的
方法
Q-Learning通过更新Q值表学习最优策略。算法实现简单,适合离散动作空间。对于高维状态/动作空间表现较差,
需要探索策略。
DQN使用神经网络逼近Q值函数,适合高维状态空间,
结合经验回放缓解样本相关性问题。
训练时间较长,难以处理连续动作空间,
容易出现过估计问题。
Double DQN改进DQN,通过双网络缓解Q值过估计问题。
算法更稳定,Q值估计更准确,收敛效果好。
实现复杂度比DQN略高。
TD (temporal
difference) Learning
基于时序差分的学习方法,结合蒙特卡洛与
动态规划优点,适用于在线学习。
对初始值敏感,需要手动设计探索策略。
基于策略
的方法
REINFORCE基本的策略梯度算法,实现简单,可处理连续动作空间高方差、样本效率低、收敛速度慢。
PG基于策略直接优化目标函数,使用梯度更新策略;
简单易实现,可直接处理连续动作空间。
高方差、收敛速度慢、可能陷入局部最优。
TRPO信任域策略优化,保证策略更新单调改进,稳定性高,
适合大规模复杂任务。
实现复杂、计算开销大、实现难度大。
基于值和
策略的方法
AC结合策略梯度和值函数,减少方差,提高收敛速度;
适合连续动作空间问题。
Actor和Critic网络需要同时训练,
设计和调参复杂。
A2C同步版本的AC算法,使用优势函数A代替Q值或V值。性能不如异步版本,样本利用效率低。
A3C异步版本的AC,使用多线程并行更新全局网络。实现复杂,异步更新可能导致不稳定性。
DDPG基于Actor-Critic的离线算法,高效处理连续
动作空间任务,样本效率高。
对超参数敏感,探索能力不足,可能不稳定。
PPO改进TRPO,使用简单但有效的剪辑机制来限制策略更新幅度;
实现简单、稳定性好、收敛速度快;适合大规模任务。
超参数较多,可能需要较多调试。
SAC基于最大熵的强化学习,平衡探索和利用,目标是最大化
策略期望回报与熵值,样本效率高,稳定性好。
计算开销较大,超参数较多,调参复杂。
多智能体
方法
MADDPGDDPG的多智能体扩展,针对每个智能体
使用独立Actor和共享Critic。
对环境建模要求高,协作机制复杂,
可能不稳定。
MAPPOPPO算法的多智能体扩展,适用于多智能体协
作或竞争环境;可扩展性好、训练稳定。
计算开销较大,通信和协作机制设计复杂。
), ArticleFig(id=1325493712540795833, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, language=CN, label=表3, caption=

强化学习算法对比

, figureFileSmall=null, figureFileBig=null, tableContent=
类别算法特点及优势劣势
基于值的
方法
Q-Learning通过更新Q值表学习最优策略。算法实现简单,适合离散动作空间。对于高维状态/动作空间表现较差,
需要探索策略。
DQN使用神经网络逼近Q值函数,适合高维状态空间,
结合经验回放缓解样本相关性问题。
训练时间较长,难以处理连续动作空间,
容易出现过估计问题。
Double DQN改进DQN,通过双网络缓解Q值过估计问题。
算法更稳定,Q值估计更准确,收敛效果好。
实现复杂度比DQN略高。
TD (temporal
difference) Learning
基于时序差分的学习方法,结合蒙特卡洛与
动态规划优点,适用于在线学习。
对初始值敏感,需要手动设计探索策略。
基于策略
的方法
REINFORCE基本的策略梯度算法,实现简单,可处理连续动作空间高方差、样本效率低、收敛速度慢。
PG基于策略直接优化目标函数,使用梯度更新策略;
简单易实现,可直接处理连续动作空间。
高方差、收敛速度慢、可能陷入局部最优。
TRPO信任域策略优化,保证策略更新单调改进,稳定性高,
适合大规模复杂任务。
实现复杂、计算开销大、实现难度大。
基于值和
策略的方法
AC结合策略梯度和值函数,减少方差,提高收敛速度;
适合连续动作空间问题。
Actor和Critic网络需要同时训练,
设计和调参复杂。
A2C同步版本的AC算法,使用优势函数A代替Q值或V值。性能不如异步版本,样本利用效率低。
A3C异步版本的AC,使用多线程并行更新全局网络。实现复杂,异步更新可能导致不稳定性。
DDPG基于Actor-Critic的离线算法,高效处理连续
动作空间任务,样本效率高。
对超参数敏感,探索能力不足,可能不稳定。
PPO改进TRPO,使用简单但有效的剪辑机制来限制策略更新幅度;
实现简单、稳定性好、收敛速度快;适合大规模任务。
超参数较多,可能需要较多调试。
SAC基于最大熵的强化学习,平衡探索和利用,目标是最大化
策略期望回报与熵值,样本效率高,稳定性好。
计算开销较大,超参数较多,调参复杂。
多智能体
方法
MADDPGDDPG的多智能体扩展,针对每个智能体
使用独立Actor和共享Critic。
对环境建模要求高,协作机制复杂,
可能不稳定。
MAPPOPPO算法的多智能体扩展,适用于多智能体协
作或竞争环境;可扩展性好、训练稳定。
计算开销较大,通信和协作机制设计复杂。
), ArticleFig(id=1325493712645653434, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, language=EN, label=Table 4, caption=Comparison of SFC Placement Algorithms Based on Reinforcement Learning, figureFileSmall=null, figureFileBig=null, tableContent=
分类放置算法强化学习算法优化目标VNF放置策略虚拟链路放置策略
基础DRL
算法
NFVdeep[7]PG运行成本、吞吐量节点序号
DRL-QOR[43]DQNQoE节点序号
DQN-AR[44]DQN运行成本、带宽资源节点序号链路序号
ADAP[45]DQN带宽消耗、CPU消耗2种基于贪婪的候选算法重赋链路权重
SDRM-EC-DPR[46]Dueling Double DQN收入、成本边缘/云层设备选择
DDQP[53]Double DQN接受率、服务可用性放置与备份节点序号
PA-DRL[54]DQN传输时延、网络负载平衡放置与备份节点序号
Actor-Critic
类算法
E2D2PG[47]DDPG接受率二进制节点放置决策重赋链路权重
改进DDPG[48]DDPG时延、部署成本资源分配与放置节点
A-DDPG[49]基于注意力的DDPG收入、成本二进制节点放置决策二进制链路放置决策
A-DDPG[50]基于注意力的DDPG收入、成本二进制节点放置决策二进制链路放置决策
E-DDPG[51]DDPG时延二进制节点放置决策二进制链路放置决策
Opt-SAC[52]SAC时延节点放置概率
A2C[55]A2C/PPO可用性、能耗节点序号与冗余实例数量
Cand-RL[56]PPO部署成本、能耗候选节点序号与备份决策最短路径
Chebyshev-MODRL[59]Actor-Critic时延,拥塞率节点序号最短路径
DSPPV[60]A3C时延、资源消耗、吞吐量节点序号
SPC3D[61]A3C时延,资源消耗、吞吐量候选节点序号
图神经网络
辅助方法
DeepOpt[62]REINFORCE部署成本二进制节点放置决策
Kolin[64]REINFORCE部署成本、流量转发成本节点放置概率重赋链路权重
GDRL-VNFP[65]A3C部署成本、流量转发成本节点序号最短路径
DRL-D[66]TD长期平均收入二进制候选节点放置决策
多智能体协
同放置方法
MA-DQL[10]DQN(独立学习)接受率、部署成本IoT设备选择
MADDPG[67]MADDPG(集中训练分散执行)负载平衡、时延节点序号链路序号
two-stage GCN-based DRL[68]MAPPO(集中训练分散执行)接受率、成本路由路径路由路径
MADRL-P&R[69]DDPG(独立学习)成本、时延节点智能体放置决策链路智能体放置决策
DT-assisted MADDPG[70]MADDPG(集中训练分散执行)能耗、时延、部署成本二进制服务域决策路由路径
GCN-MADDPG[71]MADDPG(集中训练分散执行)资源使用率、时延节点序号
), ArticleFig(id=1325493713014752187, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, language=CN, label=表4, caption=

基于强化学习的SFC放置算法对比

, figureFileSmall=null, figureFileBig=null, tableContent=
分类放置算法强化学习算法优化目标VNF放置策略虚拟链路放置策略
基础DRL
算法
NFVdeep[7]PG运行成本、吞吐量节点序号
DRL-QOR[43]DQNQoE节点序号
DQN-AR[44]DQN运行成本、带宽资源节点序号链路序号
ADAP[45]DQN带宽消耗、CPU消耗2种基于贪婪的候选算法重赋链路权重
SDRM-EC-DPR[46]Dueling Double DQN收入、成本边缘/云层设备选择
DDQP[53]Double DQN接受率、服务可用性放置与备份节点序号
PA-DRL[54]DQN传输时延、网络负载平衡放置与备份节点序号
Actor-Critic
类算法
E2D2PG[47]DDPG接受率二进制节点放置决策重赋链路权重
改进DDPG[48]DDPG时延、部署成本资源分配与放置节点
A-DDPG[49]基于注意力的DDPG收入、成本二进制节点放置决策二进制链路放置决策
A-DDPG[50]基于注意力的DDPG收入、成本二进制节点放置决策二进制链路放置决策
E-DDPG[51]DDPG时延二进制节点放置决策二进制链路放置决策
Opt-SAC[52]SAC时延节点放置概率
A2C[55]A2C/PPO可用性、能耗节点序号与冗余实例数量
Cand-RL[56]PPO部署成本、能耗候选节点序号与备份决策最短路径
Chebyshev-MODRL[59]Actor-Critic时延,拥塞率节点序号最短路径
DSPPV[60]A3C时延、资源消耗、吞吐量节点序号
SPC3D[61]A3C时延,资源消耗、吞吐量候选节点序号
图神经网络
辅助方法
DeepOpt[62]REINFORCE部署成本二进制节点放置决策
Kolin[64]REINFORCE部署成本、流量转发成本节点放置概率重赋链路权重
GDRL-VNFP[65]A3C部署成本、流量转发成本节点序号最短路径
DRL-D[66]TD长期平均收入二进制候选节点放置决策
多智能体协
同放置方法
MA-DQL[10]DQN(独立学习)接受率、部署成本IoT设备选择
MADDPG[67]MADDPG(集中训练分散执行)负载平衡、时延节点序号链路序号
two-stage GCN-based DRL[68]MAPPO(集中训练分散执行)接受率、成本路由路径路由路径
MADRL-P&R[69]DDPG(独立学习)成本、时延节点智能体放置决策链路智能体放置决策
DT-assisted MADDPG[70]MADDPG(集中训练分散执行)能耗、时延、部署成本二进制服务域决策路由路径
GCN-MADDPG[71]MADDPG(集中训练分散执行)资源使用率、时延节点序号
), ArticleFig(id=1325493713383850940, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, language=EN, label=Table 5, caption=Comparison of SFC Scheduling Algorithms Based on Reinforcement Learning, figureFileSmall=null, figureFileBig=null, tableContent=
调度算法强化学习算法优化目标VNF调度动作优先级机制
DQS[72]DQN传输、传播、
处理、排队时延
MDC序号基于未处理VNF数量:未处理VNF越多优先级越高
文献[73]DQN完成时间服务与节点序号动态优先级:可暂停当前VNF执行高收益任务;
集成最短作业优先和最小资源优先规则
VNF-CSWD[74]Double DQN传输、传播、
处理、排队时延
VNF节点序号最短剩余时间优先:根据排队时延和完成时延计算优先级
DRL-MSSNL-
SAGIN[75]
Double DQN接受率(有限时延)VNF节点序号基于数据量的SFC优先级:资源冲突
时按数据量升序排序处理
文献[76]Q-Learning完成时间VNF节点序号时延敏感度权重:高时延敏感服务获得更大权重系数
CA2C[77]Compound A2C放置与调度成本、
信息年龄
节点放置决策、功率
和子载波分配
基于信息年龄:信息年龄值越高优先级越高
PPO-MCTS[78]PPO处理时延候选节点与调度策略学习型公平优先级:通过神经网络动态分配优先级确保公平性
文献[79]Q-Learning总加权完成时间服务与节点匹配基于业务类型的静态权重:按紧急程度和服务区域分为4级
文献[80]PPO完成时间、资源利
用率、接受率
服务与节点对所有服务请求默认具有相同处理优先级,
通过强化学习算法动态调整处理顺序
D3QN[81]Dueling Double DQN接受率(有限时延)5种调度算法的选择复合规则自适应选择:根据网络状态在
5种优先级规则中选择最优方案
), ArticleFig(id=1325493713803281341, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, language=CN, label=表5, caption=

基于强化学习的SFC调度算法对比

, figureFileSmall=null, figureFileBig=null, tableContent=
调度算法强化学习算法优化目标VNF调度动作优先级机制
DQS[72]DQN传输、传播、
处理、排队时延
MDC序号基于未处理VNF数量:未处理VNF越多优先级越高
文献[73]DQN完成时间服务与节点序号动态优先级:可暂停当前VNF执行高收益任务;
集成最短作业优先和最小资源优先规则
VNF-CSWD[74]Double DQN传输、传播、
处理、排队时延
VNF节点序号最短剩余时间优先:根据排队时延和完成时延计算优先级
DRL-MSSNL-
SAGIN[75]
Double DQN接受率(有限时延)VNF节点序号基于数据量的SFC优先级:资源冲突
时按数据量升序排序处理
文献[76]Q-Learning完成时间VNF节点序号时延敏感度权重:高时延敏感服务获得更大权重系数
CA2C[77]Compound A2C放置与调度成本、
信息年龄
节点放置决策、功率
和子载波分配
基于信息年龄:信息年龄值越高优先级越高
PPO-MCTS[78]PPO处理时延候选节点与调度策略学习型公平优先级:通过神经网络动态分配优先级确保公平性
文献[79]Q-Learning总加权完成时间服务与节点匹配基于业务类型的静态权重:按紧急程度和服务区域分为4级
文献[80]PPO完成时间、资源利
用率、接受率
服务与节点对所有服务请求默认具有相同处理优先级,
通过强化学习算法动态调整处理顺序
D3QN[81]Dueling Double DQN接受率(有限时延)5种调度算法的选择复合规则自适应选择:根据网络状态在
5种优先级规则中选择最优方案
), ArticleFig(id=1325493714205934526, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, language=EN, label=Table 6, caption=Comparison of SFC Reconfiguration Algorithms Based on Reinforcement Learning, figureFileSmall=null, figureFileBig=null, tableContent=
重配置算法强化学习算法优化目标VNF重配置动作应用场景
文献[82]DQNQoS、能耗迁移决策(是否迁移、
目标节点)与资源重分配
5G网络切片架构下VNF迁移优化
ISFCDAA[83]Double DQN迁移成本、实例化成本、
资源均衡度
重配置策略(纵向扩展、
横向扩展、迁移)
NFV数据中心网络应对SFC动态需求的重配置
DDQN-VNFPA[84]Double DQN部署成本、接受
率、运行成本
对节点是否进行优化
决策(额外算法)
SDN/NFV环境下应对动态负载的VNF重配置
MQDR[85]DQN接受率二进制节点迁移决策SDN/NFV环境下处理动态到达SFC请求的重配置
LS-SFCM[86]DDPG时延、迁移成本可迁移节点序号卫星通信网络针对时延敏感型业务的SFC迁移优化
DQN-MTD[87]DQN宕机时间、迁移成功率二进制节点迁移决策边缘计算环境用户移动路径可预测场景的迁移时机优化
DRL-ADMO[88]Double DQN时延、迁移成功
率、迁移时间
迁移路径选择多接入边缘计算环境下用户移动导致的
SFC动态迁移优化
DDQ[89]Dyna-Q收入、重配置成本迁移与重路由决策IoT网络基于GNN预测资源需求的SFC重配置优化
SARM[90]DQN时延、迁移成本、带宽成本下一个时隙的短期状态值基于LSTM与DQN预测计算负载的主动重配置优化
PAVM[91]DQN时延、节点与链路负载率迁移节点序号NFV环境下基于优先级感知的可靠VNF迁移优化
APSD[92]DQN时延、接受率、部署成本重配置节点序号NFV环境下优先级感知与自动伸缩的重配置
SADDQN/
MADDQN[93]
Double DQN /
MADDQN
能耗、迁移成本、
收入、负载均衡
迁移决策(是否迁移、
目标节点)
SDN/NFV云环境下面向容错SFC弹性重配置
RA-SFCM[94]A2C可靠性、迁移成本可迁移节点序号边缘计算环境下多智能体的可靠性保障型SFC迁移优化
), ArticleFig(id=1325493715904627647, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, language=CN, label=表6, caption=

基于强化学习的SFC重配置算法对比

, figureFileSmall=null, figureFileBig=null, tableContent=
重配置算法强化学习算法优化目标VNF重配置动作应用场景
文献[82]DQNQoS、能耗迁移决策(是否迁移、
目标节点)与资源重分配
5G网络切片架构下VNF迁移优化
ISFCDAA[83]Double DQN迁移成本、实例化成本、
资源均衡度
重配置策略(纵向扩展、
横向扩展、迁移)
NFV数据中心网络应对SFC动态需求的重配置
DDQN-VNFPA[84]Double DQN部署成本、接受
率、运行成本
对节点是否进行优化
决策(额外算法)
SDN/NFV环境下应对动态负载的VNF重配置
MQDR[85]DQN接受率二进制节点迁移决策SDN/NFV环境下处理动态到达SFC请求的重配置
LS-SFCM[86]DDPG时延、迁移成本可迁移节点序号卫星通信网络针对时延敏感型业务的SFC迁移优化
DQN-MTD[87]DQN宕机时间、迁移成功率二进制节点迁移决策边缘计算环境用户移动路径可预测场景的迁移时机优化
DRL-ADMO[88]Double DQN时延、迁移成功
率、迁移时间
迁移路径选择多接入边缘计算环境下用户移动导致的
SFC动态迁移优化
DDQ[89]Dyna-Q收入、重配置成本迁移与重路由决策IoT网络基于GNN预测资源需求的SFC重配置优化
SARM[90]DQN时延、迁移成本、带宽成本下一个时隙的短期状态值基于LSTM与DQN预测计算负载的主动重配置优化
PAVM[91]DQN时延、节点与链路负载率迁移节点序号NFV环境下基于优先级感知的可靠VNF迁移优化
APSD[92]DQN时延、接受率、部署成本重配置节点序号NFV环境下优先级感知与自动伸缩的重配置
SADDQN/
MADDQN[93]
Double DQN /
MADDQN
能耗、迁移成本、
收入、负载均衡
迁移决策(是否迁移、
目标节点)
SDN/NFV云环境下面向容错SFC弹性重配置
RA-SFCM[94]A2C可靠性、迁移成本可迁移节点序号边缘计算环境下多智能体的可靠性保障型SFC迁移优化
)], attaches=null, journal=Journal(id=1324759606947315908, delFlag=0, nameCn=计算机研究与发展, nameEn=Journal of Computer Research and Development, nameHistory1=null, nameHistory2=null, issn=1000-1239, eissn=null, cn=11-1777/TP, coden=null, periodic=0, language=CN, oaType=null, ccby=null, superviseOffice=null, ownerOffice=null, pubOffice=null, editorOffice=null, officeType=null, aims=null, clcCode=null, officeProv=null, officeCity=null, officeAddr=null, officeZip=null, officeEmail=null, officePhone=null, editDirector=null, officeDirector=null, officeDirectorPhone=null, officeStaffNum=null, officeEmpNum=null, coverPicUrl=k2l8zgqwstuxxevFX2qxVA==, journalPrice=null, startedYear=null, abbrevIsoEn=Journal of Computer Research and Development, journalRemark=null, publicationField=null, createdTime=1793776903141, updatedTime=1793850698612, createdBy=18614031015, updatedBy=13701087609, firstLetterCn=J, firstLetterEn=J, subjectCode=Electronic Information, subjectName=null, subjectCodeEn=Electronic Information, subjectNameEn=null, picCn=k2l8zgqwstuxxevFX2qxVA==, picEn=r6k0Ry+RD+c/VXj5QarmDw==, jcr=null, cjcr=null, exts=[JournalExt(id=1325069127710286688, language=CN, name=计算机研究与发展, nameHistory1=null, nameHistory2=null, managedBy=, sponsoredBy=, publishedBy=, editorOffice=, officeProv=null, officeCity=null, officeAddr=, officeZip=, editDirector=, officeDirector=null, officePhone=null, coverPicUrl=null, journalRemark=, submitArticleUrl=null, websiteUrl=, createdTime=1793850698640, updatedTime=1793850698640, createdBy=13701087609, updatedBy=13701087609, submissionGuidelinesUrl=, submissionAuthorUrl=https://crad.ict.ac.cn/journalx/authorLogOn.action, submissionEditorUrl=https://crad.ict.ac.cn/journalx/editorLogOn.action, submissionReviewUrl=https://crad.ict.ac.cn/journalx/expertLogOn.action, submissionCeEditorUrl=https://crad.ict.ac.cn/journalx/editorInChiefLogOn.action, submissionAeEditorUrl=, option={"copyright":""}), JournalExt(id=1325069127777395553, language=EN, name=Journal of Computer Research and Development, nameHistory1=null, nameHistory2=null, managedBy=, sponsoredBy=, publishedBy=, editorOffice=, officeProv=null, officeCity=null, officeAddr=, officeZip=, editDirector=, officeDirector=null, officePhone=null, coverPicUrl=null, journalRemark=, submitArticleUrl=null, websiteUrl=, createdTime=1793850698656, updatedTime=1793850698656, createdBy=13701087609, updatedBy=13701087609, submissionGuidelinesUrl=, submissionAuthorUrl=https://crad.ict.ac.cn/journalx/authorLogOn.action, submissionEditorUrl=https://crad.ict.ac.cn/journalx/editorLogOn.action, submissionReviewUrl=https://crad.ict.ac.cn/journalx/expertLogOn.action, submissionCeEditorUrl=https://crad.ict.ac.cn/journalx/editorInChiefLogOn.action, submissionAeEditorUrl=, option={"copyright":""})], databaseList=null, tenantJournalId=1324759914146529478, websiteList=[Website(id=1325070300139892771, webName=null, webTitle=null, webDomain=null, webCopyrigh=null, webIpcNo=null, seoTitle=null, seoKeywords=null, seoDescription=null, tenantJournalId=null, journalId=1324759914146529478, journalNameCn=null, journalNameEn=null, grayFlag=null, tenantId=1146029695717560320, platformId=null, journalGroupId=null, journalGroupNameCn=null, journalGroupNameEn=null, type=1, domain=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN, language=CN, createTime=1793850978169, createBy=18614031015, updateTime=1793851040259, updateBy=18614031015, name=计算机研究与发展-中文, tplId=1146099689490845704, title=计算机研究与发展, delFlag=0, indexPage=/home, props=[WebsiteProps(id=1325070893751349360, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=articleTextType, value=kx, createTime=1793851119697, updateTime=1793851119697, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893730377837, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=banner, value=null, createTime=1793851119692, updateTime=1793851119692, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893772320883, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=grayFlag, value=0, createTime=1793851119702, updateTime=1793851119702, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893721989228, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=logo, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/file/pic?fileId=U27tDsy717YF7XSPA+Vwdw==, createTime=1793851119690, updateTime=1793851119690, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893784903797, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=minRunFlag, value=0, createTime=1793851119705, updateTime=1793851119705, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893742960751, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=picServerUrl, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/file/pic, createTime=1793851119695, updateTime=1793851119695, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893780709492, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=silenceFlag, value=0, createTime=1793851119704, updateTime=1793851119704, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893738766446, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=staticResourcePath, value=https://castjournals.cast.org.cn/joweb/cast_kjdb_cn_619/, createTime=1793851119694, updateTime=1793851119694, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893755543665, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=themeColor, value=null, createTime=1793851119698, updateTime=1793851119698, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893763932274, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=themeStyle, value=null, createTime=1793851119700, updateTime=1793851119700, creator=18614031015, updator=18614031015)]), Website(id=1325070300580294698, webName=null, webTitle=null, webDomain=null, webCopyrigh=null, webIpcNo=null, seoTitle=null, seoKeywords=null, seoDescription=null, tenantJournalId=null, journalId=1324759914146529478, journalNameCn=null, journalNameEn=null, grayFlag=null, tenantId=1146029695717560320, platformId=null, journalGroupId=null, journalGroupNameCn=null, journalGroupNameEn=null, type=1, domain=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN, language=EN, createTime=1793850978274, createBy=18614031015, updateTime=1793851036638, updateBy=18614031015, name=计算机研究与发展-英文, tplId=1146101810881728533, title=Journal of Computer Research and Development, delFlag=0, indexPage=/home, props=[WebsiteProps(id=1325070925288321148, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=articleTextType, value=kx, createTime=1793851127216, updateTime=1793851127216, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925263155321, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=banner, value=null, createTime=1793851127210, updateTime=1793851127210, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925309292671, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=grayFlag, value=0, createTime=1793851127221, updateTime=1793851127221, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925254766712, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=logo, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/file/pic?fileId=U27tDsy717YF7XSPA+Vwdw==, createTime=1793851127208, updateTime=1793851127208, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925326069889, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=minRunFlag, value=0, createTime=1793851127225, updateTime=1793851127225, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925279932539, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=picServerUrl, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/file/pic, createTime=1793851127214, updateTime=1793851127214, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925317681280, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=silenceFlag, value=0, createTime=1793851127223, updateTime=1793851127223, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925271543930, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=staticResourcePath, value=https://castjournals.cast.org.cn/joweb/cast_kjdb_en_623/, createTime=1793851127212, updateTime=1793851127212, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925292515453, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=themeColor, value=null, createTime=1793851127217, updateTime=1793851127217, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925300904062, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=themeStyle, value=null, createTime=1793851127219, updateTime=1793851127219, creator=18614031015, updator=18614031015)])], journalTitle=计算机研究与发展, weixinUrl=null, journalUrl=https://crad.ict.ac.cn/, iacademicId=null, status=1, seqNo=null, journalTitleEn=Journal of Computer Research and Development, journalPhotoCn=k2l8zgqwstuxxevFX2qxVA==, journalPhotoEn=r6k0Ry+RD+c/VXj5QarmDw==, journalFirstLetter=J, journalRecommend=null, journalNew=null, journalCollection=null, jcrJf=null, cjcrJf=null, jcrJfStr=null, cjcrJfStr=null, submissionFirstDecision=null, sciSubjectClassification=null, casSubjectClassification=null, citeScore=null, totalCitationFrequency=null, icpCode=null, psCode=null, advertisingLicenseCode=null, copyrightInformation=null, country=null, option=, provinceCode=null, provinceName=null, collectFlag=false, interPubPlatform=, interPubPlatformUrl=null), detailUrlCn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/10.7544/issn1000-1239.202550002, detailUrlEn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/10.7544/issn1000-1239.202550002, pdfUrlCn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/PDF/10.7544/issn1000-1239.202550002, pdfUrlEn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/PDF/10.7544/issn1000-1239.202550002, aliStartDate=0, aliEndDate=0, collectionFlag=false, citedCount=null, citedUrl=null, previewStatus=0, delFlag=0, hasFullText=1, orderTime=1769875200000, fullTextJson=null, articleText=null, reference=null)
收藏切换
基于强化学习的服务功能链部署综述
爱因坦 1 , 阮宏玮 1 , 刘翰林 1 , 李华 1, 2, 3, *
计算机研究与发展 | 网络与信息安全 2026,63(2): 505-524
收起
收藏切换
计算机研究与发展 |网络与信息安全 2026 , 63 (2) : 505 -524
基于强化学习的服务功能链部署综述
全屏
[Author(id=1325493689522455429, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, orderNo=0, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=aiyintan@mail.imu.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1325493689904137095, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, authorId=1325493689522455429, language=EN, stringName=Yintan Ai, firstName=Yintan, middleName=null, lastName=Ai, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1College of Computer Science, Inner Mongolia University, Hohhot 010021, bio={"content":"

born in 1994. PhD candidate. His main research interests include software-defined networking, service function chaining, and multi-agent reinforcement learning

"}, bioImg=null, bioContent=

born in 1994. PhD candidate. His main research interests include software-defined networking, service function chaining, and multi-agent reinforcement learning

, aboutCorrespAuthor=null), CN=AuthorExt(id=1325493690390676360, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, authorId=1325493689522455429, language=CN, stringName=爱因坦, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1内蒙古大学计算机学院 呼和浩特 010021, bio={"img":"9hXPvmADSjdXt/3DBe4MvA==","content":"

1994年生。博士研究生。主要研究方向为软件定义网络、服务功能链、多智能体强化学习

"}, bioImg=9hXPvmADSjdXt/3DBe4MvA==, bioContent=

1994年生。博士研究生。主要研究方向为软件定义网络、服务功能链、多智能体强化学习

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1325493684887749498, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, xref=1, ext=[AuthorCompanyExt(id=1325493685646918523, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, companyId=1325493684887749498, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1College of Computer Science, Inner Mongolia University, Hohhot 010021), AuthorCompanyExt(id=1325493685734998908, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, companyId=1325493684887749498, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1内蒙古大学计算机学院 呼和浩特 010021)])]), Author(id=1325493691191788426, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, orderNo=1, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=csrhw@imu.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1325493691984511884, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, authorId=1325493691191788426, language=EN, stringName=Hongwei Ruan, firstName=Hongwei, middleName=null, lastName=Ruan, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1College of Computer Science, Inner Mongolia University, Hohhot 010021, bio={"content":"

born in 1981. PhD. His main research interests include software-defined networking and software testing. ()

"}, bioImg=null, bioContent=

born in 1981. PhD. His main research interests include software-defined networking and software testing. ()

, aboutCorrespAuthor=null), CN=AuthorExt(id=1325493693632873357, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, authorId=1325493691191788426, language=CN, stringName=阮宏玮, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1内蒙古大学计算机学院 呼和浩特 010021, bio={"img":"VJ2TOW7EPUJtSWrVbLpJpA==","content":"

1981年生。博士。主要研究方向为软件定义网络、软件测试

"}, bioImg=VJ2TOW7EPUJtSWrVbLpJpA==, bioContent=

1981年生。博士。主要研究方向为软件定义网络、软件测试

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1325493684887749498, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, xref=1, ext=[AuthorCompanyExt(id=1325493685646918523, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, companyId=1325493684887749498, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1College of Computer Science, Inner Mongolia University, Hohhot 010021), AuthorCompanyExt(id=1325493685734998908, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, companyId=1325493684887749498, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1内蒙古大学计算机学院 呼和浩特 010021)])]), Author(id=1325493693775479695, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, orderNo=2, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=imucslhl@mail.imu.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1325493694870193041, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, authorId=1325493693775479695, language=EN, stringName=Hanlin Liu, firstName=Hanlin, middleName=null, lastName=Liu, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1College of Computer Science, Inner Mongolia University, Hohhot 010021, bio={"content":"

born in 1996. PhD candidate. Her main research interests include intent-based networking and graph neural networks. ()

"}, bioImg=null, bioContent=

born in 1996. PhD candidate. Her main research interests include intent-based networking and graph neural networks. ()

, aboutCorrespAuthor=null), CN=AuthorExt(id=1325493694966662034, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, authorId=1325493693775479695, language=CN, stringName=刘翰林, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1内蒙古大学计算机学院 呼和浩特 010021, bio={"img":"D33AmwLpNXR1CyGpMxJd+Q==","content":"

1996年生。博士研究生。主要研究方向为意图驱动网络、图神经网络

"}, bioImg=D33AmwLpNXR1CyGpMxJd+Q==, bioContent=

1996年生。博士研究生。主要研究方向为意图驱动网络、图神经网络

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1325493684887749498, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, xref=1, ext=[AuthorCompanyExt(id=1325493685646918523, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, companyId=1325493684887749498, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1College of Computer Science, Inner Mongolia University, Hohhot 010021), AuthorCompanyExt(id=1325493685734998908, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, companyId=1325493684887749498, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1内蒙古大学计算机学院 呼和浩特 010021)])]), Author(id=1325493695398675348, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, orderNo=3, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=cslihua@imu.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=1, authorType=1, ext={EN=AuthorExt(id=1325493696673743769, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, authorId=1325493695398675348, language=EN, stringName=Hua Li, firstName=Hua, middleName=null, lastName=Li, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, 3, *, address=1College of Computer Science, Inner Mongolia University, Hohhot 010021
2Engineering Research Center of Ecological Big Data (Inner Mongolia University), Ministry of Education, Hohhot 010021
3Inner Mongolia Engineering Laboratory for Cloud Computing and Service Software (Inner Mongolia University), Hohhot 010021, bio={"content":"

born in 1964. PhD, professor, PhD supervisor. Her main research interests include cloud-network integration, big data analysis and evaluation methods, and software service computing and testing

"}, bioImg=null, bioContent=

born in 1964. PhD, professor, PhD supervisor. Her main research interests include cloud-network integration, big data analysis and evaluation methods, and software service computing and testing

, aboutCorrespAuthor=null), CN=AuthorExt(id=1325493698351465370, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, authorId=1325493695398675348, language=CN, stringName=李华, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, 3, *, address=1内蒙古大学计算机学院 呼和浩特 010021
2生态大数据教育部工程研究中心(内蒙古大学) 呼和浩特 010021
3内蒙古自治区云计算与服务软件工程实验室(内蒙古大学) 呼和浩特 010021, bio={"img":"spUYtazfBuMiefNWiI0mDg==","content":"

1964年生。博士,教授,博士生导师。主要研究方向为云网融合、大数据分析与评估方法、软件服务计算与测试

"}, bioImg=spUYtazfBuMiefNWiI0mDg==, bioContent=

1964年生。博士,教授,博士生导师。主要研究方向为云网融合、大数据分析与评估方法、软件服务计算与测试

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1325493684887749498, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, xref=1, ext=[AuthorCompanyExt(id=1325493685646918523, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, companyId=1325493684887749498, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1College of Computer Science, Inner Mongolia University, Hohhot 010021), AuthorCompanyExt(id=1325493685734998908, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, companyId=1325493684887749498, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1内蒙古大学计算机学院 呼和浩特 010021)]), AuthorCompany(id=1325493686091514749, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, xref=2, ext=[AuthorCompanyExt(id=1325493686494167934, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, companyId=1325493686091514749, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2Engineering Research Center of Ecological Big Data (Inner Mongolia University), Ministry of Education, Hohhot 010021), AuthorCompanyExt(id=1325493686557082495, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, companyId=1325493686091514749, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2生态大数据教育部工程研究中心(内蒙古大学) 呼和浩特 010021)]), AuthorCompany(id=1325493687366583168, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, xref=3, ext=[AuthorCompanyExt(id=1325493687404331905, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, companyId=1325493687366583168, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=3Inner Mongolia Engineering Laboratory for Cloud Computing and Service Software (Inner Mongolia University), Hohhot 010021), AuthorCompanyExt(id=1325493687416914818, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1325493670253822813, companyId=1325493687366583168, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=3内蒙古自治区云计算与服务软件工程实验室(内蒙古大学) 呼和浩特 010021)])])]
爱因坦1 , 阮宏玮1 , 刘翰林1 , 李华1, 2, 3, *
作者信息
  • 1内蒙古大学计算机学院 呼和浩特 010021
  • 2生态大数据教育部工程研究中心(内蒙古大学) 呼和浩特 010021
  • 3内蒙古自治区云计算与服务软件工程实验室(内蒙古大学) 呼和浩特 010021
通讯作者:
李华()
作者简介:

1994年生。博士研究生。主要研究方向为软件定义网络、服务功能链、多智能体强化学习

1981年生。博士。主要研究方向为软件定义网络、软件测试

1996年生。博士研究生。主要研究方向为意图驱动网络、图神经网络

1964年生。博士,教授,博士生导师。主要研究方向为云网融合、大数据分析与评估方法、软件服务计算与测试

Survey on Reinforcement Learning-Based Service Function Chain Deployment
Yintan Ai1 , Hongwei Ruan1 , Hanlin Liu1 , Hua Li1, 2, 3, *
Affiliations
  • 1College of Computer Science, Inner Mongolia University, Hohhot 010021
  • 2Engineering Research Center of Ecological Big Data (Inner Mongolia University), Ministry of Education, Hohhot 010021
  • 3Inner Mongolia Engineering Laboratory for Cloud Computing and Service Software (Inner Mongolia University), Hohhot 010021
  • born in 1994. PhD candidate. His main research interests include software-defined networking, service function chaining, and multi-agent reinforcement learning

    born in 1981. PhD. His main research interests include software-defined networking and software testing. ()

    born in 1996. PhD candidate. Her main research interests include intent-based networking and graph neural networks. ()

    born in 1964. PhD, professor, PhD supervisor. Her main research interests include cloud-network integration, big data analysis and evaluation methods, and software service computing and testing

出版时间: 2026-02-01 doi: 10.7544/issn1000-1239.202550002
文章导航
收藏切换

在软件定义网络和网络功能虚拟化范式下,服务功能链(service function chaining,SFC)技术通过灵活编排虚拟网络功能实现了网络服务的定制化部署。然而,动态网络环境下SFC部署面临着决策空间大、环境复杂多变等挑战。强化学习因其自适应学习复杂环境特征并动态决策的能力,在解决SFC部署问题上展现出显著优势。该研究首先阐述了SFC部署的基本概念与技术架构,并具体介绍了基于强化学习的SFC部署框架。随后,从算法设计、应用场景和优化策略等角度,梳理并深入分析了强化学习在SFC放置、调度和重配置3个关键阶段中的研究进展与创新应用。最后,总结了现有研究在算法设计、性能优化和实际部署方面的优势与局限,并分析了该领域的技术挑战与未来发展趋势。

服务功能链  /  强化学习  /  资源优化  /  虚拟网络功能  /  智能部署

With the rapid development of emerging network technologies such as B5G/6G and Internet of things, network services present characteristics of diverse business types, differentiated quality requirements, and dynamic resource demands. Under the software defined networking (SDN) and network function virtualization (NFV) paradigm, service function chaining (SFC) technology enables customized network service deployment through flexible orchestration of virtual network functions (VNF). However, SFC deployment in dynamic network environments faces challenges such as large decision space, and complex and changing environments. Reinforcement learning (RL) demonstrates significant advantages in solving SFC deployment problems due to its capability to adaptively learn complex environmental characteristics and make dynamic decisions. Although research on applying RL to SFC deployment has made certain progress, a systematic and comprehensive review analysis remains lacking. To address this gap, the fundamental concepts and technical architecture of SFC deployment are elaborated, along with a specific introduction to the RL-based SFC deployment framework. Subsequently, from the perspectives of algorithm design, application scenarios, and optimization strategies, the research progress and innovative applications of reinforcement learning in three key phases—SFC placement, scheduling, and reconfiguration—are systematically examined and thoroughly analyzed. Finally, the advantages and limitations of existing research in terms of algorithm design, performance optimization, and practical deployment are summarized, while technical challenges and future development trends in this field are analyzed.

service function chaining (SFC)  /  reinforcement learning  /  resource optimization  /  virtual network function  /  intelligent deployment
爱因坦, 阮宏玮, 刘翰林, 李华. 基于强化学习的服务功能链部署综述. 计算机研究与发展, 2026 , 63 (2) : 505 -524 . DOI: 10.7544/issn1000-1239.202550002
Yintan Ai, Hongwei Ruan, Hanlin Liu, Hua Li. Survey on Reinforcement Learning-Based Service Function Chain Deployment[J]. Journal of Computer Research and Development, 2026 , 63 (2) : 505 -524 . DOI: 10.7544/issn1000-1239.202550002
随着互联网技术的迅速发展,全球社会正朝着更紧密互联、数字化和多元化的方向迈进。互联网已深度融入人们的日常生活、工作和学习之中,成为推动技术创新和经济增长的关键驱动力。据统计,2024年已有55亿人接入互联网,占全球人口的68%,相比2023年的65%又有显著增长[1]。与此同时,物联网、工业互联网等新型应用场景正驱动网络服务向更加智能化、自动化的方向演进[2]。在此背景下,传统网络在应对不断增长的网络流量和日益提升的服务质量(quality of service,QoS)需求方面,正面临服务部署效率不足、资源利用率偏低以及管理灵活性有限等多重挑战。
网络功能虚拟化(network function virtualization,NFV)和软件定义网络(software defined networking,SDN)的出现为解决传统网络架构的局限性提供了新思路[3-4]。NFV通过将网络功能从专用硬件解耦,实现了网络服务的虚拟化部署[3];SDN则通过控制平面和数据平面分离,实现了网络流量的灵活调度[4]。在此基础上发展起来的服务功能链(service function chaining,SFC)技术,能够将虚拟网络功能(virtual network function,VNF)按序连接,为用户提供定制化的网络服务[5]。NFV和SDN环境的融合为网络资源的灵活调度和按需分配提供了技术基础,使得基于多维度优化目标(如时延、带宽利用率、能耗等)的智能部署成为可能[6]。
然而,在复杂多变的网络环境下,SFC部署面临着诸多挑战:1)决策空间庞大,传统优化方法难以在有限时间内得到满意解;2)网络状态动态变化,静态部署策略难以适应;3)多样化服务需求与有限网络资源之间的矛盾日益突出;4)QoS保证与资源利用效率的平衡难以把握[7]。这些挑战促使研究者探索新的智能优化方法。强化学习(reinforcement learning,RL)[8]凭借其自适应学习和动态决策能力,在SFC部署中展现出独特优势。特别是随着深度学习技术的突破,基于深度强化学习(deep reinforcement learning,DRL)[7]、图神经网络(graph neural network,GNN)辅助的强化学习[9]以及多智能体强化学习(multi-agent reinforcement learning,MARL)[10]等方法在解决大规模动态优化问题上取得了显著进展。
根据欧洲电信标准协会(European Telecommunications Standards Institute,ETSI)发布的面向下一个10年NFV演进白皮书的分析,SFC作为NFV的关键应用场景,其发展将从自动化向智能化演进,人工智能将在服务链编排和优化中发挥越来越核心的作用[11]。特别是在B5G/6G、边缘计算、工业互联网等新型场景中,SFC部署需要更加灵活精准地响应多样化的服务需求,这也使得基于强化学习的SFC部署方法成为亟需探索的重要方向[12]。
为系统化地研究基于强化学习的服务功能链部署进展,本文对现有文献进行了全面的调研和深入分析,如表1所示。研究发现,早期的相关综述主要聚焦于SFC的基本架构、实现方法和优化策略[6,13-16],对基于强化学习的方法分析较少。近年来,虽有部分文献开始关注强化学习在SFC以及虚拟网络功能转发图VNF-FG(virtual network function forwarding graph)部署中的应用[12,17-18],但缺乏系统性和全面性的分析。此外,现有研究在SFC的放置、调度和重配置等关键环节的优化上关注不足,对算法在不同场景下的适用性及局限性也未进行充分探讨。
本文系统分析并总结了基于强化学习的服务功能链部署研究现状,围绕SFC的放置、调度和重配置3个关键阶段,梳理了现有研究的主要进展与成果。同时,从算法设计、应用场景和优化策略等角度,深入探讨了强化学习在SFC部署中的创新应用,揭示了优化策略的演进规律及不同部署阶段之间的关联性。这些分析不仅为复杂网络环境下的算法选择提供了理论指导,也为实际应用中的技术实现提供了参考支持。此外,系统分析了现有研究的优势与局限性,总结了当前研究中面临的关键挑战,并提出了未来可能的研究方向。
服务功能链是一种先进且灵活的网络技术,旨在根据具体的服务需求与网络资源的可用性,对数据流进行分类管理并动态部署策略,从而提供定制化的网络服务[5]。SFC通过将网络流量引导至一系列按特定顺序排列的网络功能模块,确保数据流逐步经过处理以实现既定的服务效果[6]。
本节首先介绍NFV的基本概念,在此基础上阐述SFC技术的基本架构,并分析SFC部署的关键流程与步骤,最后讨论强化学习在SFC部署中的应用框架。
NFV是一种通过解耦网络功能与专用硬件,从而提升网络灵活性与扩展性的关键技术[3]。其核心理念是将传统依赖专用硬件实现的网络功能抽象并虚拟化,使其能够以VNF的形式运行于标准化、高性能的通用服务器之上,从而打破传统网络设备与软件的捆绑。通过这一解耦过程,NFV能够支持计算、存储和网络资源的按需动态分配,实现资源的高效共享与优化利用,进而提升整体硬件资源的利用率。此外,NFV架构的引入不仅加速了新型网络服务的开发与部署,提升了网络管理的敏捷性与运维效率,同时还能有效降低资本支出(capital expenditure,CAPEX)和运营支出(operational expenditure,OPEX),满足现代云计算环境下日益复杂的业务需求,进一步增强网络的可靠性与弹性[3]。
NFV概念最早由ETSI于2012年提出,并在此基础上成立了网络功能虚拟化行业规范组(network function virtualization industry specification group,NFV ISG),以推动NFV相关技术的标准化与生态系统建设。自成立以来,NFV ISG持续发布涵盖NFV技术机遇、应用场景、架构框架等多个方面的重要技术文档,为NFV的理论研究、技术发展及产业推广提供了重要的标准化指导和技术支撑。ETSI定义的NFV架构框架如图1所示。
NFV架构主要由网络功能虚拟化基础设施(network function virtualization infrastructure,NFVI)、NFV和管理与编排(management and orchestration,MANO)三大核心模块构成[20]。其中,NFVI由物理硬件和虚拟化层组成,为VNF的运行提供基础计算、存储及网络资源支持,并通过虚拟化技术实现资源的高效管理和共享。VNF作为网络功能的虚拟化实例,涵盖了传统网络中的路由、防火墙、负载均衡等核心功能,并具备根据流量需求动态调整的能力,从而提高网络的灵活性和适应性。MANO则负责整个NFV环境的资源协调与管理,主要由NFV编排器(NFV orchestrator,NFVO)、虚拟网络功能管理器(virtualized network function manager,VNFM)和虚拟基础设施管理(virtualized infrastructure manager,VIM)三部分组成,支持VNF的全生命周期管理,确保资源的高效分配与网络功能的灵活调度。此外,在电信运营场景中,运营支撑系统(operation support system,OSS)和业务支撑系统(business support system,BSS)同样是关键组成部分。其中,OSS主要负责服务提供、网络配置及维护,而BSS则侧重于客户管理、账单处理和订单管理。
SFC的概念最初由互联网工程任务组中的(Internet Engineering Task Force,IETF)SFC工作组提出,旨在基于SDN的原理设计一种灵活的SFC架构。其核心理念是将逻辑服务功能链与数据平面的实际流量转发操作分离,并规定如何根据预定义的SFC策略将流量在部署的服务功能(service function,SF)之间进行有序引导[5]。通过控制平面与数据平面分离的设计原则,SFC架构实现了对底层网络拓扑的独立性,从而大大提高了网络管理的灵活性和可扩展性[14]。此外,ETSI工作组致力于将IETF定义的SFC组件整合至NFV架构中,提出了VNF转发图(VNFFG)的概念,用于描述服务功能链及其业务[20]。VNFFG与SFC定义一致,描述了VNF的顺序及路径。图2展示了NFV/SDN架构下的SFC体系结构。
SFC的部署涉及服务层、管理与编排层、控制层和转发层。
1)服务层。负责逻辑上连接各VNF,并由OSS/BSS管理用户请求和策略配置。
2)管理与编排层。以NFVO为核心,负责虚拟资源的生命周期管理及编排,提供统一接口管理SFC的实例化与配置。
3)控制层。以SDN控制器为核心,将相应的SFC策略解析为具体的服务功能路径(service function path,SFP),并依据VNF状态及链路负载情况实时调整路径。
4)转发层。由SFC分类器、SF转发器、SF及SF代理等组件组成,负责数据包分类、路径标识及流量转发,确保数据按既定SFP顺序通过各服务节点。
SFC的部署是一个复杂且多阶段的过程,涵盖了从服务描述到功能放置与调度的全过程。每个阶段都涉及网络资源管理、QoS保障和系统灵活性等方面的挑战。为了实现高效部署与优化,可将SFC部署划分为以下5个主要阶段:服务功能链的描述、构建、放置、调度及重配置,如图3所示。
1)SFC的描述
描述是SFC部署的起始阶段,定义了SFC请求的功能属性和资源需求,包括每个VNF的处理功能、资源需求(CPU、内存、存储等)以及服务质量约束(如时延、带宽等)。
传统的描述方法如网络服务描述语言(Web services description language,WSDL)[21]虽然在静态网络服务中表现良好,但在动态网络中表现出灵活性不足。为解决这一问题,IETF提出了基于(yet another next generation,YANG)数据模型[22]的解决方案。YANG模型凭借其灵活性和层次化配置能力,逐渐成为主流方案[22]。例如,文献[23]基于YANG模型提出了上下文树语法,通过定义弹性服务描述框架,显著提升了SFC在分布式环境中的可操作性。此外,云应用拓扑与编排规范(topology and orchestration specification for cloud applications,TOSCA)[24]也系统地定义了服务功能的类型及其相互关系,在ETSI和开放网络自动化平台(open network automation platform,ONAP)中得到了广泛应用。
2)SFC的构建
SFC的构建涉及将多个VNF按照特定顺序组织成完整的服务链。尽管各VNF在功能上相对独立,但它们之间存在严格的处理顺序依赖关系,这使得VNF的排序成为服务链构建的核心问题。为了解决这些依赖关系,研究者们提出了多种有效的服务链组合算法。文献[23]设计了一种基于贪婪策略的算法,通过最小化数据流量来优化VNF的连接顺序。
此外,SFC的构建还涉及并行服务功能链的设计,多个服务功能可以并行处理数据流,从而提升系统吞吐量和响应速度。文献[25]通过分析VNF间的依赖关系,提出了将可并行的VNF部署在同一服务器上的方法以降低开销。考虑到服务器资源的有限性,且并行处理可能引发额外开销及数据包沉积等问题。相较于构建完全并行或串行服务链,文献[26]提出了PPC算法,该算法能够在确保性能的前提下,达成合理的部分并行化部署,为服务链构建中的并行优化提供了更具适应性与有效性的解决方案。
3)SFC的放置
作为SFC部署过程中的关键环节,SFC放置问题主要解决VNF的资源分配和网络部署。该问题可分为VNF节点放置和虚拟链路放置2个子问题[17]。节点放置需要在物理基础设施中为每个VNF选择合适的服务器,而虚拟链路放置则负责在物理基础设施中构建连接这些VNF的网络路径。由于涉及多维资源约束和性能目标,VNF放置通常被证明是NP难问题[27]。
从优化目标来看,现有研究集中在成本、时延、可靠性及能效优化等方面[17]。从算法来看,VNF放置问题的求解经历了从确定性算法到智能优化方法的发展历程。传统求解方法主要包括数学规划、启发式算法和元启发式算法3类。在数学规划方面,文献[28]将VNF放置建模为混合整数线性规划(mixed integer linear programming,MILP)问题,通过引入二进制变量表示VNF与物理节点的映射关系,并设计线性约束描述资源限制和QoS需求。文献[29]提出了一种区分等级的可生存服务功能链映射方法,将可生存SFC映射建模为MILP问题,并分别为关键SFC和普通SFC设计了主备路径构建算法与失效重映射算法。虽然MILP方法能获得最优解,但计算复杂度随网络规模呈指数增长,难以应用于大规模网络。为降低计算复杂度,研究者提出了多种启发式算法。文献[27]将VNF放置问题近似表述为一个整数线性规划(integer linear programming,ILP)模型,并采用一种启发式算法来解决该问题。面对动态网络环境带来的新挑战,研究者开始探索基于强化学习的智能优化方法。通过建立环境状态、动作空间和奖励机制,强化学习能够自适应学习部署策略,为VNF放置问题提供了新的解决思路。
4)SFC的调度
SFC调度阶段主要解决VNF的执行顺序和资源分配问题,其核心目标是在满足QoS要求的前提下,最大化系统吞吐量或最小化服务完成时间。文献[30]将VNF调度问题形式化为车间作业调度模型JSP(job shop problem),为后续研究奠定了理论基础。为降低服务完成时间从而使运营商能够服务更多客户,文献[31]将SFC调度与流量路由问题联合构建为MILP模型,并且提出了一种基于遗传算法的启发式算法以获取局部最优解。
5)SFC的重配置
SFC重配置通过动态调整已部署的服务链,应对网络环境的变化与业务需求的动态调整[32-33]。不同于静态部署,重配置需在服务连续性保障的前提下优化资源利用率与服务性能[34]。重配置的触发条件通常包括:①网络负载变化(如流量激增或资源瓶颈)[32];②基础设施状态改变(如节点故障或拓扑更新);③服务需求调整(如QoS要求变化或业务优先级调整)[33-34]。
从本质上看,SFC部署问题是一个多阶段、多目标、多约束的资源优化问题,其复杂性主要源于多个方面:物理网络中计算、存储、网络资源的异构性与VNF多样化需求构成了多维约束条件;网络状态和服务需求的动态变化与不确定性使静态优化方法难以适应实际环境;资源利用率、能耗效率、服务质量和运营成本等多个优化目标之间存在冲突关系需要权衡;网络规模和服务链数量增长导致的决策空间组合爆炸使穷举搜索变得不切实际;此外,SFC部署各阶段之间的相互影响与密切关联形成了阶段耦合性,前一阶段的决策直接影响后续阶段的优化空间,因此理想的优化策略应当考虑全局最优而非各阶段的局部最优。
面对SFC部署各阶段的挑战,传统优化方法和智能优化方法各有优劣。整数线性规划和混合整数线性规划等精确求解方法能够提供全局最优解,保证解的质量,但其计算复杂度极高(通常为NP难),随着网络规模的增长呈指数级上升,导致其在大规模网络中求解时间过长,难以满足实时部署需求[28]。此外,这类方法对问题模型的依赖性强,当网络环境发生变化时,需要重新构建模型并求解,缺乏灵活性。相较之下,启发式和元启发式算法(如贪婪算法、遗传算法、模拟退火等)通过特定的搜索策略,能够在可接受的时间内找到近似最优解,计算复杂度较低,适用于中等规模的网络环境[27,31]。这类方法在求解效率和解的质量之间取得了平衡,但往往依赖于人为设计的启发规则,在面对复杂多变的网络环境时,这些固定规则可能难以适应网络的动态变化,缺乏自主学习和优化能力。
近年来,强化学习等智能优化方法逐渐应用于SFC部署问题。强化学习方法之所以适用于SFC部署问题,源于其与问题本质的匹配性。SFC部署本质上是一个序列决策问题,每个VNF的部署位置和路径选择都会影响后续决策空间,符合强化学习的马尔可夫决策过程(Markov decision process,MDP)框架[35],强化学习通过智能体与环境交互,学习长期最优的决策策略,能够捕获决策间的依赖关系[7]。同时,SFC部署面临高维状态空间(包括网络拓扑、资源分布、流量需求等)和深度强化学习结合深度神经网络的表征能力,能够有效学习复杂的状态-动作映射,无需显式建模每个状态转移概率。此外,网络环境的动态变化可自然映射为强化学习框架中的环境变化,强化学习智能体通过持续交互和奖励反馈,能够自适应调整策略,适应环境动态性。SFC部署通常需要平衡多个相互冲突的目标,强化学习通过奖励函数的设计可以隐式处理多目标优化问题,学习在各目标间取得平衡的策略。表2总结了不同优化方法在计算复杂度、求解质量、适应性等方面的对比。
从表2可知,强化学习在应用于SFC部署时也面临挑战:训练过程需要大量环境交互,模拟器的构建和参数调优复杂,奖励稀疏问题可能导致学习效率低下。
深度强化学习已在自然语言处理、机器人控制、决策博弈等领域展现出显著价值[36],AlphaGo的成功应用[37]更是彰显了其卓越效能。强化学习的核心在于控制策略的学习与优化,通过智能体与环境的持续交互,实现累积回报最大化[8]。马尔可夫决策过程作为强化学习的理论基础,为智能体的交互机制和策略优化提供了严格的数学支撑,使得复杂的SFC部署决策问题能够在该框架下得到有效表征和求解。
强化学习算法根据其核心机制可分为基于值函数的方法、基于策略的方法和基于值与策略的方法等类别[38-39]。基于值函数的方法如Q-Learning、深度Q网络(deep Q-network,DQN)、双深度Q网络(double deep Q-network,Double DQN)等,通过估计状态-动作对的价值来选择最优动作,适合处理离散动作空间,但在连续动作空间中面临扩展性挑战。基于策略的方法如REINFORCE、策略梯度(policy gradient,PG)、信任域策略优化(trust region policy optimization,TRPO)等直接优化策略函数,可以有效处理连续动作空间,具有较好的收敛性能。基于值和策略的方法如AC(actor-critic)、深度确定性策略梯度(deep deterministic policy gradient,DDPG)、近端策略优化(proximal policy optimization,PPO)等,结合了基于值函数的方法和基于策略的方法的优势,通过Actor网络生成动作以及通过Critic网络评估动作价值,在复杂任务中表现出色。近年来,随着深度学习技术的发展,多智能体方法如多智能体深度确定性策略梯度(multi-agent deep deterministic policy gradient,MADDPG)、多智能体近端策略优化(multi-agent proximal policy optimization,MAPPO)进一步扩展了强化学习在复杂系统中的应用场景。表3详细对比了各类强化学习算法的特点及优劣势。
图4展示了针对SFC部署问题的MDP建模方法。为应对请求随机到达与离开的网络动态变化,引入时隙的概念,在每个时隙内按既定流程顺序执行SFC操作。具体而言,状态通过多维向量形式全面表示网络在各时隙下的运行状况,确保了对实际环境的精确描述;动作定义则明确了VNF的部署策略;奖励函数的设计同时考虑了运营商资源效率和客户QoS需求,实现了多目标优化;而状态转移则刻画了网络在不同时隙间的动态演变过程,为制定更加高效的部署策略提供了依据。这一模型框架有效地为SFC部署提供了形式化表述。
基于上述MDP建模,可进一步构建基于强化学习的SFC部署框架,如图5所示。该框架的核心在于智能体与环境的交互机制,通过不断循环迭代实现部署策略的动态优化。环境包括网络拓扑、实时流量状态及资源可用性等动态要素,可作为智能体感知和决策的依据。智能体在每个时隙内根据当前环境状态选择部署动作,即为VNF制定具体部署方案,这一决策直接影响系统性能。环境则通过奖励信号量化部署方案的效果,反馈给智能体用于优化其策略。智能体依据累计奖励不断调整行为,从而实现网络资源利用率与QoS之间的优化平衡。
通过上述交互过程,基于强化学习的SFC部署框架能够适应复杂网络环境的动态变化,有效提升SFC的部署效率与性能。该框架为应对多目标优化和资源分配问题提供了新思路[7]。本文后续章节将重点探讨强化学习技术在SFC放置、调度与重配置中的具体应用,系统分析相关研究进展,并展望未来发展趋势。
基于强化学习的SFC放置问题研究经历了从简单到复杂、从单一到协同的发展历程。早期研究主要采用传统强化学习方法解决小规模场景下的放置优化问题。随着深度学习技术的发展,研究者逐渐将深度强化学习引入到SFC放置决策中,以应对大规模复杂场景。近年来,图神经网络和多智能体技术的引入进一步提升了放置决策的效率和准确性。本节将从技术演进的角度详细阐述基于强化学习的SFC放置研究进展。
传统强化学习通过智能体与环境交互来学习最优策略,其简单有效的特点使其成为早期解决SFC放置问题的重要方法。
文献[40]提出了一种基于Q-Learning的动态SFC放置方案。该方法通过构建状态空间,描述网络服务功能的执行时间、节点资源使用率、服务链时延和链路带宽等信息;同时设计奖励函数以平衡资源利用与负载均衡目标。智能体通过不断地试错学习,选择资源利用率低的节点来构建服务路径,从而实现节点负载均衡和资源高效使用。然而,该方案在大规模网络场景下由于状态空间的急剧膨胀,容易出现维数灾难问题,导致学习收敛速度下降。
为缓解上述问题,文献[41]提出了一种结合强化学习和启发式算法的混合解决方案——QLFHM算法。该算法将强化学习模块用于输出满足特定要求的最短路径,随后通过负载均衡模块在这些路径中挑选最优解,进一步提高了放置策略的效率与性能。实验结果表明,该算法在动态环境下的放置成功率高于传统启发式方法,并能有效降低计算时间。
尽管这些研究充分展示了传统强化学习在SFC放置问题中的潜力,但其局限性也较为显著。Q-Learning等值函数型算法依赖于离散化的状态和动作空间,在处理大规模连续状态空间时效率较低[42]。随着网络规模和复杂度的增长,传统强化学习方法的局限性愈加凸显。
随着深度学习技术的发展,强化学习与深度学习的结合,即深度强化学习,在处理复杂决策问题方面展现出了显著优势。深度强化学习不仅能够有效解决传统强化学习在高维状态空间中面临的维数灾难问题,还能通过神经网络的强大特征提取能力,直接从原始数据中学习出有效的放置策略。近年来,基于深度强化学习的SFC放置方法取得了广泛关注。
1)基于DQN的方法
深度Q网络是深度强化学习的经典方法之一,其通过深度神经网络逼近Q值函数,实现了从状态到动作价值的端到端学习。文献[7]将深度强化学习引入SFC放置领域,提出了NFVdeep方法。该方法将SFC放置问题建模为MDP,在状态空间设计上综合考虑网络资源利用率、当前SFC部署状态及VNF特征,构建了完整的网络状态向量。在动作空间定义方面,采用整数编码表示VNF放置策略。奖励函数设计则通过加权计算接受请求收益与资源开销的差值,有效平衡了服务提供商和用户的利益。基于此MDP框架,NFVdeep采用序列化-回溯方法处理离散动作空间,并利用策略梯度法优化训练效率与收敛性。实验表明,该方法在请求吞吐量和运营成本方面均优于传统基准算法。
文献[43]从QoS保障角度出发,将SFC放置建模为参数化动作马尔可夫决策过程,提出了基于策略梯度的DRL-QOR方法。该方法以最大化用户体验质量为目标,同时满足QoS约束,采用深度神经网络近似Q函数。不同于逐步放置方式(每一步仅放置1个VNF),DRL-QOR采用单步放置策略,在一个决策步骤内生成所有VNF的放置方案。
文献[44]关注NFV环境下在线服务供应面临的资源分配挑战,提出了一种基于DQN的自适应资源分配算法。与NFVdeep不同,该方法考虑了服务生命周期的概率特性,通过随机过程建模捕捉服务请求的动态性。文献[45]针对具有优先级约束的SFC放置问题,提出了基于DQN的自适应映射方法ADAP。该方法首先将多个具有优先级约束的SFC放置问题形式化为多步骤整数线性规划问题,然后设计了2种基于贪婪策略的低复杂度算法,并利用DQN结合ε-贪婪策略实现自适应调度。
文献[46]针对边缘云环境中的服务功能链部署挑战,提出了SDRM-EC-DPR算法,旨在最大化在线SFC部署的收益。与前述工作不同,该研究在边缘云融合环境下全面考虑了排队延迟、计算延迟和通信延迟的综合影响,并基于市场供求规律构建了统一的部署成本模型,消除了边缘云环境中设备的异构性。在算法设计上,SDRM-EC-DPR基于对决型双深度Q网络(dueling double deep Q-network,D3QN)框架,进一步融合了优先经验回放机制和随机网络蒸馏技术,提升了学习效率并避免了局部最优陷阱。
2)改进的DRL算法
随着研究的深入,学者们开始关注深度强化学习框架的创新和改进。针对大规模动作空间引发的维度灾难问题,文献[47]提出了具有增强探索能力的DDPG算法。该算法通过引入启发式算法来求解可行解,从而有效确保动作的可行性并提升算法效率。文献[48]则提出一种基于改进的DDGP算法,基于价值差异的探索方法对贪婪策略进行扩展,以此来平衡智能体在动作选取时探索与利用之间的关系。同时,该算法设计了双重经验回放池机制,从而提高训练效率和收敛速度。此外,文献[49-50]在VNF放置和路由问题研究中引入在图像处理和机器翻译中广泛应用的注意力机制。通过将注意力机制应用于DDPG框架中的智能体动作建模,使模型能够自动关注对智能体决策具有重要影响的邻居节点,有效降低了对不相关节点的计算开销,显著提升了模型训练效率。实验结果表明,与传统DQN方法相比,基于注意力机制的方法在网络效用、时延控制、成本优化和请求接受率等多个性能指标上均取得了显著提升。文献[51]提出基于增强E-DDPG算法通过重构缓冲重放机制(包括多样性缓冲重放和高误差缓冲重放),显著提升了智能体的训练效率。文献[52]针对边缘智能赋能的工业物联网中动态SFC编排问题,提出了基于SAC(soft actor-critic)的深度强化学习方法。该方法充分考虑了工业物联网服务请求的时延约束、时变特性和异构性,通过优化SAC算法提升了模型的稳定性和收敛性。
3)可靠性导向的SFC放置方法
可靠性和容错性作为优化目标的SFC放置问题逐渐受到研究者的关注,主要聚焦于SFC放置过程中的故障容错、服务可用性保障和系统稳定性维持。文献[53]针对在线容错SFC放置问题,提出了基于双深度Q网络的DDQP算法和DDQP+算法,分别负责决策运行时和备用服务实例的放置位置。为了提供全面的容错放置解决方案,研究者同时提出了5种渐进的资源预留方案,从零预留到全量预留构建了完整的容错策略谱系。文献[54]提出了一种基于DRL的优先级感知算法,通过计算SFC和网络节点的优先级来确定备份方案,利用深度强化学习算法选择VNF的放置和备份位置,提升了系统的可靠性和服务质量保障能力。文献[55-56]针对SFC可用性、运营成本和能耗等因素的SFC放置问题,提出一种基于强化学习的方法,通过优化节点选择和冗余策略,有效平衡了服务可用性与资源利用效率。
4)多目标优化SFC放置方法
SFC放置通常需要同时考虑多个优化目标,如时延、资源利用率、能耗等。基于深度强化学习的多目标优化方法试图在这些冲突的目标之间寻找帕累托最优解。研究者们提出了多种策略来处理多目标问题,包括基于权重的方法[57]、基于帕累托前沿[58]的方法等。文献[59]提出了一种切比雪夫辅助的Actor-Critic SFC放置算法,创新性地将非线性切比雪夫函数与Actor-Critic算法相结合以探索帕累托前沿。该方法首先将单目标混合整数线性规划模型扩展为多目标模型;其次设计了基于指针网络(pointer network,PN)的DRL模型来处理大规模部署问题。该算法无需人为干预即可逼近帕累托前沿,通过选择不同权重来改善边缘计算资源使用和服务接受率。仿真结果表明,该算法在生成非支配前沿方面表现卓越,可有效规避边缘资源拥塞,在不同负载场景下实现高服务接受率。
5)并行SFC放置方法
针对串行SFC时延性能和处理效率方面的局限性,研究者们提出了基于强化学习的并行SFC放置方法。文献[60]针对移动边缘计算中SFC放置问题,提出了一种基于深度强化学习的并行化VNF动态SFC放置算法DSPPV。该算法使用A3C(asynchronous advantage actor-critic)算法作为DRL框架来解决分布式网络中的并行SFC放置问题。DSPPV还包含一种提取初始化VNF分布的策略,通过配置VNF队列网络,预测A3C中智能体最合适的动作,从而提高未来请求的接受率。文献[61]为解决数据中心网络中异构带宽需求下的SFC放置问题,提出了基于DRL的SPC3D算法。该算法将原始SFC划分为多个子SFC,通过分析VNF间的依赖关系实现SFC的并行化处理,有效降低了服务时延。实验结果表明,与传统方法相比,SPC3D在处理大规模异构带宽需求时表现出更好的性能和稳定性。
基于深度强化学习的SFC放置方法通过深度神经网络的强大表征能力,成功解决了传统方法在处理高维状态空间和复杂决策时的局限性。从最初的DQN到改进的DDPG,SAC等算法,以及在多目标优化、可靠性保障、时延优化、资源优化和并行放置等方面的深入研究,深度强化学习展现出显著的优势。特别是在处理动态网络环境、异构资源需求和多维度优化目标等复杂场景时,基于深度强化学习的方法能够自适应地学习最优放置策略,在性能、效率和适应性等方面均优于传统方法。然而,这类方法仍存在训练开销大、可解释性不足等问题,这也促使研究者探索将其与其他先进技术相结合的新方向。
SFC部署问题通常涉及复杂的网络拓扑结构和多维资源约束,如何高效地提取网络拓扑信息以指导部署决策是关键难题之一。图神经网络(GNN)因其在处理图结构数据中的独特优势,越来越多地被集成到强化学习框架中,用于优化复杂网络环境下的SFC部署问题。GNN通过节点嵌入和边特征聚合的方式,能够充分捕获网络节点和链路间的关联性,生成具有全局上下文信息的网络表示。
文献[9]针对传统基于深度神经网络的SFC放置方法在利用网络拓扑信息方面存在的不足,提出了一种基于编码器-解码器架构的GNN模型。该模型的编码器通过结合门控循环单元(gated recurrent unit,GRU)的GNN对网络拓扑进行特征提取,解码器则基于编码得到的网络表征逐步选择放置节点并评估VNF的处理概率,从而生成完整的SFC部署路径。文献[62]在解决VNF放置问题时,创新性地将深度强化学习与GNN相结合,提出了DeepOpt方案。该方案在GNN结构设计上采用了图网络(graph network,GN)[63]来实现节点和边缘属性的动态更新与聚合,有效提升了对网络拓扑结构的感知能力。进一步地,文献[64-66]提出的算法不仅将底层物理网络建模为图结构,还将服务请求表示为有向无环图,通过GNN分别对SFC请求和底层网络进行特征提取,实现了请求和底层资源的双重GNN表征。
综上所述,基于GNN的方法在SFC部署问题中展现了强大的能力,不仅能够有效捕获复杂网络拓扑的全局信息,还能够通过与强化学习的深度融合,进一步优化部署策略。
多智能体强化学习方法因其能够通过多个智能体的协同决策解决大规模复杂问题,成为SFC部署研究的一个重要方向。在多智能体场景中,每个智能体可以独立感知环境、执行动作,并通过相互协作完成复杂任务。与单智能体方法相比,多智能体方法不仅能够更高效地处理分布式网络环境中的部署任务,还能更好地应对大规模网络中资源分配的动态性和异构性。本节从智能体设计和协同机制角度探讨多智能体强化学习在SFC放置中的典型应用与研究进展。
1)基于SFC请求分配多智能体的方法
在网络中,SFC的放置往往涉及多个请求,而不同请求的QoS需求和资源约束各异。基于请求分配的多智能体方法通常采用“1个智能体对应1个SFC”的设计模式,每个智能体负责处理单个SFC的放置和优化任务。这种设计可以有效地实现请求级别的分布式决策,提高放置效率和系统灵活性。
文献[10]提出了一种针对物联网中资源分配问题的多智能体强化学习方法。该方法中,每个智能体管理一个SFC请求,并通过2阶段决策优化SFC的放置和资源分配。在第1阶段,智能体集中选择需要服务的物联网设备;在第2阶段,通过多智能体协作优化虚拟服务请求的映射和资源分配。文献[67]利用马尔可夫博弈建模多用户网络资源竞争问题,并通过MADDPG算法求解竞争的纳什均衡,从而优化了SFC的资源分配和放置策略。该方法在保证QoS的同时,均衡了多用户之间的资源竞争。此外,文献[68]提出了一个2阶段基于GCN的深度强化学习框架,用于解决多数据中心网络中的SFC嵌入问题。第1阶段将负载转移过程建模为MDP,通过基于GCN的PPO算法实现SFC请求的负载均衡;第2阶段利用多智能体框架,采用基于GCN的多智能体MAPPO算法进行精细化放置优化。该框架通过2阶段协同优化提升了放置效率和资源利用率。
2)基于节点协同的分布式放置方法
考虑到网络基础设施的分布式特性,基于节点协同的多智能体方法在每个网络节点分配独立的智能体,通过智能体间的信息交互和协同决策完成SFC放置。这种方法充分利用了分布式系统的优势,智能体通过消息传递机制共享局部观察信息和决策意图,实现资源分配和路径规划的全局优化。
文献[69]针对成本和时延敏感的SFC放置问题,提出了多智能体深度强化学习算法MADRL-P&R。该算法将SFC放置问题分解为2个子任务,通过设计联合奖励和内部奖励机制实现任务间的协同优化。同时,引入参数迁移的模型重训练方法,有效提升了算法对网络拓扑变化的适应能力。文献[70]聚焦于多域算力网络中的SFC映射问题,提出了基于数字孪生辅助的MADDPG框架。该框架将SFC映射建模为长期优化问题,综合考虑计算能力、链路带宽、时延性能和VNF排序等多维约束,以降低能耗、端到端时延和VNF重新放置成本为优化目标。通过引入数字孪生技术,该方法提升了决策的准确性和实时性。
3)面向多目标优化的协同决策方法
面向多目标优化的多智能体方法采用“1个智能体对应1个优化目标”的范式,通过多个智能体的协同决策来实现多目标之间的动态平衡。
文献[71]针对现有研究多集中于单目标或静态场景优化,且在动态多目标优化方面存在不足的问题,提出一种基于GCN的多智能体深度强化学习方法。该方法构建了包含物理网络拓扑特征提取的GCN神经网络和Actor-Critic模型,通过多智能体强化学习框架,使不同智能体对应不同奖励函数,实现对动态环境变化的感知与SFC放置策略的调整。实验验证表明,该方法在动态网络环境下能够提升放置性能,并实现多目标之间的有效平衡。
多智能体协同放置方法的研究表明,通过合理的智能体划分和协同机制设计,可以有效提升SFC放置的决策效率和优化效果。然而,当前研究仍存在智能体通信开销大、训练收敛性不稳定等问题。
为了系统地总结和对比基于强化学习的SFC放置方法,表4从强化学习算法、优化目标、VNF放置策略以及虚拟链路放置策略等维度进行了详细归纳。
从优化目标来看,早期研究主要关注单一目标优化,如运行成本或时延;随着研究深入,越来越多的工作开始考虑多目标优化问题,如资源利用率、负载均衡、能耗等。在放置策略方面,不同方法根据具体场景和优化目标采用了节点序号、路由路径等多样化的策略选择方式。
SFC调度问题主要关注在已知网络资源和SFC请求的情况下,如何为每个SFC制定合理的资源分配方案和执行顺序,以达到系统性能的最优。与SFC放置问题相比,调度问题更加强调服务执行的时序性和资源的动态分配。特别是在现代网络架构中,随着B5G/6G、边缘计算等技术的快速发展,对服务响应时间的要求愈发严格,这使得时延优化成为SFC调度研究的核心目标之一。本节从优先级机制角度对基于强化学习的SFC调度研究进行分类讨论,主要分为基于任务特性的优先级和基于服务需求的优先级两大类。
基于任务特性的优先级机制主要关注SFC或VNF本身的属性和特征,以任务的固有特性作为确定优先级的依据。这类方法主要聚焦于如何通过合理的调度顺序优化系统整体性能指标,如总完成时间、资源利用率或系统吞吐量等。
在移动边缘计算场景中,文献[72]将SFC调度问题建模为灵活作业车间调度问题,并提出了基于DQN的调度算法,以最小化总体调度时延为优化目标。该研究首先根据SFC中未处理的VNF数量为SFC分配优先级,具有更多VNF请求的SFC获得更高的处理优先级。其次,在动作匹配过程中也考虑了优先级因素,当多个动作满足条件时,会选择优先级更高的动作执行。这种多层次的优先级设计有效平衡了资源利用率与调度时延。
针对多资源约束下的SFC调度问题,文献[73]首先证明了该问题的NP难特性,随后设计了基于DRL的求解框架。在该框架中,智能体通过感知服务器资源使用状况和SFC状态来做出调度决策,选择最适合的服务器与SFC进行匹配。其奖励函数设计与未完成SFC数量建立关联,引导智能体朝着最小化平均流完成时间的方向优化。该研究在资源调度时允许暂停当前VNF的执行,优先处理可能带来更高收益的VNF,实现了动态优先级调度。其次,将最短作业优先和最小资源优先等优先级启发式集成到DRL训练过程中,显著提高了训练效率。
文献[74]提出基于DRL的VNF协同调度框架,用于解决传统SFC批处理放置和调度方法在处理多样化和优先级任务时的不足。该框架创新性地提出了基于最短剩余时间优先规则的优先级处理机制,通过考虑VNF的排队等待时延和服务完成时延计算其优先级,并以最小化带优先级权重的平均时延为优化目标。同时,该文设计了基于优先级的多维资源分配策略,根据VNF在队列中的优先级位置分配计算资源比例,确保高优先级VNF获得更多资源和更少的排队时延。
针对空天地一体化网络中的动态SFC调度问题,文献[75]提出了一种基于DRL的协同优化方法。该方法首先引入可重构时间扩展图来刻画空天地一体化网络的多层资源分布,并设计了一种基于数据量的SFC优先级排序机制。同时,其提出的VNF状态转换算法能够在资源冲突时智能调整处理顺序,优先处理关键VNF以确保多个SFC同时满足时延约束。
基于服务需求的优先级机制主要关注服务请求的差异化需求、服务质量要求和业务重要性,以服务或用户的外部需求为确定优先级的依据。这类方法通常考虑不同服务之间的差异,如时延敏感度、吞吐量需求、可靠性要求等,以及不同用户群体的服务等级协议(service level agreement,SLA),旨在满足多样化的服务质量需求并实现资源的公平分配。
对于具有端到端时延要求的VNF调度问题,文献[76]将其转化为MDP并提出强化学习调度算法。该算法在系统状态设计上全面考虑了NFV节点资源状态和VNF执行状态通过奖励函数中权重系数的设计实现了服务优先级的差异化处理。具体而言,为高优先级(时延敏感)服务分配较大的权重系数,为低优先级(时延不敏感)服务分配较小甚至为零的权重系数,从而实现对不同优先级服务的差异化调度。
针对工业物联网场景中基于信息年龄(age of information,AoI)的VNF调度问题,文献[77]提出了一种复合A2C深度强化学习框架。该框架同时考虑了VNF的放置位置选择和执行顺序调度,通过设计多维动作空间,实现了放置成本、调度成本和平均AoI的联合优化。该研究引入了基于AoI的优先级调度机制,将具有较高AoI值的服务请求赋予更高的处理优先级,确保信息的实时性和新鲜度。同时,将到达的服务请求分为当前到达服务、当前终止服务和活动服务3类,针对不同类别采取差异化的优先级处理策略。
针对边缘计算中VNF放置和调度的公平性问题,文献[78]提出基于PPO的协同优化算法。考虑到资源限制和任务多样性,该算法旨在确保不同服务链之间的资源分配公平,同时满足各自的端到端时延要求。该研究通过学习优先级策略函数,为候选的映射和调度决策分配优先级值,使得资源竞争激烈时能够公平地处理不同服务链的需求。该方法将多层感知器作为优先级分配规则的函数实现,动态地根据服务链的状态确定VNF的映射和调度优先级。
针对医疗急救场景中的服务功能链调度问题,文献[79]提出了一种基于业务紧急程度和服务区域的优先级分配机制。该研究将医疗业务分为远程紧急、本地紧急、远程普通和本地普通4类,分别对其赋予不同权重值,并以最小化总加权完成时间为优化目标建立服务功能链调度模型。与最小化最大完成时间目标不同,该模型更关注高优先级服务的及时完成。该研究针对不同问题规模,设计了匹配博弈算法和Q-Learning强化学习算法,通过仿真实验验证了方法的有效性。
文献[80]针对车联网(Internet of vehicles,IoV)中边缘云服务功能调度问题,提出了一种基于多智能体强化学习的分层动态调度算法。该算法创新性地考虑了车辆节点移动导致的无线信号强度动态变化,并在问题建模中明确设定了服务请求优先级约束。算法通过图注意力网络提取特征,实现了服务请求选择和执行车辆选择的解耦,在保证公平调度的同时,为移动场景下的SFC调度提供了新的解决思路。
针对网络切片中VNF调度的时延问题,文献[81]提出基于DRL的低时延VNF调度算法。该研究设计了一个基于复合规则的D3QN调度模型,引入了多种SFC优先级选择策略,包括基于截止期限、松弛时间、剩余处理时间与操作数比值、估计平均处理时间等多种优先级规则。在每个调度决策点,D3QN算法能够根据当前网络状态自适应地选择最优的优先级规则,而不是固定使用单一规则。实验结果表明,D3QN调度算法在不同网络环境下均能保持优越性能。
尽管目前基于强化学习的VNF调度研究相对新颖、文献数量有限,但已展现出良好的发展潜力和应用前景。表5总结了近年来基于强化学习的VNF调度算法特征。
从表5中可以看出,当前研究主要集中在5个方面:算法选择上,从早期的Q-Learning发展到DQN及其变体(如D3QN,Double DQN),再到策略梯度类算法(如PPO,A2C),显示出强化学习算法在该领域的快速演进;优化目标方面,时延(包括传输、传播、处理、排队时延)和完成时间是最主要的关注点,部分研究也考虑了资源利用率、接受率等多目标优化;调度动作设计上,主要表现为2种模式:一是直接选择VNF的部署节点序号,二是选择特定的调度算法或参数。应用场景则从传统数据中心扩展到边缘计算、车联网、工业物联网和空天地一体化网络,涵盖了多种新型网络场景,证明了强化学习在处理复杂约束条件下VNF调度问题的广泛适用性。
网络环境的动态性和复杂性对SFC的部署提出了更高要求。为适应网络流量波动、用户移动性变化和节点故障等情况,SFC重配置成为保障服务质量的关键技术。基于强化学习的SFC重配置方法凭借其自适应学习和动态决策能力,为解决这一问题提供了新思路。本节将从VNF迁移、动态预测重配置、可靠性导向重配置等维度系统分析基于强化学习的SFC重配置研究进展。
VNF迁移是实现SFC重配置的核心机制之一,其主要目标是通过将VNF实例从资源受限或性能下降的节点迁移至更合适的位置,以维持服务质量并优化资源利用。然而,VNF迁移过程涉及多个挑战:1)需要准确评估网络状态和服务质量变化趋势确定迁移时机;2)目标节点的选择需要综合考虑计算资源、网络带宽和服务时延等多维约束;3)迁移过程的执行需要最小化服务中断时间和资源开销。针对这些挑战,研究者们提出了多种基于强化学习的创新解决方案。
文献[82]面向5G网络切片架构,提出了一种基于深度强化学习的VNF智能迁移框架。该框架将VNF迁移问题建模为受限MDP,以最小化通用服务器平均能耗为优化目标,同时将各切片的平均时延、缓存及带宽资源消耗作为约束条件。通过设计基于卷积神经网络(convolutional neural network,CNN)近似行为值函数,能够自适应学习最优的VNF迁移策略和CPU资源分配方案。实验结果表明,该方法在满足切片服务质量需求的同时,降低了基础设施能耗,实现了资源利用率的有效提升。
针对数据中心网络中SFC的动态调整问题,文献[83]考虑到网络请求波动导致初始部署无法满足用户资源需求的情况,提出了一种智能服务功能链动态调整算法。该算法首先构建了以最小化长期调整成本和资源分布不平衡性为目标的整数线性规划模型,并将其转化为MDP。通过采用Double DQN架构,算法能够根据网络状态和资源利用情况,自适应决策VNF实例是否需要弹性伸缩或迁移。
针对SDN/NFV环境下动态SFC部署和重调整问题,文献[84]提出了基于双深度Q网络的DDQN-VNFPA算法。该算法将VNF放置问题建模为二进制整数规划问题,通过最小化包含VNF放置成本、运行成本和拒绝惩罚的加权成本函数进行优化。DDQN-VNFPA通过离线训练与在线运行相结合的方式,能够根据网络状态和预测的SFC请求动态优化VNF放置策略。文献[85]提出基于DQN和M最短路径算法的动态SFC部署和重调整方法。该方法采用双智能体协同工作机制,一个负责动态部署SFC,另一个根据当前网络条件和SFC请求信息通过VNF迁移对底层网络进行重调整,以提高服务请求接受率。
针对卫星网络中SFC迁移面临的资源受限、网络拓扑动态变化和传输时延等挑战,文献[86]提出了一种基于深度强化学习的时延敏感型SFC迁移算法。该方案主要从3个方面进行优化:首先,在目标节点选择过程中引入资源评估机制,优先选择资源利用率较低的节点;其次,设计自适应机制来感知卫星轨道和天气条件变化,实现迁移策略的动态调整,以应对网络环境的动态性;最后,针对实时性要求高的应用场景,将时延最小化作为SFC迁移算法的优化目标,通过优化路径选择来降低时延。该研究通过上述措施提高了SFC迁移算法在卫星通信环境下的适应性、智能性和性能表现,为卫星网络中的服务提供了更可靠和高效的支持。
在移动边缘计算和B5G/6G网络环境下,用户移动性导致的服务质量波动是SFC面临的重要挑战。移动性感知的VNF迁移通过预测用户移动轨迹、评估网络状态并进行及时迁移决策,以维持服务质量并降低资源开销。文献[87]针对用户移动场景下的SFC迁移时机决策问题,提出基于深度Q网络的SFC迁移时机决策算法。考虑到用户移动路径已知且到达时间可预测的情况,该研究建立了用户到达时间预测模型,将SFC迁移过程建模为马尔可夫决策过程,通过智能体感知网络资源状态变化和SFC迁移信息,选择合适的迁移时机,以降低服务中断时间并提高迁移成功率。文献[88]针对多接入边缘计算环境中用户移动性导致的SFC动态迁移问题,提出了DRL的SFC自适应动态迁移优化算法。该算法利用DQN感知网络资源状态和迁移需求,通过对多个网络功能迁移路径的智能决策和带宽自适应分配,实现了SFC的无缝迁移。
网络环境的动态性和不确定性对SFC重配置提出了严峻挑战。通过准确预测网络状态和资源需求变化,并据此及时调整部署策略,成为提升系统性能的重要研究方向。
文献[89]针对物联网场景下SFC动态重配置问题,提出了一种基于深度Dyna-Q的解决方案。该方案重点关注边缘云环境下,由于物联网业务流量动态变化导致的VNF实例资源需求波动问题。不同于传统静态部署机制,该方案能够在运行时动态调整SFC模式,通过预测VNF资源需求来优化VNF实例迁移和服务路径重路由配置。文献[90]提出了一种SFC主动重配置机制SARM算法来解决网络状态动态变化引起的服务质量下降问题。该方法将计算负载时间序列分解为长期趋势和短期波动2部分,分别采用长短期记忆网络(long short-term memory,LSTM)和DQN进行预测;同时假设SFC行为具有马尔可夫性质,利用隐马尔可夫模型(hidden Markov model,HMM)建模来预测未来资源需求。在此基础上,SARM综合考虑端到端时延和重配置成本等多个因素,通过启发式算法平衡迁移和带宽成本,实现SFC的动态重配置。
可靠性导向的重配置研究主要从故障容错、服务质量保障和优先级感知等多维度展开,旨在提升系统的鲁棒性和服务连续性。
针对网络拥塞和节点故障等导致的服务中断问题,研究者提出了多种基于优先级的容错方案。文献[91]提出了优先级感知的VNF迁移方法PAVM。该方法根据业务优先级设计差异化的迁移策略,通过深度强化学习优化目标节点选择,有效平衡了系统可靠性与资源利用效率。为进一步提升系统对动态负载的适应能力,文献[92]提出了一种基于DRL的自动缩放和优先级感知的SFC部署算法APSD。该算法根据不同网络服务的QoS需求进行优先级划分,并基于VNF的弹性特征采用混合缩放策略,实现了对服务请求和工作负载动态变化的智能响应。
为提升系统的整体可靠性,研究者探索了基于分布式架构的故障恢复方案。文献[93]在SDN/NFV云环境中提出了面向容错的SFC优化方案,将容错导向的SFC弹性优化问题建模为MDP。该方案通过设计包含能耗最小化、迁移成本最小化、收益最大化和负载均衡等因素的综合奖励函数,实现了多目标约束下的优化。特别地,文中提出的多智能体双深度Q网络方法将繁重的训练任务从控制平面转移到数据平面,有效降低了单点故障风险。文献[94]针对边缘网络中SFC迁移的可靠性问题,提出了基于多智能体深度强化学习的可靠性保障SFC迁移算法。该算法通过引入优势函数评估智能体性能,并采用中央注意力机制增强智能体间的协作。同时,基于迁移目标节点和链路容量设计了可靠性惩罚机制,在实时性、资源利用率和可靠性等方面均优于现有方法,为边缘网络中的可靠性保障提供了新思路。
基于以上分析,表6总结了近年来基于强化学习的VNF重配置算法特征。
从表6中可以看出,DQN及其变体是当前VNF重配置领域最主流的强化学习方法,其他先进算法如PPO等尚未在该领域得到充分应用。在优化目标方面,大多数算法都采用多目标优化框架,主要关注时延、迁移成本、能耗和请求接受率等性能指标,反映了VNF重配置问题在保障服务质量与控制资源开销间的平衡需求。从重配置动作的设计特点来看,现有研究呈现出明显的分化趋势:一类算法(如ISFCDAA[83],DDQN-VNFPA[84])侧重于决策层面,关注是否迁移及重配置策略的选择;另一类算法(如LS-SFCM[86],PAVM[91])则专注于执行层面,主要确定具体的迁移节点序号。这种动作空间设计的差异性反映了不同应用场景下对VNF重配置精细化控制的不同需求。
综合上述国内外研究现状,当前基于强化学习的SFC部署研究虽取得了显著进展,但仍面临诸多挑战。
当前基于强化学习的SFC部署算法在泛化能力、训练效率和多任务协同等方面存在明显不足。首先,当前基于强化学习的SFC部署模型过于依赖特定网络拓扑结构,缺乏拓扑泛化能力。在大多数研究中,状态空间设计局限于静态特征提取,模型的设计和训练是在特定网络拓扑下进行的,模型的优化目标和策略都与该拓扑高度耦合。一旦网络拓扑发生变化,模型通常需要重新训练,以适应新的拓扑环境。然而,模型在拓扑泛化能力方面的不足,严重制约了其在实际网络中的应用扩展性,尤其是在动态网络拓扑频繁变化或跨网络环境应用的场景中。未来的研究需要探讨如何提升强化学习模型的拓扑无关性,使得模型能够在不同拓扑结构间迁移,并减少重新训练的成本。
在训练效率和收敛性方面,强化学习算法在处理大规模网络的SFC部署问题时往往面临训练周期长、收敛性差等挑战。特别是在多智能体协同场景下,环境动态从单个智能体的角度表现出非平稳性,这导致传统的经验回放机制难以直接应用。同时,智能体之间的相互影响进一步增加了训练的复杂性。在复杂的部署场景中,有效的奖励信号往往稀疏且延迟,智能体难以准确评估其行为的长期影响,容易导致策略震荡或陷入局部最优。为应对这些挑战,未来研究可从设计针对非平稳环境的自适应经验回放机制、构建分层式的奖励函数和值函数估计器促进稀疏奖励信号向密集反馈的转化、开发稳定的多智能体训练算法方向着手,通过算法创新提升收敛性。
现有研究对SFC部署放置、调度、重配置各阶段缺乏系统性的协同优化机制。具体表现为在问题建模中对各阶段任务进行割裂式处理,其优化目标设计相互孤立。这种分离处理方式忽视了各阶段决策行为之间的密切关联,难以实现全局最优。未来研究应着重构建统一的端到端优化框架,设计多层次的奖励机制以平衡短期与长期优化目标,并探索基于因果推理的决策方法,从而实现SFC部署各阶段的有效协同。
智能体协作与隐私保护的平衡成为重要研究方向。在多智能体协同决策场景中,智能体间的信息共享与隐私保护构成了一对矛盾。过度共享可能导致隐私泄露,而信息封闭又会影响决策效果。联邦强化学习为解决这一问题提供了新思路:通过模型参数而非原始数据共享保护隐私;利用联邦学习框架实现分布式协同优化;结合差分隐私等技术增强安全性。
在应用场景层面,新型网络架构下的SFC部署面临诸多挑战。随着B5G/6G、算力网络等新型网络架构的快速演进,SFC部署面临更加动态和复杂的应用环境。在车联网场景中,高速移动的车辆节点频繁改变其地理位置和连接状态,导致网络拓扑迅速变化。这种动态特性要求SFC部署具有更高的实时性,能够快速调整服务链的位置与路径,同时保证通信的可靠性。在空天地一体化网络中,由于卫星节点的轨道运动和动态网络拓扑变化,SFC部署需应对频繁的链路重构和跨域资源协调问题。同时,大尺度传播时延带来的网络时延和抖动可能显著影响服务质量,要求SFC部署算法具备精确的时延预测和优化能力。在算力网络中,随着边缘计算、云计算和分布式计算资源的融合,网络中不仅需要高效传递数据,还需要协同调度分布式算力资源。算力网络中的SFC部署需要综合考虑计算资源和网络资源的联合优化。这些新型网络架构的特点要求SFC部署方法具备更强的环境适应能力和决策智能性。
此外,目前针对并行SFC部署的研究相对较少。与串行SFC相比,并行SFC不仅需要考虑VNF之间的依赖关系,还需要根据底层网络状态动态调整并行化程度。特别是在复杂网络环境中,如何将并行SFC的构建与部署调度、动态重配置等问题进行统一优化,仍然是一个亟待解决的挑战。这要求在现有研究基础上,进一步探索适应并行化特征的部署优化方法。
面对这些挑战与机遇,基于强化学习的SFC部署研究需在保持技术创新的同时,更加注重实际应用价值。通过算法优化、场景创新和技术融合,推动相关成果在实际网络中的落地应用,为未来网络的智能化发展提供有力支撑。
随着网络技术的快速演进,传统的SFC部署方法在应对动态网络环境和复杂部署场景时表现出诸多局限性,难以满足新型网络架构对实时性、可靠性和资源利用效率的严格要求。基于强化学习的SFC部署方法因其自适应性和智能化特点,成为近年来的研究热点。本文系统总结了基于强化学习的SFC部署研究现状,重点分析了强化学习在SFC放置、调度和重配置3个主要部署阶段的应用。通过分析现有研究在算法设计、性能优化和实际部署等方面的优势与局限性,探讨了该领域面临的挑战和未来研究方向。研究表明,基于强化学习的SFC部署方法相较于传统优化方法具有显著优势。强化学习通过与环境交互进行自适应学习,能够有效应对网络环境的动态变化。深度强化学习的引入解决了传统方法在处理高维状态空间时的局限性,多智能体框架的应用实现了分布式协同决策,提高了系统的可扩展性。然而,当前研究在强化学习模型的泛化能力、训练效率与收敛性、SFC 部署各阶段的协同优化机制以及新型网络架构下的应用探索等方面存在若干亟待解决的关键问题。未来研究应重点关注算法创新与实际应用的结合,在提升模型性能的同时,注重解决方案在实际网络环境中的部署落地。
作者贡献声明:爱因坦负责论文的整体设计,撰写并修改论文;阮宏玮提出指导意见并参与论文的修改完善;刘翰林负责部分资料的收集与整理工作;李华对论文选题、组织结构提供指导意见,并完成论文审阅工作。

参考文献 引证文献
排序方式:
1
International Telecommunication Union (ITU). ICT facts and figures 2024[R]. Geneva: International Telecommunication Union, 2024
2
CISCO. 2024 global networking trends report[R]. San Jose: Cisco Systems Inc, 2024
3
Mijumbi R, Serrat J, Gorricho J L, et al. Network function virtualization: State-of-the-art and research challenges[J]. IEEE Communications Surveys & Tutorials, 2015, 18(1): 236−262
4
Kreutz D, Ramos F M V, Verissimo P E, et al. Software-defined networking: A comprehensive survey[J]. Proceedings of the IEEE, 2014, 103(1): 14−76
5
Halpern J, Pignataro C. Service function chaining (SFC) architecture[R]. Fremont, CA: Internet Engineering Task Force, 2015
6
Bhamare D, Jain R, Samaka M, et al. A survey on service function chaining[J]. Journal of Network and Computer Applications, 2016, 75: 138−155
7
Xiao Yikai, Zhang Qixia, Liu Fangming, et al. NFVdeep: Adaptive online service function chain deployment with deep reinforcement learning[C]//Proc of the 27th Int Symp on Quality of Service. New York: ACM, 2019: 1−10
8
Kaelbling L P, Littman M L, Moore A W. Reinforcement learning: A survey[J]. Journal of Artificial Intelligence Research, 1996, 4: 237−285
9
Heo D N, Lange S, Kim H G, et al. Graph neural network based service function chaining for automatic network control[C]//Proc of the 21st Asia-Pacific Network Operations and Management Symp (APNOMS). Piscataway, NJ: IEEE, 2020: 7−12
10
Shah H A, Zhao Lian. Multiagent deep-reinforcement-learning-based virtual resource allocation through network function virtualization in Internet of things[J]. IEEE Internet of Things Journal, 2020, 8(5): 3410−3421
11
Cai Xuhui, Deng Hui, Deng Lingli, et al. Evolving NFV towards the next decade[R]. Sophia Antipolis, France: European Telecommunications Standards Institute, 2023
12
Zhang Biao, Fan Qilin, Zhang Xu, et al. A survey of VNF forwarding graph embedding in B5G/6G networks[J]. Wireless Networks, 2024, 30(5): 3735−3758
13
周伟林, 杨芫, 徐明伟. 网络功能虚拟化技术研究综述[J]. 计算机研究与发展, 2018, 55(4): 675−688
Zhou Weilin, Yang Yuan, Xu Mingwei. Network function virtualization technology research[J]. Journal of Computer Research and Development, 2018, 55(4): 675−688 (in Chinese)
14
Kaur K, Mangat V, Kumar K. A comprehensive survey of service function chain provisioning approaches in SDN and NFV architecture[J]. Computer Science Review, 2020, 38: 100298
15
Schardong F, Nunes I, Schaeffer-Filho A. NFV resource allocation: A systematic review and taxonomy of VNF forwarding graph embedding[J]. Computer Networks, 2021, 185: 107726
16
祖家琛, 胡谷雨, 严佳洁, 等. 网络功能虚拟化下服务功能链的资源管理研究综述[J]. 计算机研究与发展, 2021, 58(1): 137−152
Zu Jiachen, Hu Guyu, Yan Jiajie, et al. Resource management of service function chain in NFV enabled network: A survey[J]. Journal of Computer Research and Development, 2021, 58(1): 137−152 (in Chinese)
17
Yang Song, Li Fan, Trajanovski S, et al. Recent advances of resource allocation in network function virtualization[J]. IEEE Transactions on Parallel and Distributed Systems, 2021, 32(2): 295−314
18
Santos G L, Bezerra D F, Rocha E S, et al. Service function chain placement in distributed scenarios: A systematic review[J]. Journal of Network and Systems Management, 2022, 30(1): 4
19
Pattaranantakul M, Vorakulpipat C, Takahashi T. Service function chaining security survey: Addressing security challenges and threats[J]. Computer Networks, 2023, 221: 109484
20
ETSI ISG NFV. Network functions virtualisation (NFV): Architectural framework[R]. Sophia Antipolis, France: European Telecommunications Standards Institute, 2013
21
John W, Pentikousis K, Agapiou G, et al. Research directions in network service chaining[C]//Proc of the 2013 IEEE SDN for Future Networks and Services (SDN4FNS). Piscataway, NJ: IEEE, 2013: 1−7
22
Bjorklund M. YANG—A data modeling language for the network configuration protocol (NETCONF)[R]. Fremont, CA: Internet Engineering Task Force, 2010
23
Mehraghdam S, Keller M, Karl H. Specifying and placing chains of virtual network functions[C]//Proc of the 3rd IEEE Int Conf on Cloud Networking (CloudNet). Piscataway, NJ: IEEE, 2014: 7−13
24
Binz T, Breitenbücher U, Kopp O, et al. TOSCA: Portable Automated Deployment and Management of Cloud Applications[M]//Advanced Web Services. Berlin: Springer, 2013: 527−549
25
Sun Chen, Bi Jun, Zheng Zhilong, et al. NFP: Enabling network function parallelism in NFV[C]//Proc of the Conf of the ACM Special Interest Group on Data Communication. New York: ACM, 2017: 43−56
Sun Chen, Bi Jun, Zheng Zhilong, et al. NFP: Enabling network function parallelism in NFV[C]//Proc of the Conf of the ACM Special Interest Group on Data Communication. New York: ACM, 2017: 43−56
26
Lin I C, Yeh Y H, Lin K C J. Toward optimal partial parallelization for service function chaining[J]. IEEE/ACM Transactions on Networking, 2021, 29(5): 2033−2044
27
Bari F, Chowdhury S R, Ahmed R, et al. Orchestrating virtualized network functions[J]. IEEE Transactions on Network and Service Management, 2016, 13(4): 725−739
28
Addis B, Belabed D, Bouet M, et al. Virtual network functions placement and routing optimization[C]//Proc of the 4th IEEE Int Conf on Cloud Networking (CloudNet). Piscataway, NJ: IEEE, 2015: 171−177
29
刘艺, 张红旗, 杨英杰, 等. 一种区分等级的可生存服务功能链映射方法[J]. 计算机研究与发展, 2018, 55(4): 748−767
Liu Yi, Zhang Hongqi, Yang Yingjie, et al. A hierarchical method for survivable service function chain embedding[J]. Journal of Computer Research and Development, 2018, 55(4): 748−767 (in Chinese)
30
Riera J F, Hesselbach X, Escalona E, et al. On the complex scheduling formulation of virtual network functions over optical networks[C]//Proc of the 16th Int Conf on Transparent Optical Networks (ICTON). Piscataway, NJ: IEEE, 2014: 1−5
31
Qu Long, Assi C, Shaban K. Delay-aware scheduling and resource optimization with network function virtualization[J]. IEEE Transactions on Communications, 2016, 64(9): 3746−3758
32
阳勇, 孟相如, 康巧燕, 等. 基于资源需求预测的动态服务功能链迁移方法[J]. 计算机研究与发展, 2023, 60(5): 1151−1163
Yang Yong, Meng Xiangru, Kang Qiaoyan, et al. Dynamic service function chain migration method based on resource requirements prediction[J]. Journal of Computer Research and Development, 2023, 60(5): 1151−1163 (in Chinese)
33
Eramo V, Miucci E, Ammar M, et al. An approach for service function chain routing and virtual function network instance migration in network function virtualization architectures[J]. IEEE/ACM Transactions on Networking, 2017, 25(4): 2008−2025
34
Liu Junjie, Lu Wei, Zhou Fen, et al. On dynamic service function chain deployment and readjustment[J]. IEEE Transactions on Network and Service Management, 2017, 14(3): 543−553
35
Bellman R. A Markovian decision process[J]. Journal of Mathematics and Mechanics, 1957, 6(5): 679−684
36
Mnih V, Kavukcuoglu K, Silver D, et al. Human-level control through deep reinforcement learning[J]. Nature, 2015, 518(7540): 529−533
37
Silver D, Huang A, Maddison C J, et al. Mastering the game of Go with deep neural networks and tree search[J]. Nature, 2016, 529(7587): 484−489
38
刘全, 翟建伟, 章宗长, 等. 深度强化学习综述[J]. 计算机学报, 2018, 41(1): 1−27
Liu Quan, Zhai Jianwei, Zhang Zongzhang, et al. A survey on deep reinforcement learning[J]. Chinese Journal of Computers, 2018, 41(1): 1−27 (in Chinese)
39
丁世飞, 杜威, 张健, 等. 多智能体深度强化学习研究进展[J]. 计算机学报, 2024, 47(7): 1547−1567
Ding Shifei, Du Wei, Zhang Jian, et al. Research progress of multi-agent deep reinforcement learning[J]. Chinese Journal of Computers, 2024, 47(7): 1547−1567 (in Chinese)
40
Kim S I, Kim H S. A research on dynamic service function chaining based on reinforcement learning using resource usage[C]//Proc of the 9th Int Conf on Ubiquitous and Future Networks (ICUFN). Piscataway, NJ: IEEE, 2017: 582−586
41
Sun Jian, Huang Guanhua, Sun Gang, et al. A Q-learning-based approach for deploying dynamic service function chains[J]. Symmetry, 2018, 10(11): 646
42
Watkins C J C H, Dayan P. Q-learning[J]. Machine Learning, 1992, 8(3): 279−292
43
Chen Jing, Chen Jia, Zhang Hongke. DRL-QOR: Deep reinforcement learning-based QoS/QoE-aware adaptive online orchestration in NFV-enabled networks[J]. IEEE Transactions on Network and Service Management, 2021, 18(2): 1758−1774
44
Nouruzi A, Zakeri A, Javan M R, et al. Online service provisioning in NFV-enabled networks using deep reinforcement learning[J]. IEEE Transactions on Network and Service Management, 2022, 19(3): 3276−3289
45
Li Guanglei, Feng Bohao, Zhou Huachun, et al. Adaptive service function chaining mappings in 5G using deep Q-learning[J]. Computer Communications, 2020, 152: 305−315
46
刘光远, 陈世莹, 庞紫园. 边缘云环境中收益最大化的在线服务功能链部署机制[J]. 清华大学学报: 自然科学版, 2025, 65(8): 1516−1529
Liu Guangyuan, Chen Shiying, Pang Ziyuan. Deployment mechanism for maximizing revenue from online service function chains in edge cloud environments[J]. Journal of Tsinghua University: Science and Technology, 2025, 65(8): 1516−1529
47
Quang P T A, Hadjadj-Aoul Y, Outtagarts A. A deep reinforcement learning approach for VNF forwarding graph embedding[J]. IEEE Transactions on Network and Service Management, 2019, 16(4): 1318−1331
48
唐伦, 贺兰钦, 连沁怡, 等. 基于改进深度强化学习的虚拟网络功能部署优化算法[J]. 电子与信息学报, 2021, 43(6): 1724−1732
Tang Lun, He Lanqin, Lian Qinyi, et al. Virtual network function placement optimization algorithm based on improve deep reinforcement learning[J]. Journal of Electronics & Information Technology, 2021, 43(6): 1724−1732 (in Chinese)
49
He Nan, Yang Song, Li Fan, et al. A-DDPG: Attention mechanism-based deep reinforcement learning for NFV[C]//Proc of the 29th IEEE/ACM Int Symp on Quality of Service (IWQOS). Piscataway, NJ: IEEE, 2021: 1−10
50
He Nan, Yang Song, Li Fan, et al. Leveraging deep reinforcement learning with attention mechanism for virtual network function placement and routing[J]. IEEE Transactions on Parallel and Distributed Systems, 2023, 34(4): 1186−1201
51
Liu Yicen, Zhang Junning. Service function chain embedding meets machine learning: Deep reinforcement learning approach[J]. IEEE Transactions on Network and Service Management, 2024, 21(3): 3465−3481
52
Huang Zhongwei, Zhong Weijian, Li Dagang, et al. Delay constrained SFC orchestration for edge intelligence-enabled IIoT: A DRL approach[J]. Journal of Network and Systems Management, 2023, 31(3): 53
53
Wang Lei, Mao Weixi, Zhao Jin, et al. DDQP: A double deep Q-learning approach to online fault-tolerant SFC placement[J]. IEEE Transactions on Network and Service Management, 2021, 18(1): 118−132
54
Qu Hua, Wang Ke, Zhao Jihong. Reliable service function chain deployment method based on deep reinforcement learning[J]. Sensors, 2021, 21(8): 2733
55
Santos G L, Lynn T, Kelner J, et al. Availability-aware and energy-aware dynamic SFC placement using reinforcement learning[J]. The Journal of Supercomputing, 2021, 77(11): 12711−12740
56
Santos G L, Endo P T, Lynn T, et al. A reinforcement learning-based approach for availability-aware service function chain placement in large-scale networks[J]. Future Generation Computer Systems, 2022, 136: 93−109
57
Marler R T, Arora J S. The weighted sum method for multi-objective optimization: new insights[J]. Structural and Multidisciplinary Optimization, 2010, 41(6): 853−862
58
Konak A, Coit D W, Smith A E. Multi-objective optimization using genetic algorithms: A tutorial[J]. Reliability Engineering & System Safety, 2006, 91(9): 992−1007
59
Bi Yu, Meixner C C, Bunyakitanon M, et al. Multi-objective deep reinforcement learning assisted service function chains placement[J]. IEEE Transactions on Network and Service Management, 2021, 18(4): 4134−4150
60
Li Huaping, Kordi M E. DSPPV: Dynamic service function chains placement with parallelized virtual network functions in mobile edge computing[J]. Internet of Things, 2023, 22: 100733
61
Jia Jing, Hua Jing. Dynamic SFC placement with parallelized VNFs in data center networks: A DRL-based approach[J]. ICT Express, 2024, 10(1): 104−110
62
Sun Penghao, Lan Julong, Li Junfei, et al. Combining deep reinforcement learning with graph neural networks for optimal VNF placement[J]. IEEE Communications Letters, 2020, 25(1): 176−180
63
Battaglia P W, Hamrick J B, Bapst V, et al. Relational inductive biases, deep learning, and graph networks[J]. arXiv preprint, arXiv: 1806.01261, 2018
64
Xie Yanghao, Huang Lin, Kong Yuyang, et al. Virtualized network function forwarding graph placing in SDN and NFV-enabled IoT networks: A graph neural network assisted deep reinforcement learning method[J]. IEEE Transactions on Network and Service Management, 2021, 19(1): 524−537
65
Qiu Rixuan, Bao Jiawen, Li Yuancheng, et al. Virtual network function deployment algorithm based on graph convolution deep reinforcement learning[J]. The Journal of Supercomputing, 2023, 79(6): 6849−6870
66
Fan Qilin, Pan Pan, Li Xiuhua, et al. DRL-D: Revenue-aware online service function chain deployment via deep reinforcement learning[J]. IEEE Transactions on Network and Service Management, 2022, 19(4): 4531−4545
67
Zhu Yuchao, Yao Haipeng, Mai Tianle, et al. Multiagent reinforcement-learning-aided service function chain deployment for Internet of things[J]. IEEE Internet of Things Journal, 2022, 9(17): 15674−15684
68
Xiao Da, Zhang J A, Liu Xin, et al. A two-stage GCN-based deep reinforcement learning framework for SFC embedding in multi-datacenter networks[J]. IEEE Transactions on Network and Service Management, 2023, 20(4): 4297−4312
69
Wang Shaoyang, Yuen C, Ni Wei, et al. Multiagent deep reinforcement learning for cost-and delay-sensitive virtual network function placement and routing[J]. IEEE Transactions on Communications, 2022, 70(8): 5208−5224
70
Wang Kan, Yuan Peng, Jan M A, et al. Digital twin-assisted service function chaining in multi-domain computing power networks with multi-agent reinforcement learning[J]. Future Generation Computer Systems, 2024, 158: 294−307
71
Wang Shuyi, Cao Haotong, Yang Longxiang, et al. GCN-based multi-agent deep reinforcement learning for dynamic service function chain deployment in IoT[J]. IEEE Transactions on Consumer Electronics, 2024, 70(3): 6105−6118
72
Wang Tianfeng, Zu Jiachen, Hu Guyu, et al. Adaptive service function chain scheduling in mobile edge computing via deep reinforcement learning[J]. IEEE Access, 2020, 8: 164922−164935
73
He Rui, Ren Bangbang, Xie Junjie, et al. A reinforcement learning method for scheduling service function chains with multi-resource constraints[J]. Computer Networks, 2023, 235: 109985
74
Yao Jiamin, Wang Junli, Wang Cheng, et al. DRL-based VNF cooperative scheduling framework with priority-weighted delay[J]. IEEE Transactions on Mobile Computing, 2024, 23(12): 11375−11388
75
Jia Ziye, Cao Yilu, He Lijun, et al. Service function chain dynamic scheduling in space-air-ground integrated networks[J]. IEEE Transactions on Vehicular Technology, 2025, 74(7): 11235−11248
76
Li Junling, Shi Weisen, Zhang Ning, et al. Delay-aware VNF scheduling: A reinforcement learning approach with variable action set[J]. IEEE Transactions on Cognitive Communications and Networking, 2020, 7(1): 304−318
77
Akbari M, Abedi M R, Joda R, et al. Age of information aware VNF scheduling in industrial IoT using deep reinforcement learning[J]. IEEE Journal on Selected Areas in Communications, 2021, 39(8): 2487−2500
78
Kuai Zhenran, Wang Tianyu, Wang Shaowei. Fair virtual network function mapping and scheduling using proximal policy optimization[J]. IEEE Transactions on Communications, 2022, 70(11): 7434−7445
79
张庆华, 张先超, 王寅昊, 等. 面向医疗急救的信息网络服务功能链调度方法[J]. 电子学报, 2023, 51(11): 3128−3136
Zhang Qinghua, Zhang Xianchao, Wang Yinhao, et al. Service function chains scheduling of information networks used for emergency treatment[J]. Acta Electronica Sinica, 2023, 51(11): 3128−3136 (in Chinese)
80
Lu Yuxi, Zhang Peiying, Duan Youxiang, et al. Dynamic scheduling of iov edge cloud service functions under NFV: A multi-agent reinforcement learning approach[J]. IEEE Transactions on Vehicular Technology, 2023, 73(4): 5730−5741
81
Liu Zhiwei, Shu Zhaogang, Chen Shuwu, et al. Low-latency virtual network function scheduling algorithm based on deep reinforcement learning[J]. Computer Networks, 2024, 246: 110418
82
唐伦, 周钰, 谭颀, 等. 基于强化学习的5G网络切片虚拟网络功能迁移算法[J]. 电子与信息学报, 2020, 42(3): 669−677
Tang Lun, Zhou Yu, Tan Qi, et al. Virtual network function migration algorithm based on reinforcement learning for 5G network slicing[J]. Journal of Electronics & Information Technology, 2020, 42(3): 669−677 (in Chinese)
83
Wang Yuantao, Shu Zhaogang, Chen Shuwu, et al. A cost and demand sensitive adjustment algorithm for service function chain in data center network[J]. Computer Networks, 2024, 242: 110254
84
Pei Jianing, Hong Peilin, Pan Miao, et al. Optimal VNF placement via deep reinforcement learning in SDN/NFV-enabled networks[J]. IEEE Journal on Selected Areas in Communications, 2019, 38(2): 263−278
85
Ran Jing, Wang Wenkai, Hu Hefei. Dynamic service function chain deployment and readjustment method based on deep reinforcement learning[J]. Sensors, 2023, 23(6): 3054
86
Zhang Peiying, Li Yilin, Tan Lizhuang, et al. Latency-sensitive service function chains intelligent migration in satellite communication driven by deep reinforcement learning[J]. Transactions on Emerging Telecommunications Technologies, 2024, 35(11): e70006
87
Hu Hefei, Zhang Wei, Xu Lingyi, et al. A mobility-aware service function chain migration strategy based on deep reinforcement learning[J]. Journal of Network and Systems Management, 2023, 31(1): 21
88
Xu Lingyi, Liu Wenbin, Wang Zhiwei, et al. Mobile-aware service function chain intelligent seamless migration in multi-access edge computing[J]. Journal of Network and Systems Management, 2024, 32(3): 49
89
Liu Yicen, Lu Yu, Li Xi, et al. On dynamic service function chain reconfiguration in IoT networks[J]. IEEE Internet of Things Journal, 2020, 7(11): 10969−10984
90
Cai Jun, Qian Kaili, Luo Jianzhen, et al. SARM: Service function chain active reconfiguration mechanism based on load and demand prediction[J]. International Journal of Intelligent Systems, 2022, 37(9): 6388−6414
91
Qu Hua, Wang Ke, Zhao Jihong. Priority-awareness VNF migration method based on deep reinforcement learning[J]. Computer Networks, 2022, 208: 108866
92
Yu Xue, Wang Ran, Hao Jie, et al. Priority-aware deployment of autoscaling service function chains based on deep reinforcement learning[J]. IEEE Transactions on Cognitive Communications and Networking, 2024, 10(3): 1050−1062
93
Chen Jing, Chen Jia, Guo Kuo, et al. Fault tolerance oriented SFC optimization in SDN/NFV-enabled cloud environment based on deep reinforcement learning[J]. IEEE Transactions on Cloud Computing, 2024, 12(1): 200−218
94
Li Yilin, Zhang Peiying, Kumar N, et al. Reliability-assured service function chain migration strategy in edge networks using deep reinforcement learning[J]. Journal of Network and Computer Applications, 2024, 231: 103999
2026年第63卷第2期
221
91
0
引用本文
BibTeX
文章信息
doi: 10.7544/issn1000-1239.202550002
  • 接收时间:2025-01-01
  • 首发时间:2026-11-06
  • 出版时间:2026-02-01
补充材料
相关文章
文章信息
作者
出版历史
  • 收稿日期:2025-01-01
  • 修回日期:2025-08-11
  • 录用日期:2025-09-02
基金
作者信息
    1内蒙古大学计算机学院 呼和浩特 010021
    2生态大数据教育部工程研究中心(内蒙古大学) 呼和浩特 010021
    3内蒙古自治区云计算与服务软件工程实验室(内蒙古大学) 呼和浩特 010021

通讯作者:

李华()
参考文献
分享链接
https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/10.7544/issn1000-1239.202550002
分享至
全文二维码

扫描看全文

引用本文
BibTeX
本文的引用情况
2种不同金属材料的力学参数
科
Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)
属
Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏