Article(id=1327017497221509691, tenantId=1146029695717560320, journalId=1324759914146529478, issueId=1327017337032651028, articleNumber=null, orderNo=null, doi=10.7544/issn1000-1239.202440969, pmid=null, cstr=null, oa=null, hot=null, price=null, onlineType=0, articleFormat=0, articleType=null, articleTypeStr=research-article, receivedDate=1735401600000, receivedDateStr=2024-12-29, revisedDate=1743436800000, revisedDateStr=2025-04-01, acceptedDate=null, acceptedDateStr=null, onlineDate=1794315226130, onlineDateStr=2026-11-10, pubDate=1764518400000, pubDateStr=2025-12-01, doiRegisterDate=null, doiRegisterDateStr=null, onlineIssueDate=1794315226130, onlineIssueDateStr=2026-11-10, onlineJustAcceptDate=null, onlineJustAcceptDateStr=null, onlineFirstDate=null, onlineFirstDateStr=null, sourceXml=null, magXml=null, createTime=1794315226130, creator=13701087609, updateTime=1794315226130, updator=13701087609, issue=Issue{id=1327017337032651028, tenantId=1146029695717560320, journalId=1324759914146529478, year='2025', volume='62', issue='12', pageStart='2889', pageEnd='3146', issueExtLink='null', onlineDate='null', pubDate='1764518400000', pubDateStr='2025-12-01', beforeIssueId=null, nextIssueId=null, price=null, status=1, issueComplete=1, articleOrder=1, issueType=-1, specialIssue=null, createTime=1794315187938, creator='13701087609', updateTime=1794377025235, updator='13701087609', preIssue=null, nextIssue=null, articleTotal=null, ext={EN=IssueExt(id=1327276701995872766, tenantId=1146029695717560320, journalId=1324759914146529478, issueId=1327017337032651028, language=EN, specialIssueTitle=, coverIllustrator=null, specialIssueEditor=, specialIssueAbout=), CN=IssueExt(id=1327276701995872767, tenantId=1146029695717560320, journalId=1324759914146529478, issueId=1327017337032651028, language=CN, specialIssueTitle=, coverIllustrator=null, specialIssueEditor=, specialIssueAbout=)}, issueFiles=null, downloadFileDto=null}, startPage=3002, endPage=3012, ext={EN=ArticleExt(id=1327017499033449020, articleId=1327017497221509691, tenantId=1146029695717560320, journalId=1324759914146529478, language=EN, title=Hierarchical Sinkhorn Distance-Based Clustering Federated Learning Algorithm for Non-IID Data, columnId=1325489891789910943, journalTitle=Journal of Computer Research and Development, columnName=Artificial Intelligence, runingTitle=null, highlight=null, articleAbstract=

Federated learning, as an emerging distributed neural network training paradigm in edge computing, faces a key challenge of data heterogeneity over clients, which significantly impacts global model performance. While clustering federated learning, serving as a promising solution, has shown improvements in model accuracy, its effectiveness is limited by the insufficient utilization of local data distribution statistics. To address this issue, we propose a novel federated learning framework named HS-CFA (hierarchical Sinkhorn distance-based clustering federated algorithm), which is designed to optimize the aggregation weights of local models under the heterogeneous data environments, thereby enhancing the performance of the global model. It utilizes the entropy-regularized optimal transport cost to capture the characteristics of local data distribution and dynamically adjusts the aggregation weights of local models with a hierarchical clustering strategy. Specially, HS-CFA employs the Sinkhorn distance as a lightweight optimal transport cost metric to measure distributional dissimilarities across clients. Furthermore, it adopts a hierarchical two-layer clustering mechanism, combining density-based spatial clustering and average aggregation during the server training phase, facilitating the dynamic and adaptive adjustment of local model aggregation weights. Experimental results on multiple benchmark datasets demonstrate that HS-CFA significantly enhances the accuracy and robustness of the global model in scenarios with highly heterogeneous distribution settings.

, authors=Rui Zhao, Xiaowen Duan, Xin Liu, Rui Zhou, Qingguo Zhou*, authorsList=Rui Zhao, Xiaowen Duan, Xin Liu, Rui Zhou, Qingguo Zhou, authorCompany=null, correspAuthors=Qingguo Zhou, authorNote=

born in 1989. PhD candidate. Student member of CCF. His main research interests include distributed neural networks and vehicular-roadside cooperation

born in 2000. Master candidate. Her main research interest includes federated learning algorithm optimization

born in 1995. PhD, associate professor. Member of CCF. His main research interests include cybersecurity and artificial intelligence

born in 1981. PhD, professor. Senior member of CCF. His main research interests include distributed real-time systems and artificial intelligence

born in 1973. PhD, professor. Senior member of CCF. His main research interests include distributed and embedded real-time systems, neural networks, and intelligent transportation

, correspAuthorsNote=null, copyrightStatement=Copyright © 2025 Journal of Computer Research and Development. All rights reserved., copyrightOwner=null, extLink=null, articleAbsUrl=null, sourceXml=null, magXml=null, pdfUrl=null, pdf=null, pdfFileSize=null, pdfExtLink=null, richHtmlUrl=null, mobilePdfUrl=null, reviewReport=null, pdfFirstPage=null, abstractGraph=null, abstractGraphContent=null, abstractVideo=null, citation=null, cebUrl=null, magXmlContent=null, mapNumber=null, fund=null), CN=ArticleExt(id=1327017501436785231, articleId=1327017497221509691, tenantId=1146029695717560320, journalId=1324759914146529478, language=CN, title=非独立同分布数据下层次化Sinkhorn距离聚类联邦学习算法, columnId=1325489892104483745, journalTitle=计算机研究与发展, columnName=人工智能, runingTitle=null, highlight=null, articleAbstract=

联邦学习作为一种边缘计算中的新兴分布式神经网络训练方法面临客户端数据异构性挑战,其中聚类联邦学习被认为是一种颇具潜力的解决方案,然而现有聚类联邦学习算法未深入探究量化客户端数据分布差异. 针对该问题提出了一种新颖的层次化Sinkhorn距离聚类联邦学习算法(hierarchical Sinkhorn distance-based clustering federated learning algorithm,HS-CFA),采用最优传输代价衡量局部训练时客户端数据分布特性,提出层次化聚类策略动态调整全局模型聚合时的参与权重. 具体而言,HS-CFA引入Sinkhorn距离量化客户端间的分布差异,提出使用基于密度的聚类算法在服务器端进行动态层次聚类. 在多个基准数据集上的实验结果表明,相比传统算法,所提算法在高度数据分布异构性的场景中显著提升了全局模型的准确率和鲁棒性.

, authors=赵瑞, 段小文, 刘新, 周睿, 周庆国*, authorsList=赵瑞, 段小文, 刘新, 周睿, 周庆国, authorCompany=null, correspAuthors=周庆国, authorNote=

1989年生. 博士研究生. CCF学生会员. 主要研究方向为分布式神经网络、车路协同

2000年生. 硕士研究生. 主要研究方向为联邦学习算法优化

1995年生. 博士,副教授. CCF会员. 主要研究方向为网络安全、人工智能

1981年生. 博士,教授. CCF高级会员. 主要研究方向为分布式实时系统、人工智能

1973年生. 博士,教授. CCF高级会员. 主要研究方向为分布式与嵌入式实时系统、神经网络、智慧交通

, correspAuthorsNote=
周庆国()
, copyrightStatement=版权所有 © 2025 计算机研究与发展编辑部, copyrightOwner=null, extLink=null, articleAbsUrl=null, sourceXml=YxJSRHfubdasGjtcP5s9wA==, magXml=pSU8f8+a/fy9twgOyVRmOw==, pdfUrl=null, pdf=sWpWoSFYP45/iH4qIJOh5g==, pdfFileSize=1698499, pdfExtLink=null, richHtmlUrl=null, mobilePdfUrl=null, reviewReport=null, pdfFirstPage=null, abstractGraph=KhIs9oid2z/LLiNYH26XVg==, abstractGraphContent=null, abstractVideo=null, citation=null, cebUrl=null, magXmlContent=KLZBsaATzZgOLvvaRMJOdA==, mapNumber=null, fund=null)}, authors=[Author(id=1327017501789106772, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, orderNo=0, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=zhaor19@lzu.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327017503429079638, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, authorId=1327017501789106772, language=EN, stringName=Rui Zhao, firstName=Rui, middleName=null, lastName=Zhao, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=School of Information Science and Engineering, Lanzhou University, Lanzhou 730030, bio={"content":"

born in 1989. PhD candidate. Student member of CCF. His main research interests include distributed neural networks and vehicular-roadside cooperation

"}, bioImg=null, bioContent=

born in 1989. PhD candidate. Student member of CCF. His main research interests include distributed neural networks and vehicular-roadside cooperation

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327017503508771415, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, authorId=1327017501789106772, language=CN, stringName=赵瑞, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=兰州大学信息科学与工程学院 兰州 730030, bio={"img":"5z792S3Cve2MqrrSDhWHgQ==","content":"

1989年生. 博士研究生. CCF学生会员. 主要研究方向为分布式神经网络、车路协同

"}, bioImg=5z792S3Cve2MqrrSDhWHgQ==, bioContent=

1989年生. 博士研究生. CCF学生会员. 主要研究方向为分布式神经网络、车路协同

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327017501692637776, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, xref=null, ext=[AuthorCompanyExt(id=1327017501701026385, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, companyId=1327017501692637776, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=School of Information Science and Engineering, Lanzhou University, Lanzhou 730030), AuthorCompanyExt(id=1327017501713609298, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, companyId=1327017501692637776, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=兰州大学信息科学与工程学院 兰州 730030)])]), Author(id=1327017503584268889, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, orderNo=1, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327017503672349275, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, authorId=1327017503584268889, language=EN, stringName=Xiaowen Duan, firstName=Xiaowen, middleName=null, lastName=Duan, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=School of Information Science and Engineering, Lanzhou University, Lanzhou 730030, bio={"content":"

born in 2000. Master candidate. Her main research interest includes federated learning algorithm optimization

"}, bioImg=null, bioContent=

born in 2000. Master candidate. Her main research interest includes federated learning algorithm optimization

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327017503768818268, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, authorId=1327017503584268889, language=CN, stringName=段小文, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=兰州大学信息科学与工程学院 兰州 730030, bio={"img":"GK7ReiKCk/dvDzUs14n9Tw==","content":"

2000年生. 硕士研究生. 主要研究方向为联邦学习算法优化

"}, bioImg=GK7ReiKCk/dvDzUs14n9Tw==, bioContent=

2000年生. 硕士研究生. 主要研究方向为联邦学习算法优化

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327017501692637776, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, xref=null, ext=[AuthorCompanyExt(id=1327017501701026385, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, companyId=1327017501692637776, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=School of Information Science and Engineering, Lanzhou University, Lanzhou 730030), AuthorCompanyExt(id=1327017501713609298, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, companyId=1327017501692637776, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=兰州大学信息科学与工程学院 兰州 730030)])]), Author(id=1327017503869481566, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, orderNo=2, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327017503957561952, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, authorId=1327017503869481566, language=EN, stringName=Xin Liu, firstName=Xin, middleName=null, lastName=Liu, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=School of Information Science and Engineering, Lanzhou University, Lanzhou 730030, bio={"content":"

born in 1995. PhD, associate professor. Member of CCF. His main research interests include cybersecurity and artificial intelligence

"}, bioImg=null, bioContent=

born in 1995. PhD, associate professor. Member of CCF. His main research interests include cybersecurity and artificial intelligence

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327017504045642337, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, authorId=1327017503869481566, language=CN, stringName=刘新, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=兰州大学信息科学与工程学院 兰州 730030, bio={"img":"9Jgtoacm0M3K0pOijTeNdQ==","content":"

1995年生. 博士,副教授. CCF会员. 主要研究方向为网络安全、人工智能

"}, bioImg=9Jgtoacm0M3K0pOijTeNdQ==, bioContent=

1995年生. 博士,副教授. CCF会员. 主要研究方向为网络安全、人工智能

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327017501692637776, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, xref=null, ext=[AuthorCompanyExt(id=1327017501701026385, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, companyId=1327017501692637776, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=School of Information Science and Engineering, Lanzhou University, Lanzhou 730030), AuthorCompanyExt(id=1327017501713609298, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, companyId=1327017501692637776, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=兰州大学信息科学与工程学院 兰州 730030)])]), Author(id=1327017504146305635, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, orderNo=3, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327017504225997413, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, authorId=1327017504146305635, language=EN, stringName=Rui Zhou, firstName=Rui, middleName=null, lastName=Zhou, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=School of Information Science and Engineering, Lanzhou University, Lanzhou 730030, bio={"content":"

born in 1981. PhD, professor. Senior member of CCF. His main research interests include distributed real-time systems and artificial intelligence

"}, bioImg=null, bioContent=

born in 1981. PhD, professor. Senior member of CCF. His main research interests include distributed real-time systems and artificial intelligence

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327017504326660710, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, authorId=1327017504146305635, language=CN, stringName=周睿, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=兰州大学信息科学与工程学院 兰州 730030, bio={"img":"tB5ZLluBNY+E1rkL3tHIAQ==","content":"

1981年生. 博士,教授. CCF高级会员. 主要研究方向为分布式实时系统、人工智能

"}, bioImg=tB5ZLluBNY+E1rkL3tHIAQ==, bioContent=

1981年生. 博士,教授. CCF高级会员. 主要研究方向为分布式实时系统、人工智能

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327017501692637776, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, xref=null, ext=[AuthorCompanyExt(id=1327017501701026385, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, companyId=1327017501692637776, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=School of Information Science and Engineering, Lanzhou University, Lanzhou 730030), AuthorCompanyExt(id=1327017501713609298, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, companyId=1327017501692637776, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=兰州大学信息科学与工程学院 兰州 730030)])]), Author(id=1327017504410546792, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, orderNo=4, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=zhouqg@lzu.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=1, authorType=1, ext={EN=AuthorExt(id=1327017504515404394, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, authorId=1327017504410546792, language=EN, stringName=Qingguo Zhou, firstName=Qingguo, middleName=null, lastName=Zhou, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=*, address=School of Information Science and Engineering, Lanzhou University, Lanzhou 730030, bio={"content":"

born in 1973. PhD, professor. Senior member of CCF. His main research interests include distributed and embedded real-time systems, neural networks, and intelligent transportation

"}, bioImg=null, bioContent=

born in 1973. PhD, professor. Senior member of CCF. His main research interests include distributed and embedded real-time systems, neural networks, and intelligent transportation

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327017504595096171, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, authorId=1327017504410546792, language=CN, stringName=周庆国, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=*, address=兰州大学信息科学与工程学院 兰州 730030, bio={"img":"iiBMQmp/Uia7AOhP8CtOBw==","content":"

1973年生. 博士,教授. CCF高级会员. 主要研究方向为分布式与嵌入式实时系统、神经网络、智慧交通

"}, bioImg=iiBMQmp/Uia7AOhP8CtOBw==, bioContent=

1973年生. 博士,教授. CCF高级会员. 主要研究方向为分布式与嵌入式实时系统、神经网络、智慧交通

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327017501692637776, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, xref=null, ext=[AuthorCompanyExt(id=1327017501701026385, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, companyId=1327017501692637776, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=School of Information Science and Engineering, Lanzhou University, Lanzhou 730030), AuthorCompanyExt(id=1327017501713609298, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, companyId=1327017501692637776, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=兰州大学信息科学与工程学院 兰州 730030)])])], keywords=[Keyword(id=1327017504687370860, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, language=EN, orderNo=1, keyword=edge computing), Keyword(id=1327017504771256941, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, language=EN, orderNo=2, keyword=federated learning), Keyword(id=1327017504867725934, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, language=EN, orderNo=3, keyword=optimal transport), Keyword(id=1327017504976777839, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, language=EN, orderNo=4, keyword=data heterogeneity), Keyword(id=1327017505048081008, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, language=EN, orderNo=5, keyword=clustering optimization), Keyword(id=1327017505123578481, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, language=CN, orderNo=1, keyword=边缘计算), Keyword(id=1327017505186493042, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, language=CN, orderNo=2, keyword=联邦学习), Keyword(id=1327017505261990515, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, language=CN, orderNo=3, keyword=最优传输), Keyword(id=1327017505329099380, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, language=CN, orderNo=4, keyword=数据异构性), Keyword(id=1327017505408791157, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, language=CN, orderNo=5, keyword=聚类优化)], refs=[Reference(id=1327017508785205896, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=1, rfOrder=0, authorNames=null, journalName=null, refType=null, unstructuredReference=Noor M M, Hassan W H. Current research on Internet of things (IoT) security: A survey[J]. Computer Networks, 2019, 148: 283−294, articleTitle=null, refAbstract=null), Reference(id=1327017508843926153, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=2, rfOrder=1, authorNames=null, journalName=null, refType=null, unstructuredReference=Kumar S R, Nystrom M, Lambert J, et al. Adversarial machine learning-industry perspectives[C]//Proc of the 41st IEEE Symp on Security and Privacy. Piscataway, NJ: IEEE, 2020: 69−75, articleTitle=null, refAbstract=null), Reference(id=1327017508915229322, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=3, rfOrder=2, authorNames=null, journalName=null, refType=null, unstructuredReference=Sarkar S. Internet of Things[M]. San Francisco: Morgan Kaufmann, 2016, articleTitle=null, refAbstract=null), Reference(id=1327017508973949579, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=4, rfOrder=3, authorNames=null, journalName=null, refType=null, unstructuredReference=Mao Yuyi, You Changsheng, Zhang Jun, et al. A survey on mobile edge computing: The communication perspective[J]. IEEE Communications Surveys & Tutorials, 2017, 19(4): 2322−2358, articleTitle=null, refAbstract=null), Reference(id=1327017509108167308, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=5, rfOrder=4, authorNames=null, journalName=null, refType=null, unstructuredReference=Yu Wei, Liang Fan, He Xiaofei, et al. A survey on the edge computing for the Internet of Things[J]. IEEE Access, 2017, 6: 6900−6919, articleTitle=null, refAbstract=null), Reference(id=1327017509171081869, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=6, rfOrder=5, authorNames=null, journalName=null, refType=null, unstructuredReference=Shakarami A, Ghobaei-Arani M, Shahidinegad A. A survey on the computation offloading approaches in mobile edge computing: A machine learning-based perspective[J]. Computer Networks, 2020, 182: 107496, articleTitle=null, refAbstract=null), Reference(id=1327017509229802126, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=7, rfOrder=6, authorNames=null, journalName=null, refType=null, unstructuredReference=Kairouz P, Mcmahan H B, Avent B, et al. Advances and open problems in federated learning[J]. Foundations and Trends® in Machine Learning, 2021, 14(1/2): 1−210, articleTitle=null, refAbstract=null), Reference(id=1327017509296910991, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=8, rfOrder=7, authorNames=null, journalName=null, refType=null, unstructuredReference=Abreha H G, Hayajneh M, Serhani M A. Federated learning in edge computing: A systematic survey[J]. Sensors, 2022, 22(2): 450, articleTitle=null, refAbstract=null), Reference(id=1327017509364019856, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=9, rfOrder=8, authorNames=null, journalName=null, refType=null, unstructuredReference=张雪晴, 刘延伟, 刘金霞, 等. 面向边缘智能的联邦学习综述[J]. 计算机研究与发展, 2023, 60(6): 1276−1295, articleTitle=null, refAbstract=null), Reference(id=1327017509431128721, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=9, rfOrder=9, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhang Xueqing, Liu Yanwei, Liu Jinxia, et al. An overview of federated learning in edge intelligence[J]. Journal of Computer Research and Development, 2023, 60(6): 1276−1295 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1327017509498237586, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=10, rfOrder=10, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhu Hangyu, Xu Jinjin, Liu Shiqing, et al. Federated learning on non-IID data: A survey[J]. Neurocomputing, 2021, 465: 371−390, articleTitle=null, refAbstract=null), Reference(id=1327017509590512275, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=11, rfOrder=11, authorNames=null, journalName=null, refType=null, unstructuredReference=Sattler F, Wiedemann S, Muller K R, et al. Robust and communication-efficient federated learning from non-IID data[J]. IEEE Transactions on Neural Networks and Learning Systems, 2020, 31(9): 3400−3413, articleTitle=null, refAbstract=null), Reference(id=1327017509653426836, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=12, rfOrder=12, authorNames=null, journalName=null, refType=null, unstructuredReference=Qu Liangqiong, Zhou Yuyin, Liang P P, et al. Rethinking architecture design for tackling data heterogeneity in federated learning[C]//Proc of the 2022 IEEE/CVF Conf on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society, 2022: 10061−10071, articleTitle=null, refAbstract=null), Reference(id=1327017509716341397, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=13, rfOrder=13, authorNames=null, journalName=null, refType=null, unstructuredReference=张瑞麟, 杜晋华, 尹浩. 跨设备联邦学习中的客户端选择算法[J]. 软件学报, 2024, 35(12): 5725−5740, articleTitle=null, refAbstract=null), Reference(id=1327017509783450262, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=13, rfOrder=14, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhang Ruilin, Du Jinhua, Yin Hao. Client selection algorithm in cross-device federated learning[J]. Journal of Software, 2024, 35(12): 5725−5740 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1327017509858947735, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=14, rfOrder=15, authorNames=null, journalName=null, refType=null, unstructuredReference=郭松岳, 王阳谦, 柏思远, 等. 面向数据混合分布的联邦自适应交互模型[J]. 计算机研究与发展, 2023, 60(6): 1346−1357, articleTitle=null, refAbstract=null), Reference(id=1327017509938639512, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=14, rfOrder=16, authorNames=null, journalName=null, refType=null, unstructuredReference=Guo Songyue, Wang Yangqian, Bai Siyuan, et al. Federated adaptive interaction model for mixed distribution data[J]. Journal of Computer Research and Development, 2023, 60(6): 1346−1357 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1327017510018331289, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=15, rfOrder=17, authorNames=null, journalName=null, refType=null, unstructuredReference=Wang Jianyu, Charles Z, Xu Zheng, et al. A field guide to federated optimization[J]. arXiv preprint, arXiv: 2107.06917, 2021, articleTitle=null, refAbstract=null), Reference(id=1327017510102217370, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=16, rfOrder=18, authorNames=null, journalName=null, refType=null, unstructuredReference=Ghosh A, Chung Jichan, Yin Dong, et al. An efficient framework for clustered federated learning[C]//Proc of the 33rd Advances in Neural Information Processing Systems. New York: Curran Associates Inc, 2020: 19586−19597, articleTitle=null, refAbstract=null), Reference(id=1327017510169326235, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=17, rfOrder=19, authorNames=null, journalName=null, refType=null, unstructuredReference=Briggs C, Fan Zhong, Andras P. Federated learning with hierarchical clustering of local updates to improve training on non-IID data[C]//Proc of the 2020 Int Joint Conf on Neural Networks. Piscataway, NJ: IEEE, 2020: 1−9, articleTitle=null, refAbstract=null), Reference(id=1327017510232240796, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=18, rfOrder=20, authorNames=null, journalName=null, refType=null, unstructuredReference=Kim Y, Hakim E A, Haraldson J, et al. Dynamic clustering in federated learning[C]//Proc of the 2021 IEEE Int Conf on Communications. Piscataway, NJ: IEEE, 2021: 1−6, articleTitle=null, refAbstract=null), Reference(id=1327017510328709789, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=19, rfOrder=21, authorNames=null, journalName=null, refType=null, unstructuredReference=Li Chengxi, Li Gang, Varshney P K. Federated learning with soft clustering[J]. IEEE Internet of Things Journal, 2021, 9(10): 7773−7782, articleTitle=null, refAbstract=null), Reference(id=1327017510391624350, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=20, rfOrder=22, authorNames=null, journalName=null, refType=null, unstructuredReference=Sattler F, Muller K R, Samek W. Clustered federated learning: Model-agnostic distributed multitask optimization under privacy constraints[J]. IEEE Transactions on Neural Networks and Learning Systems, 2020, 32(8): 3710−3722, articleTitle=null, refAbstract=null), Reference(id=1327017510467121823, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=21, rfOrder=23, authorNames=null, journalName=null, refType=null, unstructuredReference=Morafah M, Vahidian S, Wang Weijia, et al. FLIS: Clustered federated learning via inference similarity for non-IID data distribution[J]. IEEE Open Journal of the Computer Society, 2023, 4: 109−120, articleTitle=null, refAbstract=null), Reference(id=1327017510546813600, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=22, rfOrder=24, authorNames=null, journalName=null, refType=null, unstructuredReference=Cuturi M. Sinkhorn distances: Lightspeed computation of optimal transport[C]//Proc of the 26th Advances in Neural Information Processing Systems. New York: Curran Associates Inc, 2013: 2292−2300, articleTitle=null, refAbstract=null), Reference(id=1327017510651671201, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=23, rfOrder=25, authorNames=null, journalName=null, refType=null, unstructuredReference=Schubert E, Sander J, Ester M, et al. DBSCAN revisited, revisited: Why and how you should (still) use DBSCAN[J]. ACM Transactions on Database Systems, 2017, 42(3): 1−21, articleTitle=null, refAbstract=null), Reference(id=1327017510722974370, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=24, rfOrder=26, authorNames=null, journalName=null, refType=null, unstructuredReference=Sejourbe T, Feydy J, Vialard F X, et al. Sinkhorn divergences for unbalanced optimal transport[J]. arXiv preprint, arXiv: 1910.12958, 2019, articleTitle=null, refAbstract=null), Reference(id=1327017512434250404, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=25, rfOrder=27, authorNames=null, journalName=null, refType=null, unstructuredReference=Mcmahan B, Moore E, Ramage D, et al. Communication-efficient learning of deep networks from decentralized data[C]//Proc of the 20th Int Conf on Artificial Intelligence and Statistics. New York: PMLR, 2017: 1273−1282, articleTitle=null, refAbstract=null), Reference(id=1327017512505553573, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=26, rfOrder=28, authorNames=null, journalName=null, refType=null, unstructuredReference=万伟, 胡胜山, 陆建荣, 等. 联邦学习在高度数据异构场景下的泛化鲁棒性增强[J]. 中国科学: 信息科学, 2024, 54(3): 566−581, articleTitle=null, refAbstract=null), Reference(id=1327017512572662438, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=26, rfOrder=29, authorNames=null, journalName=null, refType=null, unstructuredReference=Wan Wei, Hu Shengshan, Lu Jianrong, et al. Enhancing generalization robustness of federated learning in highly heterogeneous environments[J]. SCIENTIA SINICA Informationis, 2024, 54(3): 566−581 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1327017512643965607, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=27, rfOrder=30, authorNames=null, journalName=null, refType=null, unstructuredReference=Li Tian, Sahu A K, Zaheer M, et al. Federated optimization in heterogeneous networks[J]. Proceedings of Machine Learning and Systems, 2020, 2: 429−450, articleTitle=null, refAbstract=null), Reference(id=1327017512723657384, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=28, rfOrder=31, authorNames=null, journalName=null, refType=null, unstructuredReference=Karimireddy S P, Kale S, Mohri M, et al. SCAFFOLD: Stochastic controlled averaging for federated learning[C]//Proc of the 37th Int Conf on Machine Learning. New York: PMLR, 2020: 5132−5143, articleTitle=null, refAbstract=null), Reference(id=1327017512811737769, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=29, rfOrder=32, authorNames=null, journalName=null, refType=null, unstructuredReference=Yan Bingjie, Liu Boyi, Wang Lujia, et al. FedCM: A real-time contribution measurement method for participants in federated learning[C]//Proc of the 2021 Int Joint Conf on Neural Networks. Piscataway, NJ: IEEE, 2021: 1−8, articleTitle=null, refAbstract=null), Reference(id=1327017512895623850, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=30, rfOrder=33, authorNames=null, journalName=null, refType=null, unstructuredReference=Sun Yan, Shen Li, Huang Tiansheng, et al. FedSpeed: Larger local interval, less communication round, and higher generalization accuracy[C/OL]//Proc of the 11th Int Conf on Learning Representations. 2023[2025-03-03]. https://openreview.net/forum?id=bZjxxYURKT, articleTitle=null, refAbstract=null), Reference(id=1327017512971121323, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=31, rfOrder=34, authorNames=null, journalName=null, refType=null, unstructuredReference=Ghadimi E, Teixeira A, Shames I, et al. Optimal parameter selection for the alternating direction method of multipliers (ADMM): Quadratic problems[J]. IEEE Transactions on Automatic Control, 2014, 60(3): 644−658, articleTitle=null, refAbstract=null), Reference(id=1327017513029841580, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=32, rfOrder=35, authorNames=null, journalName=null, refType=null, unstructuredReference=Fama F, Kalalas C, Lagen S, et al. Measuring data similarity for efficient federated learning: A feasibility study[C]//Proc of the 13th Int Conf on Computing, Networking and Communications. Piscataway, NJ: IEEE, 2024: 394−400, articleTitle=null, refAbstract=null), Reference(id=1327017513096950445, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=33, rfOrder=36, authorNames=null, journalName=null, refType=null, unstructuredReference=Xiao Han, Rasul K, Vollgraf R. Fashion-MNIST: A novel image dataset for benchmarking machine learning algorithms[J]. arXiv preprint, arXiv: 1708.07747, 2017, articleTitle=null, refAbstract=null), Reference(id=1327017513159865006, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=34, rfOrder=37, authorNames=null, journalName=null, refType=null, unstructuredReference=Krizhevisky A, Hinton G E. Learning multiple layers of features from tiny images[R]. Toronto: University of Toronto, 2009, articleTitle=null, refAbstract=null), Reference(id=1327017513210196655, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=35, rfOrder=38, authorNames=null, journalName=null, refType=null, unstructuredReference=Hsu T H, Qi Hang, Brown M. Measuring the effects of non-identical data distribution for federated visual classification[J]. arXiv preprint, arXiv: 1909.06335, 2019, articleTitle=null, refAbstract=null), Reference(id=1327017513273111216, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=36, rfOrder=39, authorNames=null, journalName=null, refType=null, unstructuredReference=Krizhevisky A, Sutskever I, HINTON G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84−90, articleTitle=null, refAbstract=null), Reference(id=1327017513340220081, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=37, rfOrder=40, authorNames=null, journalName=null, refType=null, unstructuredReference=Paszke A, Gross S, Massa F, et al. PyTorch: An imperative style, high-performance deep learning library[C]//Proc of the 33rd Int Conf on Neural Information Processing Systems. New York: Curran Associates Inc, 2019: 8026−8039, articleTitle=null, refAbstract=null), Reference(id=1327017513419911858, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=38, rfOrder=41, authorNames=null, journalName=null, refType=null, unstructuredReference=Reisizadeh A, Farnia F, Pedarsani R, et al. Robust federated learning: The case of affine distribution shifts[C]//Proc of the 34th Int Conf on Neural Information Processing Systems. New York: Curran Associates Inc, 2020: 21554−21565, articleTitle=null, refAbstract=null)], funds=null, companyList=[AuthorCompany(id=1327017501692637776, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, xref=null, ext=[AuthorCompanyExt(id=1327017501701026385, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, companyId=1327017501692637776, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=School of Information Science and Engineering, Lanzhou University, Lanzhou 730030), AuthorCompanyExt(id=1327017501713609298, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, companyId=1327017501692637776, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=兰州大学信息科学与工程学院 兰州 730030)])], figs=[ArticleFig(id=1327017505551397494, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, language=EN, label=Figure 1, caption=Schematic diagram of HS-CFA algorithm, figureFileSmall=MgU4suahM2VVUqh3+L+Drw==, figureFileBig=KhIs9oid2z/LLiNYH26XVg==, tableContent=null), ArticleFig(id=1327017505631089271, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, language=CN, label=图1, caption=HS-CFA算法示意图, figureFileSmall=MgU4suahM2VVUqh3+L+Drw==, figureFileBig=KhIs9oid2z/LLiNYH26XVg==, tableContent=null), ArticleFig(id=1327017505865970296, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, language=EN, label=Figure 2, caption=Test accuracy curves of HS-CFA and baseline algorithms with different Dirichlet coefficients on CIFAR-10 dataset, figureFileSmall=vwXX1efeD2GOzxP+mq+s7Q==, figureFileBig=AB8duUPn5mq5ka2wOC9Wlw==, tableContent=null), ArticleFig(id=1327017505949856377, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, language=CN, label=图2, caption=CIFAR-10数据集上不同狄利克雷系数下HS-CFA与基准算法测试准确率曲线, figureFileSmall=vwXX1efeD2GOzxP+mq+s7Q==, figureFileBig=AB8duUPn5mq5ka2wOC9Wlw==, tableContent=null), ArticleFig(id=1327017506016965242, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, language=EN, label=Figure 3, caption=Effect of different neighborhood radius of DBSCAN on test accuracy of global model, figureFileSmall=504V5uF2Rgm0Ss7QU7RJ3Q==, figureFileBig=lkrg18Of+G7UVStt7DWzDw==, tableContent=null), ArticleFig(id=1327017506084074107, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, language=CN, label=图3, caption=DBSCAN中不同邻域半径对于全局模型测试准确率的影响, figureFileSmall=504V5uF2Rgm0Ss7QU7RJ3Q==, figureFileBig=lkrg18Of+G7UVStt7DWzDw==, tableContent=null), ArticleFig(id=1327017506180543100, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, language=EN, label=Figure 4, caption=Effect of different minimum sample amounts of DBSCAN on the accuracy of global model, figureFileSmall=dtjs7cMwTS/EpZNjWsufYg==, figureFileBig=w69op7+vWsahNXLfVQ/nbg==, tableContent=null), ArticleFig(id=1327017506247651965, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, language=CN, label=图4, caption=DBSCAN中不同最小样本数对于全局模型准确率的影响, figureFileSmall=dtjs7cMwTS/EpZNjWsufYg==, figureFileBig=w69op7+vWsahNXLfVQ/nbg==, tableContent=null), ArticleFig(id=1327017507912790654, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, language=EN, label=Table 1, caption=Test Accuracy on Fashion-MNIST Under Different Dirichlet Coefficients %, figureFileSmall=null, figureFileBig=null, tableContent=
算法$ \delta $=0.1$ \delta $=0.3
注:“±”前数值表示平均值,“±”后数值表示标准差.
FedAvg[25]69.91±1.8382.25±0.91
FedProx[27]70.13±1.7582.07±0.88
FedRobust[38]71.04±1.3483.14±0.67
SCAFFOLD[28]72.23±1.2284.17±0.59
IFCA[16]71.52±1.0684.02±0.49
FLSC[19]72.07±0.8384.88±0.41
FL-HC[17]71.08±0.9083.65±0.52
FLIS[21]72.21±0.9285.13±0.47
HS-CFA(本文)72.79±0.8185.07±0.36
), ArticleFig(id=1327017508009259647, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, language=CN, label=表1, caption=

不同狄利克雷系数下在Fashion-MNIST的测试准确率

, figureFileSmall=null, figureFileBig=null, tableContent=
算法$ \delta $=0.1$ \delta $=0.3
注:“±”前数值表示平均值,“±”后数值表示标准差.
FedAvg[25]69.91±1.8382.25±0.91
FedProx[27]70.13±1.7582.07±0.88
FedRobust[38]71.04±1.3483.14±0.67
SCAFFOLD[28]72.23±1.2284.17±0.59
IFCA[16]71.52±1.0684.02±0.49
FLSC[19]72.07±0.8384.88±0.41
FL-HC[17]71.08±0.9083.65±0.52
FLIS[21]72.21±0.9285.13±0.47
HS-CFA(本文)72.79±0.8185.07±0.36
), ArticleFig(id=1327017508135088768, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, language=EN, label=Table 2, caption=Test Accuracy on Fashion-MNIST Under Different Rotation Mixing Coefficients %, figureFileSmall=null, figureFileBig=null, tableContent=
算法$ \beta $=30%$ \beta $=50%
注:“±”前数值表示平均值,“±”后数值表示标准差.
FedAvg[25]90.49±0.3291.79±0.29
FedProx[27]89.91±0.1591.81±0.12
FedRobust[38]90.25±0.0892.33±0.07
SCAFFOLD[28]91.84±0.8793.56±0.64
IFCA[16]91.21±0.9292.67±0.70
FLSC[19]90.64±0.1792.30±0.11
FL-HC[17]91.77±0.5392.59±0.25
FLIS[21]91.82.±0.6592.49±0.32
HS-CFA(本文)91.76±0.0992.80±0.13
), ArticleFig(id=1327017508252529281, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, language=CN, label=表2, caption=

不同旋转混合系数下在Fashion-MNIST的测试准确率

, figureFileSmall=null, figureFileBig=null, tableContent=
算法$ \beta $=30%$ \beta $=50%
注:“±”前数值表示平均值,“±”后数值表示标准差.
FedAvg[25]90.49±0.3291.79±0.29
FedProx[27]89.91±0.1591.81±0.12
FedRobust[38]90.25±0.0892.33±0.07
SCAFFOLD[28]91.84±0.8793.56±0.64
IFCA[16]91.21±0.9292.67±0.70
FLSC[19]90.64±0.1792.30±0.11
FL-HC[17]91.77±0.5392.59±0.25
FLIS[21]91.82.±0.6592.49±0.32
HS-CFA(本文)91.76±0.0992.80±0.13
), ArticleFig(id=1327017508323832450, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, language=EN, label=Table 3, caption=Test Accuracy on CIFAR-100 Under Different Dirichlet Coefficients and Rotation Mixing Coefficients %, figureFileSmall=null, figureFileBig=null, tableContent=
算法$ \delta $=0.3$ \beta $=30%
注:“±”前数值表示平均值,“±”后数值表示标准差.
FedAvg[25]53.42±2.0171.89±1.31
FedProx[27]53.73±1.7872.07±1.07
FedRobust[38]55.78±1.2773.14±0.59
SCAFFOLD[28]56.41±1.5874.33±0.62
IFCA[16]49.93±0.7671.02±0.79
FLSC[19]53.15±0.8175.18±0.40
FL-HC[17]52.60±1.0174.29±0.52
FLIS[21]54.29±0.9775.33±0.35
HS-CFA(本文)55.72±0.8475.31±0.16
), ArticleFig(id=1327017508378358403, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, language=CN, label=表3, caption=

不同狄利克雷系数和旋转混合系数下在CIFAR-100的测试准确率

, figureFileSmall=null, figureFileBig=null, tableContent=
算法$ \delta $=0.3$ \beta $=30%
注:“±”前数值表示平均值,“±”后数值表示标准差.
FedAvg[25]53.42±2.0171.89±1.31
FedProx[27]53.73±1.7872.07±1.07
FedRobust[38]55.78±1.2773.14±0.59
SCAFFOLD[28]56.41±1.5874.33±0.62
IFCA[16]49.93±0.7671.02±0.79
FLSC[19]53.15±0.8175.18±0.40
FL-HC[17]52.60±1.0174.29±0.52
FLIS[21]54.29±0.9775.33±0.35
HS-CFA(本文)55.72±0.8475.31±0.16
), ArticleFig(id=1327017508453855876, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, language=EN, label=Table 4, caption=Processing Time of Each Module of HS-CFA on CIFAR-10 Dataset s, figureFileSmall=null, figureFileBig=null, tableContent=
本地训练相似度计算第1层聚合第2层聚合
0.512451.133260.474510.08035
), ArticleFig(id=1327017508512576133, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, language=CN, label=表4, caption=

HS-CFA各个模块在CIFAR-10数据集的处理时间

, figureFileSmall=null, figureFileBig=null, tableContent=
本地训练相似度计算第1层聚合第2层聚合
0.512451.133260.474510.08035
), ArticleFig(id=1327017508600656518, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, language=EN, label=Table 5, caption=Impact of Sampling Amounts on the Module Processing Time of Similarity Computation and Model Accuracy, figureFileSmall=null, figureFileBig=null, tableContent=
采样数目相似度计算处理时间/s模型准确率/%
40.2783170.96
161.1332674.87
643.2455175.43
1288.7981275.51
), ArticleFig(id=1327017508667765383, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, language=CN, label=表5, caption=

采样数目对相似度计算模块处理时间和模型准确率影响

, figureFileSmall=null, figureFileBig=null, tableContent=
采样数目相似度计算处理时间/s模型准确率/%
40.2783170.96
161.1332674.87
643.2455175.43
1288.7981275.51
)], attaches=null, journal=Journal(id=1324759606947315908, delFlag=0, nameCn=计算机研究与发展, nameEn=Journal of Computer Research and Development, nameHistory1=null, nameHistory2=null, issn=1000-1239, eissn=null, cn=11-1777/TP, coden=null, periodic=0, language=CN, oaType=null, ccby=null, superviseOffice=null, ownerOffice=null, pubOffice=null, editorOffice=null, officeType=null, aims=null, clcCode=null, officeProv=null, officeCity=null, officeAddr=null, officeZip=null, officeEmail=null, officePhone=null, editDirector=null, officeDirector=null, officeDirectorPhone=null, officeStaffNum=null, officeEmpNum=null, coverPicUrl=k2l8zgqwstuxxevFX2qxVA==, journalPrice=null, startedYear=null, abbrevIsoEn=Journal of Computer Research and Development, journalRemark=null, publicationField=null, createdTime=1793776903141, updatedTime=1793850698612, createdBy=18614031015, updatedBy=13701087609, firstLetterCn=J, firstLetterEn=J, subjectCode=Electronic Information, subjectName=null, subjectCodeEn=Electronic Information, subjectNameEn=null, picCn=k2l8zgqwstuxxevFX2qxVA==, picEn=r6k0Ry+RD+c/VXj5QarmDw==, jcr=null, cjcr=null, exts=[JournalExt(id=1325069127710286688, language=CN, name=计算机研究与发展, nameHistory1=null, nameHistory2=null, managedBy=, sponsoredBy=, publishedBy=, editorOffice=, officeProv=null, officeCity=null, officeAddr=, officeZip=, editDirector=, officeDirector=null, officePhone=null, coverPicUrl=null, journalRemark=, submitArticleUrl=null, websiteUrl=, createdTime=1793850698640, updatedTime=1793850698640, createdBy=13701087609, updatedBy=13701087609, submissionGuidelinesUrl=, submissionAuthorUrl=https://crad.ict.ac.cn/journalx/authorLogOn.action, submissionEditorUrl=https://crad.ict.ac.cn/journalx/editorLogOn.action, submissionReviewUrl=https://crad.ict.ac.cn/journalx/expertLogOn.action, submissionCeEditorUrl=https://crad.ict.ac.cn/journalx/editorInChiefLogOn.action, submissionAeEditorUrl=, option={"copyright":""}), JournalExt(id=1325069127777395553, language=EN, name=Journal of Computer Research and Development, nameHistory1=null, nameHistory2=null, managedBy=, sponsoredBy=, publishedBy=, editorOffice=, officeProv=null, officeCity=null, officeAddr=, officeZip=, editDirector=, officeDirector=null, officePhone=null, coverPicUrl=null, journalRemark=, submitArticleUrl=null, websiteUrl=, createdTime=1793850698656, updatedTime=1793850698656, createdBy=13701087609, updatedBy=13701087609, submissionGuidelinesUrl=, submissionAuthorUrl=https://crad.ict.ac.cn/journalx/authorLogOn.action, submissionEditorUrl=https://crad.ict.ac.cn/journalx/editorLogOn.action, submissionReviewUrl=https://crad.ict.ac.cn/journalx/expertLogOn.action, submissionCeEditorUrl=https://crad.ict.ac.cn/journalx/editorInChiefLogOn.action, submissionAeEditorUrl=, option={"copyright":""})], databaseList=null, tenantJournalId=1324759914146529478, websiteList=[Website(id=1325070300139892771, webName=null, webTitle=null, webDomain=null, webCopyrigh=null, webIpcNo=null, seoTitle=null, seoKeywords=null, seoDescription=null, tenantJournalId=null, journalId=1324759914146529478, journalNameCn=null, journalNameEn=null, grayFlag=null, tenantId=1146029695717560320, platformId=null, journalGroupId=null, journalGroupNameCn=null, journalGroupNameEn=null, type=1, domain=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN, language=CN, createTime=1793850978169, createBy=18614031015, updateTime=1793851040259, updateBy=18614031015, name=计算机研究与发展-中文, tplId=1146099689490845704, title=计算机研究与发展, delFlag=0, indexPage=/home, props=[WebsiteProps(id=1325070893751349360, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=articleTextType, value=kx, createTime=1793851119697, updateTime=1793851119697, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893730377837, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=banner, value=null, createTime=1793851119692, updateTime=1793851119692, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893772320883, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=grayFlag, value=0, createTime=1793851119702, updateTime=1793851119702, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893721989228, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=logo, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/file/pic?fileId=U27tDsy717YF7XSPA+Vwdw==, createTime=1793851119690, updateTime=1793851119690, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893784903797, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=minRunFlag, value=0, createTime=1793851119705, updateTime=1793851119705, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893742960751, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=picServerUrl, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/file/pic, createTime=1793851119695, updateTime=1793851119695, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893780709492, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=silenceFlag, value=0, createTime=1793851119704, updateTime=1793851119704, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893738766446, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=staticResourcePath, value=https://castjournals.cast.org.cn/joweb/cast_kjdb_cn_619/, createTime=1793851119694, updateTime=1793851119694, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893755543665, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=themeColor, value=null, createTime=1793851119698, updateTime=1793851119698, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893763932274, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=themeStyle, value=null, createTime=1793851119700, updateTime=1793851119700, creator=18614031015, updator=18614031015)]), Website(id=1325070300580294698, webName=null, webTitle=null, webDomain=null, webCopyrigh=null, webIpcNo=null, seoTitle=null, seoKeywords=null, seoDescription=null, tenantJournalId=null, journalId=1324759914146529478, journalNameCn=null, journalNameEn=null, grayFlag=null, tenantId=1146029695717560320, platformId=null, journalGroupId=null, journalGroupNameCn=null, journalGroupNameEn=null, type=1, domain=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN, language=EN, createTime=1793850978274, createBy=18614031015, updateTime=1793851036638, updateBy=18614031015, name=计算机研究与发展-英文, tplId=1146101810881728533, title=Journal of Computer Research and Development, delFlag=0, indexPage=/home, props=[WebsiteProps(id=1325070925288321148, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=articleTextType, value=kx, createTime=1793851127216, updateTime=1793851127216, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925263155321, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=banner, value=null, createTime=1793851127210, updateTime=1793851127210, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925309292671, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=grayFlag, value=0, createTime=1793851127221, updateTime=1793851127221, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925254766712, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=logo, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/file/pic?fileId=U27tDsy717YF7XSPA+Vwdw==, createTime=1793851127208, updateTime=1793851127208, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925326069889, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=minRunFlag, value=0, createTime=1793851127225, updateTime=1793851127225, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925279932539, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=picServerUrl, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/file/pic, createTime=1793851127214, updateTime=1793851127214, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925317681280, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=silenceFlag, value=0, createTime=1793851127223, updateTime=1793851127223, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925271543930, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=staticResourcePath, value=https://castjournals.cast.org.cn/joweb/cast_kjdb_en_623/, createTime=1793851127212, updateTime=1793851127212, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925292515453, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=themeColor, value=null, createTime=1793851127217, updateTime=1793851127217, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925300904062, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=themeStyle, value=null, createTime=1793851127219, updateTime=1793851127219, creator=18614031015, updator=18614031015)])], journalTitle=计算机研究与发展, weixinUrl=null, journalUrl=https://crad.ict.ac.cn/, iacademicId=null, status=1, seqNo=null, journalTitleEn=Journal of Computer Research and Development, journalPhotoCn=k2l8zgqwstuxxevFX2qxVA==, journalPhotoEn=r6k0Ry+RD+c/VXj5QarmDw==, journalFirstLetter=J, journalRecommend=null, journalNew=null, journalCollection=null, jcrJf=null, cjcrJf=null, jcrJfStr=null, cjcrJfStr=null, submissionFirstDecision=null, sciSubjectClassification=null, casSubjectClassification=null, citeScore=null, totalCitationFrequency=null, icpCode=null, psCode=null, advertisingLicenseCode=null, copyrightInformation=null, country=null, option=, provinceCode=null, provinceName=null, collectFlag=false, interPubPlatform=, interPubPlatformUrl=null), detailUrlCn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/10.7544/issn1000-1239.202440969, detailUrlEn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/10.7544/issn1000-1239.202440969, pdfUrlCn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/PDF/10.7544/issn1000-1239.202440969, pdfUrlEn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/PDF/10.7544/issn1000-1239.202440969, aliStartDate=0, aliEndDate=0, collectionFlag=false, citedCount=null, citedUrl=null, previewStatus=0, delFlag=0, hasFullText=1, orderTime=1764518400000, fullTextJson=null, articleText=null, reference=null)
收藏切换
非独立同分布数据下层次化Sinkhorn距离聚类联邦学习算法
赵瑞 , 段小文 , 刘新 , 周睿 , 周庆国 *
计算机研究与发展 | 人工智能 2025,62(12): 3002-3012
收起
收藏切换
计算机研究与发展 |人工智能 2025 , 62 (12) : 3002 -3012
非独立同分布数据下层次化Sinkhorn距离聚类联邦学习算法
全屏
[Author(id=1327017501789106772, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, orderNo=0, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=zhaor19@lzu.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327017503429079638, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, authorId=1327017501789106772, language=EN, stringName=Rui Zhao, firstName=Rui, middleName=null, lastName=Zhao, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=School of Information Science and Engineering, Lanzhou University, Lanzhou 730030, bio={"content":"

born in 1989. PhD candidate. Student member of CCF. His main research interests include distributed neural networks and vehicular-roadside cooperation

"}, bioImg=null, bioContent=

born in 1989. PhD candidate. Student member of CCF. His main research interests include distributed neural networks and vehicular-roadside cooperation

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327017503508771415, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, authorId=1327017501789106772, language=CN, stringName=赵瑞, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=兰州大学信息科学与工程学院 兰州 730030, bio={"img":"5z792S3Cve2MqrrSDhWHgQ==","content":"

1989年生. 博士研究生. CCF学生会员. 主要研究方向为分布式神经网络、车路协同

"}, bioImg=5z792S3Cve2MqrrSDhWHgQ==, bioContent=

1989年生. 博士研究生. CCF学生会员. 主要研究方向为分布式神经网络、车路协同

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327017501692637776, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, xref=null, ext=[AuthorCompanyExt(id=1327017501701026385, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, companyId=1327017501692637776, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=School of Information Science and Engineering, Lanzhou University, Lanzhou 730030), AuthorCompanyExt(id=1327017501713609298, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, companyId=1327017501692637776, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=兰州大学信息科学与工程学院 兰州 730030)])]), Author(id=1327017503584268889, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, orderNo=1, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327017503672349275, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, authorId=1327017503584268889, language=EN, stringName=Xiaowen Duan, firstName=Xiaowen, middleName=null, lastName=Duan, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=School of Information Science and Engineering, Lanzhou University, Lanzhou 730030, bio={"content":"

born in 2000. Master candidate. Her main research interest includes federated learning algorithm optimization

"}, bioImg=null, bioContent=

born in 2000. Master candidate. Her main research interest includes federated learning algorithm optimization

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327017503768818268, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, authorId=1327017503584268889, language=CN, stringName=段小文, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=兰州大学信息科学与工程学院 兰州 730030, bio={"img":"GK7ReiKCk/dvDzUs14n9Tw==","content":"

2000年生. 硕士研究生. 主要研究方向为联邦学习算法优化

"}, bioImg=GK7ReiKCk/dvDzUs14n9Tw==, bioContent=

2000年生. 硕士研究生. 主要研究方向为联邦学习算法优化

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327017501692637776, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, xref=null, ext=[AuthorCompanyExt(id=1327017501701026385, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, companyId=1327017501692637776, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=School of Information Science and Engineering, Lanzhou University, Lanzhou 730030), AuthorCompanyExt(id=1327017501713609298, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, companyId=1327017501692637776, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=兰州大学信息科学与工程学院 兰州 730030)])]), Author(id=1327017503869481566, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, orderNo=2, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327017503957561952, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, authorId=1327017503869481566, language=EN, stringName=Xin Liu, firstName=Xin, middleName=null, lastName=Liu, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=School of Information Science and Engineering, Lanzhou University, Lanzhou 730030, bio={"content":"

born in 1995. PhD, associate professor. Member of CCF. His main research interests include cybersecurity and artificial intelligence

"}, bioImg=null, bioContent=

born in 1995. PhD, associate professor. Member of CCF. His main research interests include cybersecurity and artificial intelligence

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327017504045642337, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, authorId=1327017503869481566, language=CN, stringName=刘新, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=兰州大学信息科学与工程学院 兰州 730030, bio={"img":"9Jgtoacm0M3K0pOijTeNdQ==","content":"

1995年生. 博士,副教授. CCF会员. 主要研究方向为网络安全、人工智能

"}, bioImg=9Jgtoacm0M3K0pOijTeNdQ==, bioContent=

1995年生. 博士,副教授. CCF会员. 主要研究方向为网络安全、人工智能

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327017501692637776, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, xref=null, ext=[AuthorCompanyExt(id=1327017501701026385, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, companyId=1327017501692637776, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=School of Information Science and Engineering, Lanzhou University, Lanzhou 730030), AuthorCompanyExt(id=1327017501713609298, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, companyId=1327017501692637776, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=兰州大学信息科学与工程学院 兰州 730030)])]), Author(id=1327017504146305635, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, orderNo=3, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327017504225997413, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, authorId=1327017504146305635, language=EN, stringName=Rui Zhou, firstName=Rui, middleName=null, lastName=Zhou, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=School of Information Science and Engineering, Lanzhou University, Lanzhou 730030, bio={"content":"

born in 1981. PhD, professor. Senior member of CCF. His main research interests include distributed real-time systems and artificial intelligence

"}, bioImg=null, bioContent=

born in 1981. PhD, professor. Senior member of CCF. His main research interests include distributed real-time systems and artificial intelligence

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327017504326660710, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, authorId=1327017504146305635, language=CN, stringName=周睿, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=兰州大学信息科学与工程学院 兰州 730030, bio={"img":"tB5ZLluBNY+E1rkL3tHIAQ==","content":"

1981年生. 博士,教授. CCF高级会员. 主要研究方向为分布式实时系统、人工智能

"}, bioImg=tB5ZLluBNY+E1rkL3tHIAQ==, bioContent=

1981年生. 博士,教授. CCF高级会员. 主要研究方向为分布式实时系统、人工智能

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327017501692637776, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, xref=null, ext=[AuthorCompanyExt(id=1327017501701026385, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, companyId=1327017501692637776, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=School of Information Science and Engineering, Lanzhou University, Lanzhou 730030), AuthorCompanyExt(id=1327017501713609298, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, companyId=1327017501692637776, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=兰州大学信息科学与工程学院 兰州 730030)])]), Author(id=1327017504410546792, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, orderNo=4, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=zhouqg@lzu.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=1, authorType=1, ext={EN=AuthorExt(id=1327017504515404394, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, authorId=1327017504410546792, language=EN, stringName=Qingguo Zhou, firstName=Qingguo, middleName=null, lastName=Zhou, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=*, address=School of Information Science and Engineering, Lanzhou University, Lanzhou 730030, bio={"content":"

born in 1973. PhD, professor. Senior member of CCF. His main research interests include distributed and embedded real-time systems, neural networks, and intelligent transportation

"}, bioImg=null, bioContent=

born in 1973. PhD, professor. Senior member of CCF. His main research interests include distributed and embedded real-time systems, neural networks, and intelligent transportation

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327017504595096171, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, authorId=1327017504410546792, language=CN, stringName=周庆国, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=*, address=兰州大学信息科学与工程学院 兰州 730030, bio={"img":"iiBMQmp/Uia7AOhP8CtOBw==","content":"

1973年生. 博士,教授. CCF高级会员. 主要研究方向为分布式与嵌入式实时系统、神经网络、智慧交通

"}, bioImg=iiBMQmp/Uia7AOhP8CtOBw==, bioContent=

1973年生. 博士,教授. CCF高级会员. 主要研究方向为分布式与嵌入式实时系统、神经网络、智慧交通

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327017501692637776, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, xref=null, ext=[AuthorCompanyExt(id=1327017501701026385, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, companyId=1327017501692637776, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=School of Information Science and Engineering, Lanzhou University, Lanzhou 730030), AuthorCompanyExt(id=1327017501713609298, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017497221509691, companyId=1327017501692637776, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=兰州大学信息科学与工程学院 兰州 730030)])])]
赵瑞 , 段小文, 刘新, 周睿, 周庆国*
作者信息
  • 兰州大学信息科学与工程学院 兰州 730030
通讯作者:
周庆国()
作者简介:

1989年生. 博士研究生. CCF学生会员. 主要研究方向为分布式神经网络、车路协同

2000年生. 硕士研究生. 主要研究方向为联邦学习算法优化

1995年生. 博士,副教授. CCF会员. 主要研究方向为网络安全、人工智能

1981年生. 博士,教授. CCF高级会员. 主要研究方向为分布式实时系统、人工智能

1973年生. 博士,教授. CCF高级会员. 主要研究方向为分布式与嵌入式实时系统、神经网络、智慧交通

Hierarchical Sinkhorn Distance-Based Clustering Federated Learning Algorithm for Non-IID Data
Rui Zhao , Xiaowen Duan, Xin Liu, Rui Zhou, Qingguo Zhou*
Affiliations
  • School of Information Science and Engineering, Lanzhou University, Lanzhou 730030
  • born in 1989. PhD candidate. Student member of CCF. His main research interests include distributed neural networks and vehicular-roadside cooperation

    born in 2000. Master candidate. Her main research interest includes federated learning algorithm optimization

    born in 1995. PhD, associate professor. Member of CCF. His main research interests include cybersecurity and artificial intelligence

    born in 1981. PhD, professor. Senior member of CCF. His main research interests include distributed real-time systems and artificial intelligence

    born in 1973. PhD, professor. Senior member of CCF. His main research interests include distributed and embedded real-time systems, neural networks, and intelligent transportation

出版时间: 2025-12-01 doi: 10.7544/issn1000-1239.202440969
文章导航
收藏切换

联邦学习作为一种边缘计算中的新兴分布式神经网络训练方法面临客户端数据异构性挑战,其中聚类联邦学习被认为是一种颇具潜力的解决方案,然而现有聚类联邦学习算法未深入探究量化客户端数据分布差异. 针对该问题提出了一种新颖的层次化Sinkhorn距离聚类联邦学习算法(hierarchical Sinkhorn distance-based clustering federated learning algorithm,HS-CFA),采用最优传输代价衡量局部训练时客户端数据分布特性,提出层次化聚类策略动态调整全局模型聚合时的参与权重. 具体而言,HS-CFA引入Sinkhorn距离量化客户端间的分布差异,提出使用基于密度的聚类算法在服务器端进行动态层次聚类. 在多个基准数据集上的实验结果表明,相比传统算法,所提算法在高度数据分布异构性的场景中显著提升了全局模型的准确率和鲁棒性.

边缘计算  /  联邦学习  /  最优传输  /  数据异构性  /  聚类优化

Federated learning, as an emerging distributed neural network training paradigm in edge computing, faces a key challenge of data heterogeneity over clients, which significantly impacts global model performance. While clustering federated learning, serving as a promising solution, has shown improvements in model accuracy, its effectiveness is limited by the insufficient utilization of local data distribution statistics. To address this issue, we propose a novel federated learning framework named HS-CFA (hierarchical Sinkhorn distance-based clustering federated algorithm), which is designed to optimize the aggregation weights of local models under the heterogeneous data environments, thereby enhancing the performance of the global model. It utilizes the entropy-regularized optimal transport cost to capture the characteristics of local data distribution and dynamically adjusts the aggregation weights of local models with a hierarchical clustering strategy. Specially, HS-CFA employs the Sinkhorn distance as a lightweight optimal transport cost metric to measure distributional dissimilarities across clients. Furthermore, it adopts a hierarchical two-layer clustering mechanism, combining density-based spatial clustering and average aggregation during the server training phase, facilitating the dynamic and adaptive adjustment of local model aggregation weights. Experimental results on multiple benchmark datasets demonstrate that HS-CFA significantly enhances the accuracy and robustness of the global model in scenarios with highly heterogeneous distribution settings.

edge computing  /  federated learning  /  optimal transport  /  data heterogeneity  /  clustering optimization
赵瑞, 段小文, 刘新, 周睿, 周庆国. 非独立同分布数据下层次化Sinkhorn距离聚类联邦学习算法. 计算机研究与发展, 2025 , 62 (12) : 3002 -3012 . DOI: 10.7544/issn1000-1239.202440969
Rui Zhao, Xiaowen Duan, Xin Liu, Rui Zhou, Qingguo Zhou. Hierarchical Sinkhorn Distance-Based Clustering Federated Learning Algorithm for Non-IID Data[J]. Journal of Computer Research and Development, 2025 , 62 (12) : 3002 -3012 . DOI: 10.7544/issn1000-1239.202440969
随着物联网设备的广泛使用,大量碎片化数据的产生推动了分布式机器学习的研究发展[1]. 传统的数据中心采用集中式架构,即数据中心汇总处理经由物联网设备采集传输的数据,并依靠自身计算资源训练神经网络模型. 然而在有限通信资源下,该架构会因海量分布式数据的传输导致高延迟问题. 其次数据中心作为单点存储和计算节点,在遭遇网络攻击时,可能存在大规模的隐私数据泄露风险[2]. 更严重的是一旦服务器模型推理异常,导致物联网智能应用的全面瘫痪[3],影响所有连接的边缘设备应用.
针对上述问题,边缘计算[4-5]提出将数据处理存储前置到物联网边缘节点而非集中式在数据中心处理的优化方案,从而提高数据处理效率. 然而由于神经网络模型学习依赖大量训练数据,而边缘设备通常面临数据不足和计算能力受限[6]问题,导致单一边缘设备节点难以训练高性能模型. 为了进一步应对该挑战,2016年谷歌团队提出了分布式机器学习方法——联邦学习(federated learning,FL)[7],该训练范式允许多个边缘节点[8-9]作为客户端利用本地数据协作训练目标模型而无需共享原始数据,通过上传目标模型梯度或参数在服务器端采用聚合操作进行模型更新. 该方法不仅有效减少了传统分布式训练因数据传输带来的通信开销,同时也在保障边缘设备数据隐私的基础上,提升了目标模型性能.
然而实际应用场景中,联邦学习往往难以满足其基础假设,即各客户端的本地训练数据应保持独立同分布特性. 例如,边缘节点在数据采集过程中,由于设备的使用环境、用户行为模式以及数据来源的不同,其数据在分布、标签以及规模上可能存在显著差异,这导致本地数据呈现非独立同分布(non-independent and identically distributed,non-IID)特性[10]. 这一特性会引发联邦学习中目标模型训练过程收敛缓慢、模型泛化性能不佳等问题. 因此数据异构性是联邦学习面临的主要挑战,一方面客户端由于自身数据有限,难以单独训练高性能模型,需依赖多个客户端协作学习;另一方面,与数据分布差异较大的客户端进行协作时,目标模型泛化精度难以得到保证. 针对该问题,已有相关研究提出了多种应对策略[11-15],其中聚类联邦学习(clustered federated learning,CFL)[16-21]被认为是一种颇具潜力的算法,通过对相似数据分布的客户端模型聚类,减少分布差异性,提升聚合时目标模型的泛化能力. 尽管现有CFL研究取得了诸多进展,但相关研究未充分考虑训练时聚合模型的动态性,从而假设聚类簇的数量为固定的先验知识,缺乏自适应调整机制. 其次对于数据分布差异这一关键度量,现有算法对其缺乏深入研究,难以精准量化客户端数据分布异构性.
本文提出了一种面向non-IID分布的层次化Sinkhorn距离聚类联邦学习算法(hierarchical Sinkhorn distance-based clustering federated learning algorithm,HS-CFA). 该算法挖掘各客户端的数据分布差异,通过2层聚类机制提升联邦学习全局模型的性能. 具体而言,HS-CFA将数据分布差异视为分布之间的最优传输代价,采用熵正则化的轻量Sinkhorn距离[22]将该代价量化为客户端的相似度度量. 基于该度量,HS-CFA提出了一种高效简单的聚类策略,利用DBSCAN(density-based spatial clustering of applications with noise)聚类算法[23]将具有相似数据特征的本地模型聚类为簇,簇内采用平均聚合生成簇模型. 随后对簇模型聚类生成全局模型. 通过该聚类策略,HS-CFA实现了自适应调节本地模型聚合权重,有效提升了全局模型的泛化能力,加速联邦学习的收敛过程.
本文主要贡献有3点:
1)基于概率分布的最优传输[24]理论,HS-CFA引入Sinkhorn距离作为熵正则化最优传输代价,衡量本地数据分布之间的差异,从而解决了服务器端无法有效表征客户端数据分布特性的问题,同时降低了最优传输代价的计算复杂度.
2)设计了服务器端双层聚合机制,在无需先验聚类知识的条件下,利用客户端本地训练时与全局模型的相似度度量进行聚类,同时排除离群本地模型带来的干扰,该机制能动态平衡本地模型聚合权重.
3)在高度non-IID数据异构场景中对HS-CFA进行了大量实验,结果表明HS-CFA在目标模型精度与鲁棒性方面均优于现有的聚类联邦学习算法,在高数据异构场景中表现尤为突出.
联邦学习是一种新兴的分布式机器学习方法,由Mcmahan等人[25]提出的FedAvg算法是该领域的奠基性工作,该算法在服务器端训练阶段对上传的本地模型进行加权平均聚合,以迭代更新全局模型. 然而在面对non-IID数据分布时,FedAvg算法的精度会显著下降,甚至出现模型无法收敛的问题. 文献[26]进一步指出该问题的根源在于本地模型优化目标与全局模型优化目标的不一致性.
为解决这一问题,近年相关研究提出了多种轻量级优化算法,根据优化目标可分为2类:
一类是为了缓解本地模型数据分布导致的不一致性,对客户端训练过程的目标损失函数优化. 例如,FedProx[27]和SCAFFOLD [28]通过在本地模型的目标函数中加入正则惩罚项来对齐全局模型和本地模型;FedCM[29]和FedSpeed[30]则利用乘子交替方向法[31]在本地模型的损失函数中加入约束条件.
另一类算法则在服务器端聚合本地模型时加入聚类算法从而优化聚合权重分配策略. Sattler等人[20]首次提出CFL算法,通过迭代二分法将具有相似数据分布的本地模型进行分簇,提高簇内模型的性能,这种算法显著减轻了数据异质性的负面影响,从而提高了模型的准确性和收敛速度. Ghosh等人[16]提出 IFCA算法预先设定聚类簇,增强了联邦学习簇模型个性化能力. FLSC[19]采用松弛聚类的算法,在IFCA算法的基础上对聚类不进行严格划分,提升了个性化联邦学习效果. 然而IFCA和FLSC性能依赖于多个个性化簇模型,因此在生成具备泛化能力的单一全局模型方面存在局限性. 分层联邦学习算法FL-HC[17]组织客户端进行分层聚类来提高模型的准确性;文献[21]提出了FLIS算法,利用服务器端的代理数据集测量本地模型的相似度从而优化聚合权重. 分析现有聚类联邦学习算法,优化的关键点集中在聚类簇的划分策略以及聚类相似度度量指标的选择上.
因此与现有研究的区别在于,本文研究聚焦单一目标模型的聚类联邦学习算法,为non-IID场景下提供了一种双层聚类优化策略. 同时与现有研究方法中采用模型损失、欧几里得距离或余弦距离[32]来衡量客户端数据分布的差异不同,本文利用熵正则化的最优传输代价动态衡量数据分布相似度,进一步为了降低计算复杂度,提出使用轻量化的 Sinkhorn 距离进行高效计算,从而在保证相似度度量精度的同时,提升全局模型的训练效率和收敛性能.
经典联邦学习算法[25]用于分布式协作训练一个全局泛化模型. 算法可以概括为在一个全局训练周期中进行4个步骤:1)服务器将全局模型分发至客户端;2)客户端利用本地数据训练;3)客户端将训练后的模型参数或梯度上传至服务器;4)服务器通过加权聚合的方式进行全局训练. 通过不断迭代全局训练周期直至模型收敛.
算法1. 联邦学习算法流程.
输入:全局初始化模型$ {\theta }_*^{\,0} $,客户端总数$ N $,客户端与服务器通信轮次$ T $,客户端训练轮次$ K $;
输出:全局模型$ {\theta }_*^{\,T} $.
① 初始化全局模型参数$ {\theta }_*^{\,0} $;
② for $ t $← 0 to $ T $ do
③ 随机选择包含$ M $个客户端集合;
④ 发送 $ {\theta }_*^{\,0} $到$ M $;
⑤ 每个客户端$ i\in {M}^{t} $并行执行;
⑥  使用本地数据集进行$ E $次训练;
⑦  更新客户端模型参数$ {\theta }_{i}^{\,t} $至服务器端;
⑧ 服务器执行$ {\theta }_*^{\,t+1}=\dfrac{1}{\left|M\right|}\displaystyle\sum _{i\in M}{\theta }_{K}^{\,t}; $
⑨ end for
⑩ 返回$ {\theta }_*^{\,T}. $
算法1描述联邦学习的基础训练范式,可概括为客户端训练周期和服务器训练周期. 客户端训练周期中,客户端模型利用本地数据训练,其目标优化函数表示为
$ {F}_{i}\left(\theta \right)={{\mathbb E}}_{\left(x,y\right)\sim{D}_{i}}\left[\ell \left(\theta ;x,y\right)\right]\text{,} $
其中$ \ell (\theta ;{x},{y}) $是损失函数,$ {D}_{i} $是第$ i $个客户端的数据分布,$ {\mathbb E} $表示对客户端数据分布的期望.
在服务器训练周期中,由于无法获得客户端数据,全局模型只能采用间接训练的方式,通常采用最小化全局目标函数$ F\left(\theta \right) $,该目标函数是所有客户端数据分布的总体损失函数,其表示为:
$ F\left(\theta \right)=\dfrac{1}{\left|{S}_{i}\right|}\displaystyle\sum _{z=1}^{{S}_{i}}{{\mathbb E}}_{\left(x,y\right)\sim{D}_{i}}\left[\ell \left(\theta ;{x}_{z},{y}_{z}\right)\right]\text{,} $
其中$ {S}_{i} $是第$ i $个客户端的样本数据数量,$ ({x}_{z},{y}_{z}) $则是第$ z $个样本. 事实上联邦学习服务器采用的训练方式是分布式随机梯度下降算法,将不同客户端上传的模型看待为大小不一的批处理训练结果,在此基础上进行加权期望,权重为不同客户端训练数据集大小. 然而分布式随机梯度下降算法的前提是数据由服务器进行采样分发,因此分布式数据通常满足独立同分布特性,因而分布式训练局部优化目标一致. 但在联邦训练范式中,分布式数据是客户端自有数据,不同客户端数据之间存在异构性:1)数据量分布偏差,即相同标签的数据数量存在显著差异;2)标签分布偏差,数据标签在客户端之间分布不一致. 这种non-IID的特性导致客户端分布式训练时的优化目标存在差异,进一步在服务器利用分布式随机梯度下降进行聚合时,会显著影响全局模型泛化性能,导致客户端数据异构性越强,全局模型泛化精度越低.
最优传输理论[22,24]提供了一种在2个概率测度之间进行操作的数学方法,即通过定义代价函数,如式(3)所示.
$ c:X\times Y\to {\mathbb{R}} \cup \left\{\mathbb{\infty }\right\}. $
最优传输希望找到一个传输代价使得从$ x\in X $转移到$ y\in Y $的总传输代价最小,$ c\left(x,y\right) $表示将$ x\in X $转移到$ y\in Y $的代价,如式(4)所示.
$ \underset{T}{\mathrm{min}}M(T)={\int }_{X}^{}c\left(x,T\left(x\right)\right)\mathrm{d}\mu \left(x\right)\text{,} $
其中概率测度$ \mu \in P\left(X\right) $,概率测度$ v\in P\left(Y\right) $,因此$ v= T\times \mu $是从$ x\in X $转移到$ y\in Y $的测度. 最优传输理论中的Wasserstein距离,即是量化一个概率分布转换为另一个分布所需的最优传输代价. 该测度被广泛使用到了深度学习的生成网络模型损失函数设计中,度量生成样本与已有样本之间的相似度,对样本空间的数据分布进行学习.
然而直接在较大规模的数据分布上计算Wasserstein距离代价较高,通常引入熵正则化项减少计算成本:
$ objective\left(\pi \right)={\int }_{X\times Y}c\left(x,y\right)\mathrm{d}\left(x,y\right)+\gamma Ent\left(\pi \right)\text{,} $
其中$ \pi $为最优传输代价,$ \gamma $为控制权重,熵正则化项$ Ent\left(\pi \right) $定义为
$ Ent\left(\pi \right)={\int }_{X\times Y}\mathrm{l}\mathrm{o}\mathrm{g}\dfrac{\mathrm{d}\pi \left(x,y\right)}{\mathrm{d}\mu \left(x\right)\mathrm{d}\nu \left(y\right)}\mathrm{d}\pi \left(x,y\right). $
为了进一步加快计算,对目标函数对偶化得到Sinkhorn距离[22],这是一种快速计算求解带有熵正则化的最优传输问题的迭代方法,利用矩阵的归一化迭代逼近最优传输测度.
$ \begin{split} S inkhorn\left(\mu ,v\right)= &{\mathrm{i}\mathrm{n}\mathrm{f}}_{\left(\mu ,v\right)}{\int }_{X\times Y}c\left(x,y\right){\mathrm{e}}^{\mu \left(x\right)+v\left(y\right)}\mathrm{d}\mu \left(x\right)\mathrm{d}v\left(y\right)-\\&\dfrac{1}{\gamma }\left({\int }_{X}{\mathrm{e}}^{\mu \left(x\right)}\mathrm{d}\mu \left(x\right)-1\right)- \\&\dfrac{1}{\gamma }\left({\int }_{Y}{\mathrm{e}}^{v\left(y\right)}\mathrm{d}v\left(y\right)-1\right).\\[-1pt] \end{split}$
Sinkhorn-Knopp[20]算法通过对$ \mu $和$ v $交替更新的方式,最终得到一个收敛的$ S inkhorn\left(\mu ,v\right) $,更新计算公式如下:
$ {\mu }^{k+1}=\mathrm{log}\left(\dfrac{1}{\mu \left(x\right)}{\int }_{X}{\mathrm{e}}^{{\mathrm{\nu }}^{k}\left(y\right)-c\left(x,y\right)}\right)\mathrm{d}\nu \left(y\right)\text{,} $
$ {\nu }^{k+1}=\mathrm{log}\left(\dfrac{1}{\nu \left(x\right)}{\int }_{X}{\mathrm{e}}^{{\mu }^{k+1}\left(x\right)-c\left(x,y\right)}\right)\mathrm{d}\mu \left(x\right). $
联邦学习客户端数据私有且non-IID特性使得其存在2个显著限制:1)服务器端无法获取客户端数据进行计算;2)客户端无法获取其他客户端模型信息进行优化. 而本文提出基于 Sinkhorn 距离的层次化聚类联邦学习算法有效缓解上述问题并优化模型聚合过程.
如图1中间区域所示,HS-CFA动态评估客户端数据分布的统计特性,采用最优传输代价来量化 non-IID 客户端数据分布. 在每个全局通信轮次中,客户端利用Sinkhorn 距离计算客户端模型与全局模型之间的差异,客户端上传本地模型及相似性度量至服务器. 如图1右侧区域所示,服务器端间接利用本地模型与全局模型的相似度度量描述各客户端数据分布之间的差异性. 使用层次化聚类方式进行聚合,第1层聚类利用 DBSCAN 算法基于相似度度量生成聚类簇模型;第2层采用平均聚合对聚类簇模型聚类,生成全局模型. 随后服务器将聚类簇模型和全局模型下发至客户端进行迭代训练.
HS-CFA在客户端训练周期中通过度量本地模型与全局模型之间的差异性,间接地在不违反数据隐私的前提下向服务器提供客户端数据分布特性.
算法2. HS-CFA客户端训练.
输入:全局模型$ {\theta }_* $,聚类簇模型$ \bar \theta_{c}^{\,t} $,聚类簇数$ j $,局部数据$ \left(x,y\right)\sim{D}_{i} $;
输出:本地模型$ {\theta }_{i} $,数据分布相似度$ {S im}_{i} $,客户端标识$ i $.
/*$ T $为客户端训练和全局模型通信轮次;
$ K $为客户端训练轮次;
$ \eta $为客户端模型训练学习率;
$ {F}_{i} $为客户端训练损失函数. */
① for $ t $←0 to T do
② if $ {\theta }_{c} $ 不存在
③   $ \bar \theta_{c}^{\,t} $ = $ {\theta }_*^{\,t} $,$ {\theta }_{i}^{\,t}={\theta }_*^{\,t} $;
④ else
⑤   $ {\theta }_{i}^{\,t}={\theta }_*^{\,t}-\dfrac{1}{j-1}\left(\bar \theta_{c}^{\,t}\right) $;
⑥ end if
⑦ for k ←1 to K do
⑧   $ {\theta }_{i,k+1}^{t} $ ← $ {\theta }_{i,k}^{t}–\eta \nabla {F}_{i}(\theta |\left(x,y\right)\sim{D}_{i}) $;
⑨ end for
⑩  $ {S im}_{i}=S inkhorn\left({\theta }_{i,K}^{t},{\theta }_*^{\,t}|\left(x,y\right)\sim{D}_{i}\right) $;
⑪ {$ {\theta }_{i,K}^{t},{S im}_{i},i $}发送至服务器;
⑫ end for
算法2描述了图1中HS-CFA客户端训练过程. 考虑到本地模型对于全局数据分布未知,初始模型$ {\theta }_*^{\,t} $对所有客户端数据分布具备相同的度量尺度,因此假设初始全局模型和客户端经过K轮训练后的模型是2个函数表示,则对局部数据分布采样计算得到相关函数的值域分布,可间接地表示客户端数据分布对于全局模型和本地模型的差异性. 具体在联邦学习的每一轮中,客户端通过对训练局部轮次$ K $后产生的本地模型$ {\theta }_{i,K}^{t} $和初始全局模型$ {\theta }_*^{\,t} $根据式(8)(9)交替迭代计算Sinkhorn距离,获得相似度度量$ {S im}_{i}. $
此外与其他算法相比,HS-CFA要求服务器传输:全局模型$ {\theta }_* $和该客户端所在的聚类簇模型$ \bar \theta_{c} $两个模型以及全局聚类时产生的聚类簇数量$ j $.
如果客户端首次被选中参与训练,则本地模型初始化与全局模型保持一致,即使用聚类簇模型进行初始化. 而对于已参与过联邦训练的客户端,其本地模型的初始化方式不同于传统联邦学习算法直接使用$ {\theta }_* $,而是按照算法2中的行⑤进行更新.
HS-CFA利用该方式对本地模型训练初始化,间接获取其他聚类簇的全局模型数据分布先验知识,从而让算法避免陷入局部最优解,排除自身聚类簇模型$ \bar \theta_{c} $的影响.
根据预备知识中的联邦学习数据异构性来源于全局模型的优化目标无法对齐的性质,本文从VC理论分析联邦学习全局模型的泛化上限.
定理1[13]. 假设$ { H} $为一个VC维为$ d $的假设空间,$ {\hat{D}}_{{\mathrm{S}}_{i}} $代表联邦学习从每个源域中抽取大小为m的样本所产生的先验分布,$ {\hat{D}}_{\mathrm{T}} $为目标域数据分布,因此存在$ {\alpha }_{i}\in {\mathbb{R}}^{+} $且$ \displaystyle\sum _{i=1}^{N}{\alpha }_{i}=1 $,那么对于假设空间中的函数$ h\in { H} $,至少有$ 1-\xi $的概率存在:
$ {\epsilon}_{\mathrm{T}}\left(h\right)\le {\hat{\epsilon}}_{{\rm S}_{i}}\left( \sum_i {\alpha }_{i}{h}_{{\mathrm{S}}_{i}} \right)+\sum_i {\alpha }_{i}\left(\dfrac{1}{2}{\hat{d}}_{{ H}\times { H}}\left({\hat{D}}_{{\mathrm{S}}_{i}},{\hat{D}}_{\mathrm{T}}\right) \right)+{\sigma }_{\mathrm{v}\mathrm{c}}+\lambda \text{,} $
其中$ {\sigma }_{\mathrm{v}\mathrm{c}} $是VC理论泛化误差常量,$ \lambda $代表最低误差之和的无穷小量,$ {\hat{d}}_{{ H}\times { H}}({\hat{D}}_{{\mathrm{S}}_{i}},{\hat{D}}_{\mathrm{T}}) $代表客户端和全局数据分布之间的散度. 定理1表明全局模型的泛化上限一方面由本地模型泛化能力$ {\epsilon(h}_{{\mathrm{S}}_{i}}) $决定,另一方面则受全局模型的数据分布与局部模型的数据部分差异性的影响,考虑差异性客观存在,无法消除$ {\hat{d}}_{{ H}\times { H}}\big({\hat{D}}_{{\mathrm{S}}_{i}}, {\hat{D}}_{\mathrm{T}}\big) $的影响. 但是客户端权重$ {\alpha }_{i} $不仅影响本地模型聚合性能,同时可动态地调整客户端与数据分布之间的差异. 因此为了尽可能最大化全局模型泛化上限,需要动态调整权重$ {\alpha }_{i} $,提高对全局模型分布有益的客户端的权重,同时限制对全局模型数据分布有害的客户端的权重.
考虑到经典联邦学习算法中服务器训练实质是对客户端模型进行加权聚合以模拟分布式随机梯度下降法. 权重$ {\alpha }_{i} $被设置为$ \left|{D}_{i}\right|/\Bigg|\displaystyle\sum _{i=1}^{n}{D}_{i}\Bigg| $,即客户端i数据量与所有客户端数据量之比. 本文提出在服务器训练周期中采用DBSCAN算法和层次化聚类方法训练全局模型的策略,希望客户端的权重$ {\alpha }_{i} $的设置不再依赖数据集的大小,而基于对全局数据分布的公平化差异性贡献. 假设全局数据分布$ D\in \mathrm{ }\{{D}_{1},{D}_{2},\mathrm{ }…, {D}_{n}|n\in M\} $,其中$ M $是参与联邦学习的客户端的总数. 客户端上传的$ {S im}_{i} $相似度可度量不同的$ \{{D}_{1},{D}_{2},\mathrm{ }…, {D}_{n} $}数据分布特性,通过相似度对客户端模型划分聚类,会使相似本地数据分布的客户端模型聚类为簇. 考虑到不同聚类簇包含的客户端模型数目不一,需要平衡不同聚类簇对于全局模型的数据分布的贡献,因此HS-CFA均采取平均聚合策略.
算法3. HS-CFA双层聚合算法.
输入:参与训练的客户端模型$ \left\{{\theta }_{1},{\theta }_{2},\mathrm{ }… ,{\theta }_{i}\right\} $,参与训练的客户端模型与全局模型的相似度$ \left\{{S im}_{1},{S im}_{2},\mathrm{ }… , {S im}_{i}\right\} $,客户端标识$ i $;
输出:全局模型$ {\theta }_* $,聚类簇模型集合$ {\theta }_{\mathrm{C}\mathrm{l}\mathrm{u}\mathrm{s}\mathrm{t}\mathrm{e}\mathrm{r}} $,聚类簇数目$ j $.
/* T 为客户端训练时和全局模型通信轮次;
$ {\mathcal{N}}_{\epsilon} $为DBSCAN算法的领域范围;
$ {\mathrm{m}\mathrm{i}\mathrm{n}}_{\mathrm{s}} $为DBSCAN算法的最小采样量. */
① for t ←0 to T do
② if t = 0
③  初始化$ {\theta }_{*}^{0} $,$ {\theta }_{\mathrm{C}\mathrm{l}\mathrm{u}\mathrm{s}\mathrm{t}\mathrm{e}\mathrm{r}}=\mathrm{\varnothing } $;
④ end if
⑤ for $ M $客户端do
⑥  接收$ {\theta }_{M}=\left\{\{{\theta }_{i},{S im}_{i},i\},i\in m\right\} $;
⑦ end for
⑧  $ \left\{{\theta }_{\mathrm{C}}^{1},{\theta }_{\mathrm{C}}^{2},\mathrm{ }… ,{\theta }_{\mathrm{C}}^{j}\right\}=DBSCAN({\theta }_{M},{\mathcal{N}}_{\epsilon},{\mathrm{m}\mathrm{i}\mathrm{n}}_{\mathrm{s}}) $;
⑨ for $ {\theta }_{\mathrm{C}}^{c}\in \left\{{\theta }_{\mathrm{C}}^{1},{\theta }_{\mathrm{C}}^{2},\mathrm{ }… ,{\theta }_{\mathrm{C}}^{j}\right\} $ do
⑩   $ \bar \theta_{c}=\dfrac{1}{|{\theta }_{\mathrm{C}}^{c}|}\displaystyle\sum _{i\in {\theta }_{\mathrm{C}}^{c}}{\theta }_{i} $;/*第1次簇内聚合*/
⑪   $ {\theta }_{\mathrm{C}\mathrm{l}\mathrm{u}\mathrm{s}\mathrm{t}\mathrm{e}\mathrm{r}} $← $ {\theta }_{\mathrm{C}\mathrm{l}\mathrm{u}\mathrm{s}\mathrm{t}\mathrm{e}\mathrm{r}}\cup \bar \theta_{c} $;
⑫ end for
⑬  $ {j=|\theta }_{\mathrm{C}\mathrm{l}\mathrm{u}\mathrm{s}\mathrm{t}\mathrm{e}\mathrm{r}}| $;
⑭  $ {\theta }_*^{\,t+1} $= $ \dfrac{1}{j} $ $ \displaystyle\sum _{c=1}^{j}\bar \theta_{c}; $/*第2次簇间聚合*/
⑮  $ {\theta }_{\mathrm{C}\mathrm{l}\mathrm{u}\mathrm{s}\mathrm{t}\mathrm{e}\mathrm{r}}=\mathrm{\varnothing } $;
⑯ 下发{$ {\theta }_*^{\,t+1},\bar \theta_{c},j $}至客户端;
⑰ end for
算法3包含了服务器训练周期的2个阶段.
1)初始化阶段. 服务器初始用于聚合时保存$ t $轮次的聚类簇模型集合.
2)聚类训练阶段. 服务器端在接收到所有客户端发送的$ \{{\theta }_{i},{S im}_{i},i\} $数据后,基于DBSCAN 算法按照每个客户端的相似性度量$ {S im}_{i} $进行迭代聚类,该过程将数据分布相似的客户端划分至相同的簇内,以确保簇内客户端模型在数据分布上具有较高的一致性. 对于每个聚类簇,服务器采用平均聚合的方式生成聚类簇模型,并进一步对这些聚类簇模型进行2次聚类,最终得到全局模型. 考虑到全局数据分布的复杂性,HS-CFA通过DBSCAN不需要设定聚类簇的先验知识的特点,一方面尽可能将相似度量统计值的局部客户端模型进行聚合,另一方面自动过滤掉与聚类簇分布差异过大的离群本地模型. 从而确保2次聚类后得到的全局模型排除离群客户端数据分布的影响,有效衡量各数据分布下本地模型对全局模型聚合的贡献.
在获得全局模型后,服务器需更新该模型至客户端. 鉴于不同轮次中服务器选取的客户端可能存在差异,分发阶段采用以下策略:对于上一轮已参与训练且在当前轮次中再次被选中的客户端,服务器根据客户端标识$ i $发送聚类簇模型$ \bar \theta_{c} $、全局模型$ {\theta }_* $和聚类簇数目$ j $至该客户端. 对于首次被选中的客户端,服务器仅传输全局模型$ {\theta }_* $,确保模型初始化的一致.
为了验证HS-CFA算法的有效性,本文选择了3个评估联邦学习的基准数据集:Fashion-MNIST[33],CIFAR-10[34],CIFAR-100[34]. Fashion-MNIST包含10个类别的灰度图像,每个类别有6000张训练图像和1000张测试图像,图像分辨率为28×28像素;CIFAR-10包含10个类别的彩色图像,每个类别有5000张训练图像和1000张测试图像,图像分辨率为32×32像素;CIFAR-100包含100个类别的彩色图像,每个类别有500张训练图像和100张测试图像,图像分辨率为32×32像素.
数据集均采用2种数据异构性设置:1)通过狄利克雷分布$ Dir\left(\delta \right) $[35]对数据集进行划分,其中$ \delta $用于衡量数据的异构性,系数值越小,数据分布的不均匀程度越高,即数据越偏离独立同分布. 2)基于文献[14, 17],对数据集中的图像进行随机4种角度旋转($ {0}^{\circ}, {90}^{\circ},{180}^{\circ},{270}^{\circ} $). 旋转增强图像根据比例$ \beta $混入客户端数据集.
本文采用2种神经网络结构作为骨干网络:1)用于Fashion-MNIST数据集的2层多层感知机网络结构;2)用于CIFAR-10和CIFAR-100数据集的AlexNet[36]网络.
训练参数设置采用随机梯度下降算法,优化器动量设置为0.9,权重衰减为0.0001,学习率为0.01. 批处理大小设置为32(对于Fashion-MNIST)和64(对于CIFAR-10和CIFAR-100).
联邦学习参数设置中客户端数目$ M $为50,每个全局交互轮次中参与度$ P $为80%. 每个客户端的局部训练次数$ T $为5,全局交互轮次$ C $为100轮. 数据异构性方面,狄利克雷分布设置$ \delta $为0.1和0.3. 旋转增强混入$ \beta $为30%和50%的数据. 在DBSCAN算法中邻域阈值$ {\mathcal{N}}_{\epsilon} $设置为0.3,最小样本数$ {\mathrm{m}\mathrm{i}\mathrm{n}}_{\mathrm{s}} $设置为5. 客户端计算Sinkhorn采样设置为16. 考虑到相似度用于DBSCAN聚类计算无需过于精准,为尽可能降低客户端的计算负担,Sinkhorn的迭代次数设置为5.
所有实验均在Ubuntu 20.04操作系统下,通过32 GB的Nvidia Tesla V100服务器使用PyTorch[37]框架进行训练与测试,并根据不同的随机种子重复运行了5次.
本文选取了基线算法FedAvg[25];3种客户端优化联邦学习算法,包括FedProx[27], FedRobust[38],SCAFFOLD[28];2种针对单一目标模型的聚类联邦学习FL-HC[17]和FLIS[21];2种个性化联邦聚类算法IFCA[16]和FLSC[19]. 由于其会产生多个全局模型,测试选择了性能最优的个性化模型作为基线.
表1展示了在数据异构性场景下,Fashion-MNIST数据集的全局模型平均测试准确率,其中异构性由狄利克雷分布参数$ \delta $控制,取值分别为0.1和0.3. 在参数为0.1的情况下,所有算法的测试准确率均低于参数为0.3下的情况,同时测试准确率方差更大,表明在极端non-IID场景中,不同算法性能鲁棒性均有所下降. 相比其他基线算法,HS-CFA的方差更小. 同时HS-CFA算法在参数为0.1取得了最优测试准确率,在参数为0.3时,其测试准确率略低于FLIS,但稳定性优于FLIS. 这主要归因于HS-CFA采用的DBSCAN聚类算法能够筛除离群点,从而降低在聚合全局模型时所产生的负面影响. 总体而言,HS-CFA在不同non-IID设置下测试准确率变化较小,表明HS-CFA在处理数据分布异构性方面具有良好的鲁棒性.
表2展示了旋转混入数据异构性设置下的测试结果,其中$ \beta $表示混入旋转数据的比例. 结果显示,在客户端中混入旋转图片对所有算法的测试准确率影响较小,各算法的准确率方差均较小. 在$ \beta $=30%时,虽然HS-CFA的准确率略低于SCAFFOLD,FL-HC,FLIS算法,但准确率差距均在1%以内,同时其稳定性优于这3种算法. 在$ \beta $=50%时,HS-CFA的准确率仅次于SCAFFOLD,但其稳定性仍优于SCAFFOLD. 这表明,在数据异构性水平较低的情况下,HS-CFA算法依然展现出较强的竞争力.
图2展示了在CIFAR-10数据集中不同δ设置下相关算法测试准确率曲线. 实验表明,在数据极端不平衡的条件下,HS-CFA算法表现出最佳的性能. 相较于同样优化单一目标模型的聚类联邦学习算法FL-HC和FLIS,HS-CFA每轮训练更加平滑. 此外,HS-CFA在大约30轮时便已实现较好的收敛性,显示了聚类中利用Sinkhorn距离衡量客户端数据分布的相似性具有良好的效果,并且能够有效地反映不同客户端数据分布的差异性.
表3展示了在CIFAR-100数据集上,采用$ \mathit{\delta } $=0.3和旋转混入$ \mathit{\beta } $=30%情况下,各算法的测试准确率. 由于CIFAR-100数据集中有100个类别,因此每个类别在客户端中的数据量相对较少,这进一步加剧了数据的不平衡性. 结果表明,HS-CFA算法在测试准确率上仍然优于基准算法,并表现出较好的稳定性. 尽管其稳定性略低于 FLSC 和 IFCA,这一差异主要源于 IFCA 和 FLSC生成个性化全局模型每轮记录的均是最优的个性化模型准确率,而 HS-CFA 侧重于训练单一全局模型.
从$ \mathit{\beta } $=30%的实验设置来看,旋转混入方式引入的数据异构性对测试准确率的影响较小,各算法的测试准确率差异不显著. 总体而言,基于聚类的联邦学习算法在测试准确率上普遍优于通过调整局部客户端目标函数的算法. 这说明,在分类任务中,随着标签类别数量的增加,聚类联邦学习通过优化客户端模型的权重分配策略,更有效地缓解了数据异构性带来的影响,从而提升了模型的整体性能.
表4展示了在CIFAR-10数据集上采用基本实验设置时HS-CFA各模块的计算时间开销. 其中本地训练设置为5轮,Sinkhorn距离计算的采样样本为16,第1层聚合则采用DBSCAN对40个客户端进行聚类.
尽管 HS-CFA采用了轻量化的Sinkhorn计算方法来间接衡量本地数据的相似性,然而相似度计算时间仍然占据约50%的总时间.
相比之下,DBSCAN算法在聚类时仅使用相似度度量,因此处理时间较少. 如果在该模块中引入空间索引结构等优化措施,时间性能将进一步提升.
本文对HS-CFA中采用的DBSCAN方法的超参数进行了消融实验. 实验使用的数据集为CIFAR-10,图3展示了0.1,0.3,0.45,0.6这4种密度参数下全局模型的测试准确率.
通常情况下邻域半径为0.3时模型效果最佳. 然而训练达到80轮后,邻域半径为0.6下模型表现更优. 这一现象是由于随着训练轮次的增加,客户端数据分布的差异性持续存在,导致本地模型与全局模型的对齐难度加大. 因此,在后期训练阶段,聚类时应需要选择更大的邻域范围.
图4展示了在固定邻域半径的情况下,选择簇内最小样本数时,HS-CFA算法的测试准确率.
当最小样本数设置为1时,实际上不存在离群点,算法退化为基于Sinkhorn距离的层次化聚类,此时模型性能最差. 而当最小样本数为8时,由于聚类簇较少,产生了较多的离群点. 而过多离群点被剔除同样导致模型性能不佳.
表5展示了HS-CFA对于Sinkhorn距离计算分布相似性的采样数目的影响. 实验使用的数据集为CIFAR-10,采样方法为随机采样,采样样本数选择4,16,64,128时对全局模型的准确率和相似度计算所需时间的影响. Sinkhorn距离计算时间复杂度取决于采样数据组成成本矩阵的大小$ n $和迭代次数$ k $,时间复杂度为$ O(k{n}^{2}) $,而经典最优传输代价Wasserstein距离的复杂度为$ O ({n}^{3}\mathrm{l}\mathrm{o}\mathrm{g}n ) $.
从表5中可以看到,尽管采用了GPU加速,但是随着采样数目的增加,相似度计算的处理时间显著增长,并且模型准确率在初期阶段有显著提升,然而在采样数达到一定阈值后增加采样数对模型准确率的提升效果趋于饱和. 因此,在实际应用中,HS-CFA 需要在计算开销与模型准确率提升之间权衡,以选择合适的采样数目,从而兼顾计算效率和模型性能.
针对联邦学习中non-IID导致的数据异构性问题,本文提出了一种2层聚类联邦学习算法. 该算法采用轻量化的Sinkhorn距离来衡量不同客户端的数据分布相似性,并在聚类过程中使用DBSCAN算法,通过对簇内模型和簇间模型平均聚合优化全局模型. 该算法可有效地将具有相似数据分布的客户端进行自适应分簇进而动态修改本地模型聚合时的权重,提高了联邦学习全局模型准确性和泛化能力. 实验结果表明,本文提出的算法在处理数据异构性方面,相较于传统联邦学习算法和其他基于聚类的算法在模型准确率和鲁棒性方面均取得了良好的效果.
总体而言本文通过结合最优传输代价和聚类算法,为联邦学习数据异构性问题提供了了一种轻量且高效的优化方案. 未来的工作将重点关注如何设计更轻量级且定义良好的数据分布相似性度量方法,以进一步降低计算开销并提升模型在边缘计算中的适应性.
作者贡献声明:赵瑞提出方案整体设计、算法思路并撰写论文;段小文负责实验验证;刘新负责工程化指导;周睿和周庆国指导论文并提出修改意见.

参考文献 引证文献
排序方式:
1
Noor M M, Hassan W H. Current research on Internet of things (IoT) security: A survey[J]. Computer Networks, 2019, 148: 283−294
2
Kumar S R, Nystrom M, Lambert J, et al. Adversarial machine learning-industry perspectives[C]//Proc of the 41st IEEE Symp on Security and Privacy. Piscataway, NJ: IEEE, 2020: 69−75
3
Sarkar S. Internet of Things[M]. San Francisco: Morgan Kaufmann, 2016
4
Mao Yuyi, You Changsheng, Zhang Jun, et al. A survey on mobile edge computing: The communication perspective[J]. IEEE Communications Surveys & Tutorials, 2017, 19(4): 2322−2358
5
Yu Wei, Liang Fan, He Xiaofei, et al. A survey on the edge computing for the Internet of Things[J]. IEEE Access, 2017, 6: 6900−6919
6
Shakarami A, Ghobaei-Arani M, Shahidinegad A. A survey on the computation offloading approaches in mobile edge computing: A machine learning-based perspective[J]. Computer Networks, 2020, 182: 107496
7
Kairouz P, Mcmahan H B, Avent B, et al. Advances and open problems in federated learning[J]. Foundations and Trends® in Machine Learning, 2021, 14(1/2): 1−210
8
Abreha H G, Hayajneh M, Serhani M A. Federated learning in edge computing: A systematic survey[J]. Sensors, 2022, 22(2): 450
9
张雪晴, 刘延伟, 刘金霞, 等. 面向边缘智能的联邦学习综述[J]. 计算机研究与发展, 2023, 60(6): 1276−1295
Zhang Xueqing, Liu Yanwei, Liu Jinxia, et al. An overview of federated learning in edge intelligence[J]. Journal of Computer Research and Development, 2023, 60(6): 1276−1295 (in Chinese)
10
Zhu Hangyu, Xu Jinjin, Liu Shiqing, et al. Federated learning on non-IID data: A survey[J]. Neurocomputing, 2021, 465: 371−390
11
Sattler F, Wiedemann S, Muller K R, et al. Robust and communication-efficient federated learning from non-IID data[J]. IEEE Transactions on Neural Networks and Learning Systems, 2020, 31(9): 3400−3413
12
Qu Liangqiong, Zhou Yuyin, Liang P P, et al. Rethinking architecture design for tackling data heterogeneity in federated learning[C]//Proc of the 2022 IEEE/CVF Conf on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society, 2022: 10061−10071
13
张瑞麟, 杜晋华, 尹浩. 跨设备联邦学习中的客户端选择算法[J]. 软件学报, 2024, 35(12): 5725−5740
Zhang Ruilin, Du Jinhua, Yin Hao. Client selection algorithm in cross-device federated learning[J]. Journal of Software, 2024, 35(12): 5725−5740 (in Chinese)
14
郭松岳, 王阳谦, 柏思远, 等. 面向数据混合分布的联邦自适应交互模型[J]. 计算机研究与发展, 2023, 60(6): 1346−1357
Guo Songyue, Wang Yangqian, Bai Siyuan, et al. Federated adaptive interaction model for mixed distribution data[J]. Journal of Computer Research and Development, 2023, 60(6): 1346−1357 (in Chinese)
15
Wang Jianyu, Charles Z, Xu Zheng, et al. A field guide to federated optimization[J]. arXiv preprint, arXiv: 2107.06917, 2021
16
Ghosh A, Chung Jichan, Yin Dong, et al. An efficient framework for clustered federated learning[C]//Proc of the 33rd Advances in Neural Information Processing Systems. New York: Curran Associates Inc, 2020: 19586−19597
17
Briggs C, Fan Zhong, Andras P. Federated learning with hierarchical clustering of local updates to improve training on non-IID data[C]//Proc of the 2020 Int Joint Conf on Neural Networks. Piscataway, NJ: IEEE, 2020: 1−9
18
Kim Y, Hakim E A, Haraldson J, et al. Dynamic clustering in federated learning[C]//Proc of the 2021 IEEE Int Conf on Communications. Piscataway, NJ: IEEE, 2021: 1−6
19
Li Chengxi, Li Gang, Varshney P K. Federated learning with soft clustering[J]. IEEE Internet of Things Journal, 2021, 9(10): 7773−7782
20
Sattler F, Muller K R, Samek W. Clustered federated learning: Model-agnostic distributed multitask optimization under privacy constraints[J]. IEEE Transactions on Neural Networks and Learning Systems, 2020, 32(8): 3710−3722
21
Morafah M, Vahidian S, Wang Weijia, et al. FLIS: Clustered federated learning via inference similarity for non-IID data distribution[J]. IEEE Open Journal of the Computer Society, 2023, 4: 109−120
22
Cuturi M. Sinkhorn distances: Lightspeed computation of optimal transport[C]//Proc of the 26th Advances in Neural Information Processing Systems. New York: Curran Associates Inc, 2013: 2292−2300
23
Schubert E, Sander J, Ester M, et al. DBSCAN revisited, revisited: Why and how you should (still) use DBSCAN[J]. ACM Transactions on Database Systems, 2017, 42(3): 1−21
24
Sejourbe T, Feydy J, Vialard F X, et al. Sinkhorn divergences for unbalanced optimal transport[J]. arXiv preprint, arXiv: 1910.12958, 2019
25
Mcmahan B, Moore E, Ramage D, et al. Communication-efficient learning of deep networks from decentralized data[C]//Proc of the 20th Int Conf on Artificial Intelligence and Statistics. New York: PMLR, 2017: 1273−1282
26
万伟, 胡胜山, 陆建荣, 等. 联邦学习在高度数据异构场景下的泛化鲁棒性增强[J]. 中国科学: 信息科学, 2024, 54(3): 566−581
Wan Wei, Hu Shengshan, Lu Jianrong, et al. Enhancing generalization robustness of federated learning in highly heterogeneous environments[J]. SCIENTIA SINICA Informationis, 2024, 54(3): 566−581 (in Chinese)
27
Li Tian, Sahu A K, Zaheer M, et al. Federated optimization in heterogeneous networks[J]. Proceedings of Machine Learning and Systems, 2020, 2: 429−450
28
Karimireddy S P, Kale S, Mohri M, et al. SCAFFOLD: Stochastic controlled averaging for federated learning[C]//Proc of the 37th Int Conf on Machine Learning. New York: PMLR, 2020: 5132−5143
29
Yan Bingjie, Liu Boyi, Wang Lujia, et al. FedCM: A real-time contribution measurement method for participants in federated learning[C]//Proc of the 2021 Int Joint Conf on Neural Networks. Piscataway, NJ: IEEE, 2021: 1−8
30
Sun Yan, Shen Li, Huang Tiansheng, et al. FedSpeed: Larger local interval, less communication round, and higher generalization accuracy[C/OL]//Proc of the 11th Int Conf on Learning Representations. 2023[2025-03-03]. https://openreview.net/forum?id=bZjxxYURKT
31
Ghadimi E, Teixeira A, Shames I, et al. Optimal parameter selection for the alternating direction method of multipliers (ADMM): Quadratic problems[J]. IEEE Transactions on Automatic Control, 2014, 60(3): 644−658
32
Fama F, Kalalas C, Lagen S, et al. Measuring data similarity for efficient federated learning: A feasibility study[C]//Proc of the 13th Int Conf on Computing, Networking and Communications. Piscataway, NJ: IEEE, 2024: 394−400
33
Xiao Han, Rasul K, Vollgraf R. Fashion-MNIST: A novel image dataset for benchmarking machine learning algorithms[J]. arXiv preprint, arXiv: 1708.07747, 2017
34
Krizhevisky A, Hinton G E. Learning multiple layers of features from tiny images[R]. Toronto: University of Toronto, 2009
35
Hsu T H, Qi Hang, Brown M. Measuring the effects of non-identical data distribution for federated visual classification[J]. arXiv preprint, arXiv: 1909.06335, 2019
36
Krizhevisky A, Sutskever I, HINTON G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84−90
37
Paszke A, Gross S, Massa F, et al. PyTorch: An imperative style, high-performance deep learning library[C]//Proc of the 33rd Int Conf on Neural Information Processing Systems. New York: Curran Associates Inc, 2019: 8026−8039
38
Reisizadeh A, Farnia F, Pedarsani R, et al. Robust federated learning: The case of affine distribution shifts[C]//Proc of the 34th Int Conf on Neural Information Processing Systems. New York: Curran Associates Inc, 2020: 21554−21565
2025年第62卷第12期
115
50
0
引用本文
BibTeX
文章信息
doi: 10.7544/issn1000-1239.202440969
  • 接收时间:2024-12-29
  • 首发时间:2026-11-10
  • 出版时间:2025-12-01
补充材料
相关文章
文章信息
作者
出版历史
  • 收稿日期:2024-12-29
  • 修回日期:2025-04-01
基金
作者信息
    兰州大学信息科学与工程学院 兰州 730030

通讯作者:

周庆国()
参考文献
分享链接
https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/10.7544/issn1000-1239.202440969
分享至
全文二维码

扫描看全文

引用本文
BibTeX
本文的引用情况
2种不同金属材料的力学参数
科
Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)
属
Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏