收藏切换
FedDW: Distilling Weights Through Consistency Optimization in Heterogeneous Federated Learning
收藏切换
Jiayu Liu1, Yong Wang1, 2, *, Jing Yang1, Nianbin Wang1, 2
Journal of Computer Research and Development | 2026, 63(4) : 998 - 1009
Less
收藏切换
Journal of Computer Research and Development | 2026, 63(4): 998-1009
• Artificial Intelligence •
FedDW: Distilling Weights Through Consistency Optimization in Heterogeneous Federated Learning
Full
Jiayu Liu1, Yong Wang1, 2, *, Jing Yang1, Nianbin Wang1, 2
Affiliations
  • 1College of Computer Science and Technology, Harbin Engineering University, Harbin 150001
  • 2Modeling and Emulation in E-Government National Engineering Laboratory (Harbin Engineering University), Harbin 150001
  • born in 2000. PhD candidate. Student member of CCF. His main research interests include federated learning, reinforcement learning, and expert system of LLM mixture

    born in 1983. PhD, associate professor. Senior member of CCF. His main research interests include artificial intelligence, privacy computing, and Internet of things

    born in 1962. PhD, professor. Senior member of CCF. Her main research interests include artificial intelligence, social computing, and privacy protection

    born in 1967. PhD, professor. Senior member of CCF. His main research interests include artificial intelligence, large language model, and data mining

About Author:

born in 2000. PhD candidate. Student member of CCF. His main research interests include federated learning, reinforcement learning, and expert system of LLM mixture

born in 1983. PhD, associate professor. Senior member of CCF. His main research interests include artificial intelligence, privacy computing, and Internet of things

born in 1962. PhD, professor. Senior member of CCF. Her main research interests include artificial intelligence, social computing, and privacy protection

born in 1967. PhD, professor. Senior member of CCF. His main research interests include artificial intelligence, large language model, and data mining

Published: 2026-04-01 doi: 10.7544/issn1000-1239.202550451
Outline
收藏切换

Federated learning is a classic distributed machine learning paradigm that enables collaborative model training without centralizing data. This approach offers significant advantages in ensuring data privacy. However, it faces numerous challenges in terms of training efficiency and model performance due to the significant data heterogeneity across clients and the increasing scale of federated networks. Existing studies have shown that, in independent and identically distributed (IID) environments, the model parameter structure typically adheres to specific consistency relations, which are often preserved in the intermediate results during the neural network training process. If these consistency relations can be identified and regularized under non-independent and identically distributed (non-IID) data, it can help align the parameter distribution with the IID scenario, thereby mitigating the impact of data heterogeneity. Based on this idea, we introduce the concept of deep learning on encrypted data and propose a consistency optimization paradigm. We further explore the consistency relationship between data soft labels and the model’s classification layer weight matrix, from which we construct a new heterogeneous federated learning framework, FedDW. Experiments are conducted on four public datasets and several neural network models, including ResNet and ViT. The results show that, under highly heterogeneous client data settings, FedDW outperforms 10 state-of-the-art federated learning methods with an average accuracy improvement of approximately 3%. Moreover, we theoretically prove that FedDW offers higher training efficiency, with the additional backpropagation computation overhead being negligible.

federated learning  /  data heterogeneity  /  deep learning encrypted data  /  consistency optimization paradigm  /  weight distillation
刘佳瑜, 王勇, 杨静, 王念滨. FedDW:基于一致性优化的权重蒸馏异构联邦学习方法. 计算机研究与发展, 2026 , 63 (4) : 998 -1009 . DOI: 10.7544/issn1000-1239.202550451
Jiayu Liu, Yong Wang, Jing Yang, Nianbin Wang. FedDW: Distilling Weights Through Consistency Optimization in Heterogeneous Federated Learning[J]. Journal of Computer Research and Development, 2026 , 63 (4) : 998 -1009 . DOI: 10.7544/issn1000-1239.202550451
Year 2026 volume 63 Issue 4
121
59
Cite this Article
BibTeX
Article Info
doi: 10.7544/issn1000-1239.202550451
  • Receive Date:2025-06-06
  • Online Date:2026-11-17
  • Published:2026-04-01
Article Data
Affiliations
History
  • Received:2025-06-06
  • Revised:2025-10-21
  • Accepted:2025-10-31
Affiliations
    1College of Computer Science and Technology, Harbin Engineering University, Harbin 150001
    2Modeling and Emulation in E-Government National Engineering Laboratory (Harbin Engineering University), Harbin 150001
References
Share
https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/10.7544/issn1000-1239.202550451
Share to
QR

Scan QR to access full text

Cite this article
BibTeX
Citations
表12种不同金属材料的力学参数
科
Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)
属
Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏
  • BibTeX
  • EndNote
  • RefWorks
  • TxT