收藏切换
A Lightweight Noise Label Learning Method Based on Online Distillation
收藏切换
Yiwang Huang1, 2, 3, *, Yuxin Huang2, Sheng Liu1, 3
Journal of Computer Research and Development | 2024, 61(12) : 3121 - 3133
Less
收藏切换
Journal of Computer Research and Development | 2024, 61(12): 3121-3133
• Artificial Intelligence •
A Lightweight Noise Label Learning Method Based on Online Distillation
Full
Yiwang Huang1, 2, 3, *, Yuxin Huang2, Sheng Liu1, 3
Affiliations
  • 1School of Data Science , Tongren University, Tongren , Guizhou 554300
  • 2School of Computer Science and Mathematics, Fujian University of Technology, Fuzhou 350001
  • 3Guizhou Provincial Key Laboratory of Public Big Data (Guizhou University), Guiyang 550025
  • born in 1978. PhD, professor, Master supervisor. Member of CCF. His main research interests include artificial intelligence, business process management, and software formalization method

    born in 1998. Master candidate. Member of CCF. His research interests include computer vision and label noise learning

    born in 1984. PhD, professor. His research interest includes information processing

About Author:

born in 1978. PhD, professor, Master supervisor. Member of CCF. His main research interests include artificial intelligence, business process management, and software formalization method

born in 1998. Master candidate. Member of CCF. His research interests include computer vision and label noise learning

born in 1984. PhD, professor. His research interest includes information processing

Published: 2024-12-01 doi: 10.7544/issn1000-1239.202330382
Outline
收藏切换

Training deep learning models with noisy data containing lossy labels is a hot research topic in machine learning. Studies have shown that deep learning model training is susceptible to overfitting due to noisy data. Recently, a method combining meta-learning and label correction can make the model better adapt to the noisy data to mitigate the overfitting phenomenon. However, this meta-label correction method relies on the model’s performance, and the lightweight model does not have good generalization performance under noisy data. To address this problem, we propose a knowledge distillation-based meta-label correction learning method (KDMLC), which treats the meta label correction model (MLC) composed of a deep neural network and a multilayer perceptron as a teacher model to correct the noise labels and guide the training of the lightweight model, and at the same time, KDMLC adopts a two-layer optimization strategy to train and enhance the generalization ability of the teacher model, so as to generate a higher-quality pseudo-labels for training the lightweight model. The experiments show that KDMLC improves the test accuracy by 5.50% compared with MLC method at high noise level; meanwhile, using Cutout data enhancement on the CIFAR10 dataset, KDMLC improves the test accuracy by 9.11% compared with MLC at high noise level, and the experiments on the real noisy dataset, Clothing1M, also show that KDMLC outperforms the other methods, verifying that KDMLC is better than the other methods, which verifies the feasibility and validity of KDMLC method.

pseudo labels  /  label correction  /  meta learning  /  knowledge distillation  /  noise data
黄贻望, 黄雨鑫, 刘声. 一种基于在线蒸馏的轻量化噪声标签学习方法. 计算机研究与发展, 2024 , 61 (12) : 3121 -3133 . DOI: 10.7544/issn1000-1239.202330382
Yiwang Huang, Yuxin Huang, Sheng Liu. A Lightweight Noise Label Learning Method Based on Online Distillation[J]. Journal of Computer Research and Development, 2024 , 61 (12) : 3121 -3133 . DOI: 10.7544/issn1000-1239.202330382
Year 2024 volume 61 Issue 12
87
42
Cite this Article
BibTeX
Article Info
doi: 10.7544/issn1000-1239.202330382
  • Receive Date:2023-05-15
  • Online Date:2026-11-17
  • Published:2024-12-01
Article Data
Affiliations
History
  • Received:2023-05-15
  • Revised:2023-12-04
Affiliations
    1School of Data Science , Tongren University, Tongren , Guizhou 554300
    2School of Computer Science and Mathematics, Fujian University of Technology, Fuzhou 350001
    3Guizhou Provincial Key Laboratory of Public Big Data (Guizhou University), Guiyang 550025
References
Share
https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/10.7544/issn1000-1239.202330382
Share to
QR

Scan QR to access full text

Cite this article
BibTeX
Citations
表12种不同金属材料的力学参数
科
Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)
属
Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏
  • BibTeX
  • EndNote
  • RefWorks
  • TxT