收藏切换
Advances and Trends of Continual Learning
收藏切换
Wenbin Li1, Yakun Xiong1, Zhichen Fan1, Bo Deng2, Fuyuan Cao3, 4, Yang Gao1, *
Journal of Computer Research and Development | 2024, 61(6) : 1476 - 1496
Less
收藏切换
Journal of Computer Research and Development | 2024, 61(6): 1476-1496
• Artificial Intelligence •
Advances and Trends of Continual Learning
Full
Wenbin Li1, Yakun Xiong1, Zhichen Fan1, Bo Deng2, Fuyuan Cao3, 4, Yang Gao1, *
Affiliations
  • 1State Key Laboratory for Novel Software Technology (Nanjing University), Nanjing 210023
  • 2Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094
  • 3School of Computer and Information Technology, Shanxi University, Taiyuan 030006
  • 4School of Big Data, Shanxi University, Taiyuan 030006
  • born in 1991. PhD, associate researcher. Member of CCF. His main research interests include machine learning, metric learning, few-shot learning, and computer vision

    born in 2000. Master candidate. Member of CCF. His main research interests include machine learning, continual learning, and computer vision

    born in 1998. Master candidate. Member of CCF. His main research interests include machine learning, continual learning, and computer vision

    born in 1973. PhD, researcher. Senior member of CCF. His main research interests include distributed computing and intelligent service system

    born in 1974. PhD, professor, PhD supervisor. Senior member of CCF. His main research interests include machine learning, data mining, and big data analysis

    born in 1972. PhD, professor, PhD supervisor. Senior member of CCF. His main research interests include reinforcement learning, multi-agent systems, and big data analysis

About Author:

born in 1991. PhD, associate researcher. Member of CCF. His main research interests include machine learning, metric learning, few-shot learning, and computer vision

born in 2000. Master candidate. Member of CCF. His main research interests include machine learning, continual learning, and computer vision

born in 1998. Master candidate. Member of CCF. His main research interests include machine learning, continual learning, and computer vision

born in 1973. PhD, researcher. Senior member of CCF. His main research interests include distributed computing and intelligent service system

born in 1974. PhD, professor, PhD supervisor. Senior member of CCF. His main research interests include machine learning, data mining, and big data analysis

born in 1972. PhD, professor, PhD supervisor. Senior member of CCF. His main research interests include reinforcement learning, multi-agent systems, and big data analysis

Published: 2024-06-01 doi: 10.7544/issn1000-1239.202220820
Outline
收藏切换

With the development and successful application of deep learning, continual learning has attracted increasing attention and has been a hot topic in the field of machine learning, especially in the resource-limited and data-security scenarios with the increasing requirements of quickly learning sequential tasks and data. Unlike humans who enjoy the ability of continually learning and transferring knowledge, the existing deep learning models are prone to easily suffering from a catastrophic forgetting problem in a sequential learning process. Therefore, how to continually learn new knowledge and retain old knowledge at the same time on dynamic and non-stationary sequential task and streaming data, is the core of continual learning. Firstly, through the investigation and summary of the related work of continual learning at home and abroad in recent years, continual learning methods can be roughly divided into three categories: replay-based, constraint-based, and architecture-based. We further subdivide these three types of methods. Specifically, the replay-based methods are subdivided into three categories: sample replay, generation replay, and pseudo-sample replay, according to the sample’s sources used; the constraint-based methods are subdivided into parameter constraints, gradient constraints, and data constraints, according to the constraint’s sources; the architecture-based methods are subdivided into two categories: parameter isolation and model expansion, according to how the model structure is used. By comparing the innovation points of related work, the advantages and disadvantages of various methods are summarized. Secondly, the research progress at home and abroad is analyzed. Finally, the future development direction of continual learning combined with other fields is simply prospected.

deep learning  /  knowledge transfer  /  continual learning  /  catastrophic forgetting  /  sequential task
李文斌, 熊亚锟, 范祉辰, 邓波, 曹付元, 高阳. 持续学习的研究进展与趋势. 计算机研究与发展, 2024 , 61 (6) : 1476 -1496 . DOI: 10.7544/issn1000-1239.202220820
Wenbin Li, Yakun Xiong, Zhichen Fan, Bo Deng, Fuyuan Cao, Yang Gao. Advances and Trends of Continual Learning[J]. Journal of Computer Research and Development, 2024 , 61 (6) : 1476 -1496 . DOI: 10.7544/issn1000-1239.202220820
Year 2024 volume 61 Issue 6
93
44
Cite this Article
BibTeX
Article Info
doi: 10.7544/issn1000-1239.202220820
  • Receive Date:2022-09-23
  • Online Date:2026-11-17
  • Published:2024-06-01
Article Data
Affiliations
History
  • Received:2022-09-23
  • Revised:2023-09-11
Affiliations
    1State Key Laboratory for Novel Software Technology (Nanjing University), Nanjing 210023
    2Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094
    3School of Computer and Information Technology, Shanxi University, Taiyuan 030006
    4School of Big Data, Shanxi University, Taiyuan 030006
References
Share
https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/10.7544/issn1000-1239.202220820
Share to
QR

Scan QR to access full text

Cite this article
BibTeX
Citations
表12种不同金属材料的力学参数
科
Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)
属
Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏
  • BibTeX
  • EndNote
  • RefWorks
  • TxT