收藏切换
Survey of Storage Optimization Techniques in Large Language Model Inference
收藏切换
Xuran Ge1, Yang Ou1, *, Bo Wang2, Yu Zhao3, Lizhou Wu1, Zicong Wang1, Zhiguang Chen2, Nong Xiao1
Journal of Computer Research and Development | 2025, 62(3) : 545 - 562
Less
收藏切换
Journal of Computer Research and Development | 2025, 62(3): 545-562
• Special Issue on Storage Management and Data Analysis in the Era of Large Language Models •
Survey of Storage Optimization Techniques in Large Language Model Inference
Full
Xuran Ge1, Yang Ou1, *, Bo Wang2, Yu Zhao3, Lizhou Wu1, Zicong Wang1, Zhiguang Chen2, Nong Xiao1
Affiliations
  • 1College of Computer Science and Technology, National University of Defense Technology, Changsha 410073
  • 2School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou 510006
  • 3School of Informatics, University of Edinburgh, Edinburgh EH89AB
  • born in 1998. PhD candidate. Student member of CCF. Her main research interests include key-value storage system and large language model inference system

    born in 1987. PhD, associate professor. His main research interests include computer architecture and high performance interconnection technology

    born in 1997. PhD candidate. Student member of CCF. His main research interests include distributed shared memory and parallel programming frameworks

    born in 1997. PhD candidate. His main research interest includes efficient inference and pre-training of large language model

    born in 1991. PhD, assistant professor. His main research interests include high performance computing, spintronic design and test, and emerging computing paradigms based on non-volatile memory devices

    born in 1991. PhD, assistant professor. His main research interests include computer architecture and high performance interconnection technology

    born in 1984. PhD, professor,PhD supervisor. Member of CCF. His main research interests include big data storage, and high performance computing and supercomputers

    born in 1969. PhD, professor,PhD supervisor. His main research interests include high performance computing, storage system, and system architecture

About Author:

born in 1998. PhD candidate. Student member of CCF. Her main research interests include key-value storage system and large language model inference system

born in 1987. PhD, associate professor. His main research interests include computer architecture and high performance interconnection technology

born in 1997. PhD candidate. Student member of CCF. His main research interests include distributed shared memory and parallel programming frameworks

born in 1997. PhD candidate. His main research interest includes efficient inference and pre-training of large language model

born in 1991. PhD, assistant professor. His main research interests include high performance computing, spintronic design and test, and emerging computing paradigms based on non-volatile memory devices

born in 1991. PhD, assistant professor. His main research interests include computer architecture and high performance interconnection technology

born in 1984. PhD, professor,PhD supervisor. Member of CCF. His main research interests include big data storage, and high performance computing and supercomputers

born in 1969. PhD, professor,PhD supervisor. His main research interests include high performance computing, storage system, and system architecture

Published: 2025-03-01 doi: 10.7544/issn1000-1239.202440628
Outline
收藏切换

In recent years, LLM (large language model) has exhibited remarkable performance, profoundly transforming various aspects of human life. As these models grow in size and user demand for long-context inference increases, LLM inference systems face significant storage challenges. These challenges stem primarily from the vast number of model parameters and the key value cache required for efficient inference, both of which strain GPU memory resources. Additionally, inefficiencies in storage usage in distributed systems often result in over-provisioning and fault tolerance issues, further complicating resource management. Researchers explore memory optimization, heterogeneous storage, and distributed storage, synthesizing various research efforts to address GPU memory constraints and enhance resource utilization. Memory-optimized LLM inference systems improve GPU memory efficiency and reduce memory footprint through techniques like efficient key value cache management, compression, and attention operator optimization. Heterogeneous storage based LLM inference systems expand storage capacity by integrating various storage resources, thereby minimizing I/O overhead via tensor placement strategies, asynchronous data transfer, and intelligent memory allocation and prefetching methods. Distributed LLM systems enhance the utilization of multi-machine resources, boosting execution efficiency and fault tolerance in LLM inference tasks through batching, multi-level scheduling, and redundant replication. Finally, we review existing research and outline future research directions to further optimize storage solutions for LLM inference systems.

LLM inference system  /  memory management  /  heterogeneous storage  /  distributed storage  /  fault recovery  /  serverless LLM inference
葛旭冉, 欧洋, 王博, 赵宇, 吴利舟, 王子聪, 陈志广, 肖侬. 大语言模型推理中的存储优化技术综述. 计算机研究与发展, 2025 , 62 (3) : 545 -562 . DOI: 10.7544/issn1000-1239.202440628
Xuran Ge, Yang Ou, Bo Wang, Yu Zhao, Lizhou Wu, Zicong Wang, Zhiguang Chen, Nong Xiao. Survey of Storage Optimization Techniques in Large Language Model Inference[J]. Journal of Computer Research and Development, 2025 , 62 (3) : 545 -562 . DOI: 10.7544/issn1000-1239.202440628
Year 2025 volume 62 Issue 3
116
51
Cite this Article
BibTeX
Article Info
doi: 10.7544/issn1000-1239.202440628
  • Receive Date:2024-07-20
  • Online Date:2026-11-10
  • Published:2025-03-01
Article Data
Affiliations
History
  • Received:2024-07-20
  • Revised:2025-01-20
Affiliations
    1College of Computer Science and Technology, National University of Defense Technology, Changsha 410073
    2School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou 510006
    3School of Informatics, University of Edinburgh, Edinburgh EH89AB
References
Share
https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/10.7544/issn1000-1239.202440628
Share to
QR

Scan QR to access full text

Cite this article
BibTeX
Citations
表12种不同金属材料的力学参数
科
Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)
属
Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏
  • BibTeX
  • EndNote
  • RefWorks
  • TxT