收藏切换
A Recommendation Model Inference System Based on GPU Direct Storage Access Architecture
收藏切换
Minhui Xie, Youyou Lu*, Yangyang Feng, Jiwu Shu
Journal of Computer Research and Development | 2024, 61(3) : 589 - 599
Less
收藏切换
Journal of Computer Research and Development | 2024, 61(3): 589-599
• Special Issue on Intelligent Storage and Data Analysis for the Digital Economy •
A Recommendation Model Inference System Based on GPU Direct Storage Access Architecture
Full
Minhui Xie, Youyou Lu*, Yangyang Feng, Jiwu Shu
Affiliations
  • 1Department of Computer Science and Technology, Tsinghua University, Beijing 100084
  • born in 1997. PhD candidate. Student member of CCF. His main research interests include storage systems and machine learning systems

    born in 1987. PhD, associate professor, PhD supervisor. Senior member of CCF. His main research interest includes storage systems

    born in 1998. PhD candidate. Student member of CCF. His main research interests include storage systems and machine learning systems

    born in 1968. PhD, professor, PhD supervisor. Fellow of CCF. His main research interests include non-volatile memory systems and technologies, storage security and reliability, and parallel and distributed computing

About Author:

born in 1997. PhD candidate. Student member of CCF. His main research interests include storage systems and machine learning systems

born in 1987. PhD, associate professor, PhD supervisor. Senior member of CCF. His main research interest includes storage systems

born in 1998. PhD candidate. Student member of CCF. His main research interests include storage systems and machine learning systems

born in 1968. PhD, professor, PhD supervisor. Fellow of CCF. His main research interests include non-volatile memory systems and technologies, storage security and reliability, and parallel and distributed computing

Published: 2024-03-02 doi: 10.7544/issn1000-1239.202330402
Outline
收藏切换

Emerging deep learning recommendation models (DLRM) have been widely used in modern recommendation systems. The unique embedding layer of DLRM, commonly with tens of trillions of parameters, induces massive irregular access to storage resources, which becomes the performance bottleneck of model inference. Existing inference systems rely on CPU access to embedding parameters on DRAM and SSD. However, we find that this architecture suffers from excessive CPU-GPU communication overhead and redundant memory copies, resulting in increased latency of embedding layers and limited inference performance. In this paper, we propose GDRec, a recommendation model inference system based on the architecture of GPU direct storage access. The core idea of GDRec is to eliminate the CPU from the access path of embedding parameters and let the GPU directly access storage resources with the paradigm of zero copy. For direct access to DRAM, GDRec retrofits the unified virtual addressing feature of CUDA, to allow GPU kernels to issue fine-grained access to host DRAM. GDRec further introduces two optimizations, access coalescing and access aligning, to fully unleash the performance of DRAM access. For direct access to SSD, GDRec implements a lightweight NVMe driver on GPU, allowing GPU to submit I/O commands to read data from SSD to GPU memory directly, without extra copies on DRAM. GDRec also leverages the massive parallelism of GPU to shorten the submission time of I/O commands. Experiments on three public datasets show that GDRec can improve inference throughput by 1.9 times, compared with a highly-optimized recommendation model inference system, NVIDIA HugeCTR.

GPU direct storage access  /  parameter store  /  recommendation system  /  inference system  /  storage system
谢旻晖, 陆游游, 冯杨洋, 舒继武. 基于GPU直访存储架构的推荐模型预估系统. 计算机研究与发展, 2024 , 61 (3) : 589 -599 . DOI: 10.7544/issn1000-1239.202330402
Minhui Xie, Youyou Lu, Yangyang Feng, Jiwu Shu. A Recommendation Model Inference System Based on GPU Direct Storage Access Architecture[J]. Journal of Computer Research and Development, 2024 , 61 (3) : 589 -599 . DOI: 10.7544/issn1000-1239.202330402
Year 2024 volume 61 Issue 3
75
34
Cite this Article
BibTeX
Article Info
doi: 10.7544/issn1000-1239.202330402
  • Receive Date:2023-05-23
  • Online Date:2026-11-17
  • Published:2024-03-02
Article Data
Affiliations
History
  • Received:2023-05-23
  • Revised:2023-10-16
Affiliations
    1Department of Computer Science and Technology, Tsinghua University, Beijing 100084
References
Share
https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/10.7544/issn1000-1239.202330402
Share to
QR

Scan QR to access full text

Cite this article
BibTeX
Citations
表12种不同金属材料的力学参数
科
Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)
属
Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏
  • BibTeX
  • EndNote
  • RefWorks
  • TxT