HiTrain: Heterogeneous Memory Offloading and I/O Optimization for Large Language Model Training
1College of Intelligence and Computing, Tianjin University, Tianjin 300350
2Research Center for High Efficiency Computing Infrastructure, Zhejiang Lab, Hangzhou 311121
3Department of Computer Science and Technology, Tsinghua University, Beijing 100084
4Department of Information and Communication Engineering, Xiamen University, Xiamen, Fujian 361005
|
born in 1993. PhD candidate, assistant researcher. Member of CCF. His main research interests include distributed computing and large model system
|
|
born in 1993. PhD, assistant researcher. Member of CCF. His main research interest includes AI systems and algorithms
|
|
born in 1999. Master candidate. Student member of CCF. His main research interests include AI systems and storage systems
|
|
born in 1997. PhD, assistant researcher. Member of CCF. His main research interests include storage systems and memory systems
|
|
born in 1985. PhD, professor, PhD supervisor. Member of CCF. His main research interest includes cloud computing system
|
|
born in 1971. PhD, professor, PhD supervisor. Fellow of CCF. His main research interests include computer networks and cloud computing
|
收起