收藏切换
Using reinforcement learning to establish a prediction model of precise fluid therapy for patients with sepsis
收藏切换
PDF
Pan Pan1, Fei Xie1, Li-Xin Xie1, *, Long-Xiang Su2, 3, *
Medical Journal of Chinese People’s Liberation Army | 2022, 47(6) : 579 - 585
Less
收藏切换
Medical Journal of Chinese People’s Liberation Army | 2022, 47(6): 579-585
Clinical Research
Using reinforcement learning to establish a prediction model of precise fluid therapy for patients with sepsis
Full
Pan Pan1, Fei Xie1, Li-Xin Xie1, *, Long-Xiang Su2, 3, *
Affiliations
  • 1College of Pulmonary and Critical Care Medicine, Chinese PLA General Hospital, Beijing 100091, China
  • 2Department of Critical Care Medicine, Peking Union Medical College Hospital,Chinese Academy of Medical Sciences, Beijing 100730, China
  • 3Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital,Chinese Academy of Medical Sciences, Beijing 100730, China
Published: 2022-06-28 doi: 10.11855/j.issn.0577-7402.2022.06.0579
Outline
收藏切换

Objective To explore a possible solution in clinical practice of fluid therapy for patients with sepsis by reinforcement learning method. Methods A total of 11 913 patients with sepsis were screened by using the Medical Information Mark for Intensive Care (MIMIC) Ⅲ Database, and randomly divided into a training set and a test set according to the ratio of 8:2. Twenty-six features were used in modeling, including 24 state features of patients (bloc, vital signs, laboratory tests, blood gas analysis index and basic information), 1 action feature (liquid inflow and outflow difference) and 1 outcome feature (outcome in ICU). Data rules of SARSA model learning training set were used to get the relationship between return rewards and mortality, so as to evaluate whether the return rewards were reasonably set. Deep Q-learning (DQN), a deep learning model based on Q-learning,models the relationship between the state and behavior of the test set, predicts the patients' fluid balance, and compares the results of reinforcement learning and the actual outcomes of patients, which further proved the different effects of predicted liquid therapy and actual therapy on prognosis. Results According to the behavior category distribution, the differences of liquid inflow and outflow were divided into 5 intervals (–3000 to –239.40 ml, –239.39 to –1.94 ml, –1.93 to 160.00 ml, 160.01 to 363.58 ml, and 363.59 to 3000 ml). The SARSA model calculated the training data set, results showed that the higher the Q (s, a) return, the lower the mortality rate. The DQN model suggested that both too high and too low of the difference between the liquid input and output volume may increase the case mortality, and the mortality of patients is higher in low difference of inflow and outflow than in high difference of inflow and outflow volume. Using Doubly robust estimator to evaluate the DQN model average expected return of the test set showed the stability of the model (Q-learning iteration number >20 000). The use of validation set hinted that the mortality was obvious lower in the subgroups predicted dehydration consistent with the reality than in the other three subgroups, indicating that the model can be used in actual clinical verification. Conclusion A predictive model for possibly guiding the fluid therapy on patients with sepsis is proposed using the reinforcement learning method, which can accurately predict the direction of liquid therapy,patients got a better prognosis by using the model predicted dehydration treatment and dehydration was actually carried out.

sepsis  /  fluid therapy  /  reinforcement learning  /  prognosis  /  predictive model
Pan Pan, Fei Xie, Li-Xin Xie, Long-Xiang Su. Using reinforcement learning to establish a prediction model of precise fluid therapy for patients with sepsis[J]. Medical Journal of Chinese People’s Liberation Army, 2022 , 47 (6) : 579 -585 . DOI: 10.11855/j.issn.0577-7402.2022.06.0579
  • Young Elite Scientist Sponsorship Program by Bast of Beijing Association for Science and Technology(BYESS2022035)
  • Key Subject in the Eighth Medical Center of Chinese PLA General Hospital(2021ZD001)
  • China Postdoctoral Science Foundation(2021T140794)
Year 2022 volume 47 Issue 6
PDF
120
46
Cite this Article
BibTeX
Article Info
doi: 10.11855/j.issn.0577-7402.2022.06.0579
  • Receive Date:2021-06-29
  • Online Date:2025-12-17
  • Published:2022-06-28
Article Data
Affiliations
History
  • Received:2021-06-29
  • Accepted:2021-10-29
Funding
Young Elite Scientist Sponsorship Program by Bast of Beijing Association for Science and Technology(BYESS2022035)
Key Subject in the Eighth Medical Center of Chinese PLA General Hospital(2021ZD001)
China Postdoctoral Science Foundation(2021T140794)
Affiliations
    1College of Pulmonary and Critical Care Medicine, Chinese PLA General Hospital, Beijing 100091, China
    2Department of Critical Care Medicine, Peking Union Medical College Hospital,Chinese Academy of Medical Sciences, Beijing 100730, China
    3Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital,Chinese Academy of Medical Sciences, Beijing 100730, China

Corresponding:

* Xie Li-Xin, E-mail: ;
Su Long-Xiang, E-mail:
References
Share
https://castjournals.cast.org.cn/joweb/jfjyxzz/EN/10.11855/j.issn.0577-7402.2022.06.0579
Share to
QR

Scan QR to access full text

Cite this article
BibTeX
Citations
表12种不同金属材料的力学参数

Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)

Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏
  • BibTeX
  • EndNote
  • RefWorks
  • TxT