← Research

Offline reinforcement learning with uncertainty for treatment strategies in sepsis

Ran Liu (1, 2), Joseph L. Greenstein (1, 2), James C. Fackler (3), Jules Bergmann (3), Melania M. Bembea (3, 4), Raimond L. Winslow (1, 2) ((1) Institute for Computational Medicine, the Johns Hopkins University, (2) Department of Biomedical Engineering, the Johns Hopkins University School of Medicine, Whiting School of Engineering, (3) Department of Anesthesiology, Critical Care Medicine, the Johns Hopkins University, (4) Department of Pediatrics, the Johns Hopkins University School of Medicine)

Guideline-based treatment for sepsis and septic shock is difficult because sepsis is a disparate range of life-threatening organ dysfunctions whose pathophysiology is not fully understood. Early intervention in sepsis is crucial for patient outcome, yet those interventions have adverse effects and are frequently overadministered. Greater personalization is necessary, as no single action is suitable for all patients. We present a novel application of reinforcement learning in which we identify optimal recommendations for sepsis treatment from data, estimate their confidence level, and identify treatment options infrequently observed in training data. Rather than a single recommendation, our method can present several treatment options. We examine learned policies and discover that reinforcement learning is biased against aggressive intervention due to the confounding relationship between mortality and level of treatment received. We mitigate this bias using subspace learning, and develop methodology that can yield more accurate learning policies across healthcare applications.

PDF ↗

Models

Loading
EVIDENCE / COMPARISON

Benchmarks

Explore every benchmark

Loading

These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

Install TensorTrace