Reinforcement Learning Approach to Sedation and Delirium Management in the Intensive Care Unit

Common treatments in Intensive Care Units frequently involve prolonged sedation. Maintaining adequate sedation levels is challenging and prone to errors including: incorrect dosing, omission/delay in administration and, selecting a sub-optimal combination of sedatives. In this single-center retrospective study of 1,346 patients, we use a Deep Q Network approach to develop a multi-objective sedation management agent. The agent's objective was to achieve an adequate level of patient sedation without moving the patient's Mean Arterial Pressure (MAP) outside of a therapeutic range. To achieve this objective, the agent was allowed to periodically (every 4 hours) recommend how the dose of two commonly used sedatives (propofol, midazolam) and an opioid (fentanyl) should be adjusted: increased, decreased, or stay the same. To inform it's recommendations, the agent was provided with the patient's demographym and periodic measures including: vital signs, and depth of sedation. To mitigate the potential risk of delirium and the adverse effects of over sedation, a delirium control variable was integrated into the agent's reward function. We found that Physicians with dosing policies that agreed with our agent were 29% more likely to maintain the patient's sedation in a therapeutic range, compared to those that disagreed with our agent's policy.