REINFORCEMENT LEARNING FOR LONG-TERMADAPTATION OF AN ARTIFICIAL PANCREAS

GARELLI, FABRICIO; ROSALES, NICOLÁS; SERAFINI, MARÍA CECILIA

Barcelona

Conferencia; 15th International Conference on Advanced Technology & Treatments for Diabetes.; 2022

Background and Aims:Previous clinical trials and in-silicowork show that insulin sensitivity variation in diabetic patients cancompromise the performance of closed loop glycemic control.Generating adaptive controllers is key to overcome this issue.Methods:In this work, a Q-learning based long-term adap-tation technique for the previously introduced Automatic Reg-ulation of Glucose (ARG) algorithm [1] is presented. Thepresented configuration modifies only one parameter in thecontroller (the insulin-on-board limit) instead of replacing itentirely, avoiding the "black box" issue associated with Re-inforcement Learning. The resulting policy is evaluatedin-silicousing the UVA/Padovas?s virtual patient cohort and comparedagainst a manual rule-based strategy.Results:In-silico results shows that the RL agent manages toregulate insulin infusion when the patients sensitivity changes.Particularly, when sensitivity increases, episodes of hypoglyce-mia are avoided without significantly increasing time in hyper-glycemia, while the manual scheme does not achieve similarresults. The llustrative example in the figure shows that severehypoglycemic episodes ocurr when using manual scheme but aresuccesfully avoided by the long-term adaptation of the controller.It is also worth noting that insulin infusion is reduced showingthat the RL strategy improves insulin infusion profile, as well asoverall glycemic excursion.Evolution over time for Adult#05 using the ARG algorithmwith the RL agent (purple thinner) and with manual scheme(orange thicker). At bottom: IOB (solid line) and IOB limit(dashed line).Conclusions:Reinforcement learning for long-term adapta-tion shows great promise as it modifies insulin infusion, in-creasingoverallperformanceandsuccesfullyavoidinghypoglycemia when patient?s sensitivity changes