Reinforcement learning for tuning parameters of closed-loop controllers.

CECILIA SERAFINI; ROSALES, NICOLÁS; FABRICIO GARELLI

Conferencia; 14th International Conference on Advanced Technology & Treatments for Diabetes.; 2021

Background and AimsPrevious work with AP systems shows that initial tuning of the closed-loop parameters is key to enhance performance. Even short-term clinical trials show better results when the result analysis is concentrated in the last hours. MethodIn this work, a Reinforcement Learning based auto-tuning technique for the previously introduced Automatic Regulation of Glucose (ARG) algorithm [1] is evaluated using a self-developed code. This technique modifies only one parameter in the AP system (the insulin-on- board -IOB- limit) instead of replacing the controller entirely.The resulting tuning strategy is evaluated in-silico using the FDA-accepted UVA/Padova simulator to test the initial tuning of the IOB limit of the ARG controller and tested against a manual tuning scheme. Lastly, the RL strategy is applied retroactively to data collected in clinical trials.ResultsThe in-silico results show that the auto-tuning by means of RL achieves total elimination of hypoglycemic events in few episodes for the whole cohort and none of the manual actions achieve the same result by themselves. In summary, this preliminary in-silico study indicates that the use of RL for auto-tuning of AP systems shows great potential for future applications.When tested as a recommendation system against past clinical data, the RL policy suggests that its use could have led to better results on the past clinical trials, showing promise for future work.ConclusionSimulations show that the proposed tuning strategy improves the performance of the ARG algorithm, as it reduces excursion and insulin injection.