Peer Review of “In-Silico Works Using an Improved Hovorka Equations Model and Clinical Works on the Control of Blood Glucose Levels in People With Type 1 Diabetes: Comparison Study”

This is a peer-review report submitted for the paper “In-Silico Works Using an Improved Hovorka Equations Model and Clinical Works on the Control of Blood Glucose Levels in People With Type 1 Diabetes: Comparison Study.”

Round 1 Review

General Comments

This paper [1] presents a preliminary validation with clinical data of a new glucose-insulin model proposed by the authors in other publications. The paper is well organized and discusses a topic of interest in the field of artificial pancreas.

The authors conclude that the new model is a good predictor for blood glucose levels. However, they also mention that the model yields better glucose metrics than the observed clinical data. In my opinion, these two statements are contradictory, so I would request authors to elaborate on this point more.

Specific Comments

I am afraid that I am doubtful of some methodological aspects of the paper, so I need more justification for them. These concerns are listed in Major Comments. In addition, minor typos and other suggestions are presented in Minor Comments.

Major Comments

1. I would be grateful if the author completed the description of the data collection. It is unclear whether the study was deliberately designed to validate the “improved Hovorka model,” or in contrast, data were initially collected for other purposes. Additionally, I could not find if the study was performed at each patient’s home or, instead, it was a controlled study in the hospital. In addition, I missed information about insulin therapy (closed-loop or open-loop). Finally, in the Results section (line 140), the authors state that the high glucose levels observed in the clinical data may be because of exercise. Does it mean that the study protocol allows the patients to practice physical activity?

2. Preliminary validations of widely used glucose-insulin models such as Hovorka’s [2] or Dalla-Man’s [3] used short-duration trials (less than a day) but with frequent measurement to gain more considerable insight into glucose variations. However, the experiment devised in this manuscript has a longer duration but much less frequent measurements. As the authors state in the Results section, line 139, this lack of measurements may mislead the calculated time in normoglycemia. Could the authors explain why they did not design an experiment with more frequent measurements?

3. The authors indicated that three insulin rates were simulated (lines 75 and 76), and insulin boluses were adjusted by trial and error to optimize the glucose profile (line 101). Nevertheless, I could not find the insulin rates and boluses used in the clinical trial. Were they the same as for the simulation? If not, could you justify this decision, please? In my opinion, using different insulin inputs in the model than in the actual patient will lead to noncomparable outputs.

4. It is unclear how the authors performed the regression statistical analysis (line 135). Did they compare specific blood glucose samples in the clinical data to the corresponding simulated data points, or did they compare some fitting error metric like the root mean square error or glucose performance metric such as the time in range? In addition, it would be helpful if the authors provided which type of regression model they employed (eg, linear model, generalized model, multilevel model).

5. The authors concluded that the model “is acceptable to predict the BGL for people with T1D” (line 137). However, they also stated that “all patients showed improvement in BGL for the in-silico works.” In my opinion, these two statements are contradictory: if the in-silico model did not reproduce, with acceptable errors, the glucose profiles observed in the clinical trial, then the model cannot be considered a good predictor. Could the author explain this point more, please?

Minor Comments

6. I could not find any information about how the authors identify the model’s parameters. Could the author describe, please, how the model was calibrated?

7. I could not find any referenced table in the manuscript.

8. Lines 17-19: In the introduction, it seems that the authors presented the “improved Hovorka model” to address the poor performance achieved by current artificial pancreas systems. Could the authors please elaborate more on how the model they presented will enhance the performance of existing control algorithms?

9. The authors used “workers” in several places in the manuscript. Did they mean “works”?

10. Lines 43 and 44: Equation 1 seems to be missing the last term.

11. Lines 63 and 64: I think equation 8 contains a typo. Should the second “=” be removed?

12. Figures 2-4: The collected clinical data comprises six glucose samples per patient. However, data was represented with a continuous line. In my opinion, this representation leads to a misleading interpretation (eg, glucose follows a horizontal line in some periods, which is unrealistic). I suggest the authors mark the actual blood samples as in a scatter plot.

13. Figures 2-4: I think the x-axes should be in hours, not in minutes.

Round 2 Review

First, I would like to thank the authors for completing the clinical trial and statistical analysis description and for addressing all my comments on the previous submission. Unfortunately, I am afraid that I still have some methodological doubts regarding the comparison between the in-silico results and the clinical results. Therefore, I suggest a further revision of the manuscript.

Specific Comments

My principal methodological concerns are listed in the Major Comments section. Other doubts or typos are presented in the Minor Comments section.

Major Comments

1. If I understood well, the insulin infusion rate used to simulate the “Improved Hovorka model” (variable u(t) in equation 6) is different from the infusion rate administered to the actual patients in the clinical trial. On the one hand, virtual patients have received an insulin infusion rate calculated from a closed-loop algorithm (the enhanced model-based predicted control). On the other hand, actual patients seem to follow an open-loop therapy (multiple drug injections). Lastly, the authors also state (lines 361 and 362) that the timing of insulin bolus is different in both settings. Could the authors explain these discrepancies, please? If the goal is to compare the prediction ability of the “Improved Hovorka model,” why have the authors not simulated the model with the same insulin therapy used for the clinical trial?

2. From the regression analysis results (lines 406-409), the authors seem to conclude that the model is “applicable in predicting BGL” because the P value is <.01. However, I cannot see the relation between a significant P value and a better prediction. From the clear description the authors provided of the multiple regression analysis, I think the authors fitted the following linear model:

improved_hovorka_glucose = insulin·beta1 + meal·beta2 + e

where “improved_hovorka_glucose” is the output of the “Improved Hovorka model,” “insulin” and “meal” correspond to the values of the infusion rate and meal amount in that model, “beta1” and “beta2” are coefficients to be estimated in the analysis, and “e” is the normal distributed residuals. A P value <.05 means that data supports the rejection of the null hypothesis that beta1 = beta2 = 0 [2]. Thus, a significant P value indicates that the “insulin” and/or the “meal” inputs can explain the variations observed in “improved_hovorka_glucose.” However, I cannot see how one can conclude anything from the prediction accuracy of the “Improved Howorka model” from the fact that beta1≠0 or beta2≠0. Could the author explain this point, please?

Minor Comments

3. Equation 5: The term exp(t/maxG) should be exp(-t/maxG).

4. Equation 6: In line 103, the authors define u(t) as insulin bolus. However, in line 180, the authors refer to infusion rates. Could the author check the consistency of this definition?

Round 3 Review

I would like to thank the authors for their efforts in replying to my comments. Unfortunately, I still do not understand the article’s contribution regarding comparing clinical data. As stated by the authors, clinical data and in-silico results are not comparable due to the different methodologies and protocols applied to obtain the data; therefore, I wonder if including the clinical data set analysis is justified. In addition, I have doubts about under which conditions the authors have simulated the model, for instance, if the simulation included any kind of variability.

Specific Comments

Major Comments

1. Since clinical data results and in-silico data are incomparable, could the authors justify the motivation for including the clinical data in the article?

2. In lines 191-193, the authors indicate that comparing the in-silico data with the clinical data would help determine the model’s accuracy in mimicking the actual glucose. I believe this statement is incompatible with the fact that both clinical and in-silico data were obtained following incomparable protocols and methodologies.

3. Have the authors thought about modifying the simulation of the model to make the results more comparable with the clinical data? For instance, I would suggest they simulate the model with the same bolus, basal insulin, and meal carbohydrates utilized in the clinical trial. Then, they could compare the model output with each glucose measurement.

4. In the Discussion section (lines 374-378) and the conclusion (line 404), the authors concluded from Table 13 that the patients have less sensitivity in the morning. Since Table 13 corresponds to the results of the virtual patients in the in-silico analysis, I wonder whether the authors have included any kind of circadian variability in the simulation, for instance, some sinusoidal variability in ka1, kw1, kw11, ka2, kw2, kw22, ka3, kw3, or kw33. If this is not the case and these parameters were kept constant in the simulation, I suggest authors better justify this apparent increase in insulin sensitivity.

5. The authors said the insulin bolus was computed by trial and error. Since one of the article’s goals is determining the optimal bolus, it would be advisable to detail the method followed to calculate it.

Round 4 Review

I would like to thank the authors for replying to my comments. Unfortunately, I still believe the work has two principal limitations preventing me from accepting the manuscript. The main one is that the differences in protocols and conditions between the clinical and simulation works make it, in my opinion, unfeasible to address the goal of determining “the accuracy and effectiveness of the in-silico model in mimicking real-world BGL dynamics.” The second one is that insufficient information is reported to reproduce the calculation of the optimal bolus in the in-silico simulations.