Authors’ Response to Peer Reviews of “In-Silico Works Using an Improved Hovorka Equations Model and Clinical Works on the Control of Blood Glucose Levels in People With Type 1 Diabetes: Comparison Study”

This is the authors’ response to peer-review reports for “In-Silico Works Using an Improved Hovorka Equations Model and Clinical Works on the Control of Blood Glucose Levels in People With Type 1 Diabetes: Comparison Study.”

Round 1 Review

Reviewer AN [1]

General Comments

The paper [2] simulates the ability of a new version of the Hovorka model to simulate the blood glucose level (BGL) of type 1 diabetes (T1D) for 3 patients with meal disturbances for 24 hours. The simulation was done using MATLAB software, and the BGL profile from both simulation and clinical works were compared and analyzed. While the P values for the simulation and clinical data were <.05, indicating that the simulation work using the improved Hovorka equations was acceptable for predicting the BGL, results showed that the BGLs for all 3 people with T1D were lower in the simulation work compared to the clinical work.

Response: Thank you for the compliments.

Specific Comments

Major Comment

1. The paper is well written, the experiments seem correctly designed, and the results seem reasonable. However, the most interesting result is that the simulated BGL results were consistently lower than the clinical results. While the authors discuss some clinical reasons for this systematic difference the hypotheses are not terribly compelling. I think it is also necessary to discuss that the simulation/model may have some systematic bias due to the assumptions of its construction. The model may be an effective low BSL baseline estimate for a patient as opposed to an effective expected value estimate.

In some sense, this is an unexpected result from the model, but it does not make the model invalid. Explicitly stating this, characterizing it in an established taxonomy of unexpected behaviors for simulations, and discussing how the model can still be valid would improve the paper and increase its maturity, in terms of its application of modeling and simulation.

Papers/books to support this effort:

• Mittal S, Diallo S, Tolk A.Emergent Behavior in Complex Systems Engineering: A Modeling and Simulation Approach. John Wiley & Sons; 2018.
• Gore R, Reynolds PF. An exploration-based taxonomy for emergent behavior analysis in simulations. Presented at: 2007 Winter Simulation Conference; December 9-12, 2007; Washington, DC.

Response: Thank you for the compliments and constructive comments. We have taken the recommendations seriously by explaining it in greater length in the revised manuscript as follows:

Discussion section (lines 335-352): “The comparison of BGL against time between clinical and in-silico works can be challenging, especially when clinical data is limited, and in this case the CGM device is not used. Thus, the BGL profiles are different since patients in the clinical work use conventional method to monitor their BGL i.e. SMBG and MDI; therefore, only a snapshot of BGL at a particular time available for comparison as seen in Figures 5 to 7. Studies have shown that the use of CGM device can improve time in range in the clinical settings, thus improve the BGL profiles [3,4].

“In this case, the only available data point is the focus since the clinical data is limited and does not cover the entire time span. While BGL simulations can be helpful for predicting how a T1D patient’s BGL may change under different conditions, they are not always accurate. This is because the mathematical models used in the simulations are based on assumptions about how the body works, and these assumptions may not always hold for every individual. Additionally, the simulation may not consider all of the complex factors that can affect BGL, such as exercise, stress or illness. BGL monitoring can also be subjected to errors and variability in clinical settings. Factors such as the accuracy of the glucose meter or sensor, the timing and frequency of measurements, and the variability of patients' responses to interventions (meals, physical activities, and medication for example) can all affect the reliability and accuracy of clinical BGL monitoring.”

2. The paper refers to many tables (1-6) that are not present in the text. The data in these tables are needed for the presentation of the material (ie, they need to be present in the paper) and certainly should be present if referenced by the authors.

Response: Tables 1-6 are included in the revised manuscript plus their respective references. More tables have also been added to give insightful information to further support the findings and conclusion.

3. The importance of the issue (T1D) and regulating BGLs has the potential to impact millions of people. In addition, being able to estimate this (even a low-end estimate) with modeling reduces material costs, time, and patient risk. However, this context establishing the impact and importance of the paper is missing. Adding this will help readers appreciate the impact (and cite) the paper.

Response: Thank you for the constructive comments.

This statement was added in the revised manuscript as recommended.

Introduction section (lines 16-19): “The importance of the issue (T1D) and regulating BGL by means of the APD has the potential to impact millions of people. In addition, being able to estimate this (even a low-end estimate) with modelling work reduces materials cost, time and patient risk.”

Minor Comments

4. In the replication crisis era, the MATLAB software the scripts used to create the graphics should be provided to the reader and reviewers.

Response: Some examples of the scripts used in MATLAB are provided to the reader and reviewers in the revised manuscript in the case of enhanced model-based predicted control (eMPC) applications for the control system algorithm. Please refer to Figures 6 and 7 (lines 191 and 192).

5. The abstract reads as if it was written continuously (ie, subsections infer context from previous subsections). This is not how JMIR abstracts are written. The subsections within the abstract should be able to be read independently.

Response: Thank you for the comments. All necessary corrections have been made in the revised abstract:

“Background: People with Type 1 Diabetes (T1D) depend on external insulin to regulate their BGL within the normoglycemic range between 4.0 to 7.0 mmol/L. T1D patients routinely conduct self-monitoring of blood glucose (SMBG) through finger pricks prior to insulin injections. Artificial Pancreas (AP) is an innovative device that mimics the function of a healthy pancreas. Despite its recent advancement, the control algorithms used in the AP are still lagging in delivering the proper insulin dosage to T1D patients. Previous researchers attempted to improve the interrelation between parameters and variables in the original Hovorka model equations, later known as improved Hovorka model equations; however, the improved model equations have not yet been tested in terms of its usability to regulate and control the BGL in safe range for two or more people with T1D.

“Objectives: This study aimed to simulate the improved Hovorka model equations using actual patients’ data via MATLAB programming coupled with enhanced model-based predicted control (eMPC) and determine the optimum bolus insulin. The study then compares the performance results obtained from in-silico and clinical works.

“Methods: Three actual patients’ data were collected from Clinic 1, Clinical Training Centre, Universiti Teknologi MARA (UiTM) Hospital, Sungai Buloh, Selangor upon getting approval from UiTM Ethics Committee. The inclusion criteria of subjects were namely; T1D patients, age range between 11 to 14 years old, highly dependent on insulin injection with four or more finger pricks or self-monitoring of blood glucose (SMBG) for BGL measurements per day. T1D patients attended the clinic every three months and require blood taking as routine follow-up care. The T1D patients typically receive meals three times per day; breakfast, lunch, and dinner. As for data analysis of patients between clinical and in-silico works, P-value via multiple linear regression (MLR) was used to model the relationship between meal, insulin, and BGL.

“Results: Based on observation, in order of breakfast, lunch, and dinner: the optimum bolus insulins for patient 1 were 83.33, 33.33 and 16.67 mU/min; patient 2 were 66.67, 50.01 and 33.33 mU/min, and patient 3 were 100.02, 83.33, and 66.67 mU/min, respectively. As for the in-silico works using improved Hovorka model equations, results revealed that the percentages of time for their BGL on target in patients 1, 2, and 3 were at 79.59%, 87.76%, and 71.43%, respectively, as compared to the clinical works with less than 50%. All patients in both clinical and in-silico works had a significantly small P-value (P<0.01), which indicated there was strong relationship between the independent variables (meals and insulin) and the dependent variable (BGL).

“Conclusions: In conclusion, the in-silico work using the improved Hovorka model equations can be applicable in simulating BGL with meal disturbances for people with T1D.”

Anonymous [5]

General Comments

This paper [1] presents a preliminary validation with clinical data of a new glucose-insulin model proposed by the authors in other publications. The paper is well organized and discusses a topic of interest in the field of artificial pancreas.

The authors conclude that the new model is a good predictor for blood glucose levels. However, they also mention that the model yields better glucose metrics than the observed clinical data. In my opinion, these two statements are contradictory, so I would request authors to elaborate on this point more.

Response: Thank you for the compliments and constructive comments. We have taken the recommendations seriously by explaining it in greater length in the revised manuscript as follows:

Discussion section (lines 335-352): “The comparison of BGL against time between clinical and in-silico works can be challenging, especially when clinical data is limited, and in this case the CGM device is not used. Thus, the BGL profiles are different since patients in the clinical work use conventional method to monitor their BGL i.e. SMBG and MDI; therefore, only a snapshot of BGL at a particular time available for comparison as seen in Figures 5 to 7. Studies have shown that the use of CGM device can improve time in range in the clinical settings, thus improve the BGL profiles [3,4].

“In this case, the only available data point is the focus since the clinical data is limited and does not cover the entire time span. While BGL simulations can be helpful for predicting how a T1D patient’s BGL may change under different conditions, they are not always accurate. This is because the mathematical models used in the simulations are based on assumptions about how the body works, and these assumptions may not always hold for every individual. Additionally, the simulation may not consider all of the complex factors that can affect BGL, such as exercise, stress or illness. BGL monitoring can also be subjected to errors and variability in clinical settings. Factors such as the accuracy of the glucose meter or sensor, the timing and frequency of measurements, and the variability of patients' responses to interventions (meals, physical activities, and medication for example) can all affect the reliability and accuracy of clinical BGL monitoring.”

Specific Comments

I am afraid that I am doubtful of some methodological aspects of the paper, so I need more justification for them. These concerns are listed in Major Comments. In addition, minor typos and other suggestions are presented in Minor Comments.

Response: Thank you for the comment.

Major Comments

1. I would be grateful if the author completed the description of the data collection. It is unclear whether the study was deliberately designed to validate the “improved Hovorka model,” or in contrast, data were initially collected for other purposes. Additionally, I could not find if the study was performed at each patient’s home or, instead, it was a controlled study in the hospital. In addition, I missed information about insulin therapy (closed-loop or open-loop). Finally, in the Results section (line 140), the authors state that the high glucose levels observed in the clinical data may be because of exercise. Does it mean that the study protocol allows the patients to practice physical activity?

Response: Thank you for the constructive comments. Apparently, the study was deliberately designed to validate the improved Hovorka model using the actual patients’ data. As for the clinical data, the actual patients took their BGL via self-monitoring of blood glucose (SMBG; finger prick) at patients’ homes during breakfast, lunch, and dinner times. These finding results were then compared with the findings through simulation or in-silico works (via MATLAB) using the improved Hovorka model proposed by the authors. No physical activity was allowed in the study protocols. For more information of the study protocols and data collection, please refer to the Methods section as outlined in the revised manuscript.

Methods section (lines 32-59):

“Ethics Approval on Data Collection for Clinical Works

“Ethical approval for this study (Ref No: REC/435/19) was granted by the University Teknologi MARA (UiTM) Ethics Committee before data collection was commenced (reference letter: 600-TNCPI (5/1/6) dated 29 October 2019). Data collection and patients’ information required in this study were obtained from Clinic 1, Clinical Training Centre (CTC), UiTM Medical Specialist Centre, UiTM Hospital, Sungai Buloh, Selangor. Information sheets were given, and formal consent for participation from the parents or legal guardians of T1D patients was obtained since the subjects were all minors. During the appointment, parents or guardians of participants, i.e., T1D patients, were allowed to ask questions before signing the consent form. The participants could withdraw from the study at any time without penalty. Participants' details, such as names or other personal identifiers, remain confidential in the researcher's data.

“A total of three (3) T1D patients were recruited following informed consent. The inclusion criteria of subjects were namely; T1D patients, age range between 11 to 14 years old, highly dependent on insulin injection with four or more finger pricks or self-monitoring of blood glucose (SMBG) for BGL measurements per day, multiple daily injections (MDI) of insulin, and with well-documented bolus insulin requirement. Exclusion criteria were T1D patients with evidence of hypoglycaemia unawareness, known or suspected allergy to insulin, established neuropathy, nephropathy, and retinopathy. T1D patients attended the clinic every three months and require blood taking as routine follow-up care. The amount of blood taken by the paediatrician was 5mL for fasting glucose and fasting insulin each and was taken once only. The additional data required include namely; patient’s name, age, gender, race, body weight, body mass index (BMI), type and amounts of meals consumed specifically carbohydrates (CHO), meal time and duration, T1D history (years diagnosed with T1D), fasting plasma glucose level, fasting plasma insulin level, bolus insulin administered, and other relevant information. The T1D patients typically receive meals three times per day; breakfast, lunch, and dinner. The actual patients' data were termed clinical data (clinical work) throughout the study.”

Methods section (lines 74-78):

“Mathematical Model for In-silico Works

“Improved Hovorka equations [6,7] based on Hovorka model [8] are specifically designed for people with T1D. The diagram of the improved Hovorka equations model is illustrated as shown in Figure 1. The model has two inputs: meal disturbances and bolus insulin. It consists of three subsystems: the glucose subsystem, insulin subsystem, and insulin action subsystem.”

Lines 161-165: “Consequently, all simulated data via in-silico works for the three people with T1D were collected and plotted so as to produce such profiles of BGL versus time for each patient, respectively. Upon completion of data collection and construction of sufficient BGL versus time profiles for both clinical and in-silico works, these two finding results were then analysed and compared for any similarity and difference purposes.”

2. Preliminary validations of widely used glucose insulin models such as Hovorka’s [9] or Dalla-Man’s [10] used short-duration trials (less than a day) but with frequent measurement to gain more considerable insight into glucose variations. However, the experiment devised in this manuscript has a longer duration but much less frequent measurements. As the authors state in the Results section, line 139, this lack of measurements may mislead the calculated time in normoglycemia. Could the authors explain why they did not design an experiment with more frequent measurements?

Response: Thank you for the constructive comments. We did not design such an experiment with more frequent measurement for the case of the clinical works because all three patients were not in continuous glucose monitoring (CGM) regime as yet. A CGM device has not been used for all patients with T1D in our clinic at the Universiti Teknologi MARA (UiTM) hospital as we only asked them to use SMBG via finger prick. However, these can be explained as follows:

Discussion section (lines 335-352): “The comparison of BGL against time between clinical and in-silico works can be challenging, especially when clinical data is limited, and in this case the CGM device is not used. Thus, the BGL profiles are different since patients in the clinical work use conventional method to monitor their BGL i.e. SMBG and MDI; therefore, only a snapshot of BGL at a particular time available for comparison as seen in Figures 5 to 7. Studies have shown that the use of CGM device can improve time in range in the clinical settings, thus improve the BGL profiles [3,4].

“In this case, the only available data point is the focus since the clinical data is limited and does not cover the entire time span. While BGL simulations can be helpful for predicting how a T1D patient’s BGL may change under different conditions, they are not always accurate. This is because the mathematical models used in the simulations are based on assumptions about how the body works, and these assumptions may not always hold for every individual. Additionally, the simulation may not consider all of the complex factors that can affect BGL, such as exercise, stress or illness. BGL monitoring can also be subjected to errors and variability in clinical settings. Factors such as the accuracy of the glucose meter or sensor, the timing and frequency of measurements, and the variability of patients' responses to interventions (meals, physical activities, and medication for example) can all affect the reliability and accuracy of clinical BGL monitoring.”

3. The authors indicated that three insulin rates were simulated (lines 75 and 76), and insulin boluses were adjusted by trial and error to optimize the glucose profile (line 101). Nevertheless, I could not find the insulin rates and boluses used in the clinical trial. Were they the same as for the simulation? If not, could you justify this decision, please? In my opinion, using different insulin inputs in the model than in the actual patient will lead to noncomparable outputs.

Response: Thank you for the constructive comments. The insulin rates and boluses for both clinical and in-silico works were the same based on the meals amount taken during breakfast, lunch, and dinner for each patient as described as follows:

Discussion section (lines 354-370): “Bolus insulin is used to control BGL at mealtime. The best time for insulin injection depends on the type of insulin used and the individual’s need in order to achieve optimal BGL targets to reduce the complications of diabetes. The insulin is typically injected subcutaneously, either with a syringe or an insulin pen, and taken shortly before or after a meal [11]. The amount of bolus insulin needed depends on factors such as age, body weight, the amount of CHO in the meal consumed, insulin sensitivity, and physical activity [12-14]. These factors help patients to determine the appropriate dose of bolus insulin needed.

“Patients in the clinical work were given insulin during mealtime as opposed to the in-silico work in which the insulin was injected 30 minutes prior to a meal. Additionally, it is important to consider the timing of bolus insulin administration. Giving bolus insulin too early before a meal or too late after a meal can result in hypoglycaemia and hyperglycaemia, respectively. So, the bolus insulin administration was timed appropriately to match the timing of CHO intake.

“In this study, all patients used a rapid-acting insulin to manage their BGL during mealtime. This is to ensure their BGL would not deviate too far from the normoglycemic range, i.e., 4.0 to 7.0 mmol/L, after each meal. Table 15 summarises the amount of bolus insulin administered for all patients in the in-silico work. The optimum amount of bolus insulin was obtained on a trial-and-error basis. The aim is to get the BGL within or closer to the normoglycemic range while avoiding hypoglycaemia episodes as much as possible.”

4. It is unclear how the authors performed the regression statistical analysis (line 135). Did they compare specific blood glucose samples in the clinical data to the corresponding simulated data points, or did they compare some fitting error metric like the root mean square error or glucose performance metric such as the time in range? In addition, it would be helpful if the authors provided which type of regression model they employed (eg, linear model, generalized model, multilevel model).

Response: Thank you for the constructive comments. The regression statistical analysis was performed as described below:

Method section (Data Analysis of Patients Using Microsoft Excel subsection; lines 199-229): “Data analysis was done using relevant data collected. Previously, the BGL profiles for both works were created. The BGL profiles in the clinical work were done manually by inserting data related and plotting the BGL profiles using Microsoft Excel 2016. Conversely, the BGL profiles in the silico work were generated using MATLAB software. From there, the pattern of BGL profiles was observed and identified, such as the time of day when BGL tends to be the highest or lowest, the frequency of hypoglycemic and hyperglycemic events, and the variability of BGL. The BGL profiles were used to evaluate the patient’s glycemic control over the selected time frame by calculating the average BGL and the percentage of time spent in different glycemic ranges, such as hypoglycaemia, normoglycaemia, and hyperglycaemia, and among other things.

“Thus, Microsoft Excel 2016 was used to facilitate the data analysis work. The following data were used such as the amounts of meals (g CHO), meal time and duration, amount of plasma insulin and plasma glucose. Regression analysis was selected for data analysis, i.e., a statistical method generally used to analyse the relationship between two or more variables. Multiple linear regression (MLR) is a type of regression analysis that was done to analyse the relationship between two or more independent variables and a dependent variable. In this case, the two independent variables were the amounts of meals consumed and insulin administered, while the dependent variable was the predicted BGL. The following steps were performed to obtain MLR.

“The first step was to enter the data into the Excel spreadsheet with one column for the dependent variable and one or more columns for the independent variables. Then, from the data tab, data analysis containing various analysis tools was selected. The regression analysis was chosen where the selected input of the y range was the BGL outcome, and the inputs of the x range were meals consumed and insulin administered. A new worksheet tab appeared, giving the summary output of regression statistics and other relevant information. The probability value (P-value) and coefficient of determination (R-squared, R2) are important statistical measures used in regression analysis. The P-value was used to test the research hypothesis whether to reject or support the null hypothesis. A small P-value (P<0.05) indicates the relationship between the variables are significant and the null hypothesis is rejected. The R-squared (R2) measures how much the independent variables explain the variation in the dependent variable. A high value of R-squared indicates a better fit of the model to the data.”

5. The authors concluded that the model “is acceptable to predict the BGL for people with T1D” (line 137). However, they also stated that “all patients showed improvement in BGL for the in-silico works.” In my opinion, these two statements are contradictory: if the in-silico model did not reproduce, with acceptable errors, the glucose profiles observed in the clinical trial, then the model cannot be considered a good predictor. Could the author explain this point more, please?

Response: Thank you for the constructive comments. Your query can be explained as follows:

Discussion section (lines 329-352): “Observing the BGL trend for all patients in both works, the in-silico work performed better in managing BGL as compared to the clinical work. Patients in the clinical work rarely achieved the glycemic target, 4.0 to 7.0 mmol/L. Patients 1 and 2 only achieved the target range during morning and evening, respectively, whereas patient 3 was none at all. The patients in the in-silico work were able to achieve the glycemic target more than 70% of the time as compared to the clinical which is less than 50%.

“The comparison of BGL control between clinical data and in-silico works can be challenging, especially when clinical data is limited, and in this case the CGM device is not used. Thus, the BGL profiles are different since patients in clinical works use conventional method to monitor their BGL i.e. SMBG and MDI, therefore, only a snapshot of BGL at a particular time available for comparison (as seen in Figures 5-7). Studies had shown that the use of CGM device can improve the time in targt range in the clinical work, thus improve the BGL profiles [3,4].

“In this case, the only available data point is the focus since the clinical data is limited and does not cover the entire time span. While BGL simulations can be helpful for predicting how a T1D patient’s BGL may change under different conditions, they are not always accurate. This is because the mathematical models used in the simulations are based on assumptions about how the body works, and these assumptions may not always hold for every individual. Additionally, the simulation may not consider all of the complex factors that can affect BGL, such as exercise, stress or illness. BGL monitoring can also be subjected to errors and variability in clinical settings. Factors such as the accuracy of the glucose meter or sensor, the timing and frequency of measurements, and the variability of patients' responses to interventions (meals, physical activities, and medication for example) can all affect the reliability and accuracy of clinical BGL monitoring.”

Minor Comments

6. I could not find any information about how the authors identify the model’s parameters. Could the author describe, please, how the model was calibrated?

Response: All model parameters are taken from the previous studies, and they are all provided in the form of tables (Tables 5 and 6) as shown in the revised manuscript.

7. I could not find any referenced table in the manuscript.

Response: All tables are included with respective references in the revised manuscript.

8. Lines 17-19: In the introduction, it seems that the authors presented the “improved Hovorka model” to address the poor performance achieved by current artificial pancreas systems. Could the authors please elaborate more on how the model they presented will enhance the performance of existing control algorithms?

Response: Thank you for your query.

The improved Hovorka model has been added with additional parameters to its original Hovorka model; for example, the following equations are newly added to the original model as follows:

(dQ_1 (t))/dt=EGP_0+U_G+0.01Q_2+[x_1 k_w1+x_2 k_w2+x_3 k_w3 ]-F_R Q_1-[〖F_01〗^C/(V_G G(t) )] Q_1-0.002Q_1 (Eq.1)

(dQ_2 (t))/dt=[k_w11 x_1 (t)+k_w22 x_2 (t)+k_w33 x_3 (t)]+EGP_0 [k_w1 x_1 (t)+k_w2 x_2 (t)+k_w3 x_3 (t)]-k_12 Q_2 (Eq.2)

(dI(t))/dt=(U_I (t))/V_I -k_e I(t)- [k_w1 x_1 (t) + k_w2 x_2 (t) + k_w3 x_3 (t)] (Eq.8)

[dx_1/dt]=k_a1 x_1 (t)+k_w1 I(t)+k_w11 I(t) (Eq.10)

[dx_2/dt]=k_a2 x_2 (t)+k_w2 I(t)+k_w22 I(t) (Eq.11)

[dx_3/dt]=k_a3 x_3 (t)+k_w3 I(t)+k_w33 I(t) (Eq.12)

All these parameters and their values are included and described as shown in Tables 5 and 6 in the revised manuscript. It is also good to note that all these added equations are solely the original works of the authors based on their previous works in this artificial pancreas area (please refer to references [6,7,15-19] in the revised manuscript). The authors came up with parameter additions in the improved Hovorka equations model after carrying out a system identification technique on all parameters involved in the original Hovorka model. It was observed that by introducing the additional parameters in the improved Hovorka equations, there had been better interaction and interconnection between the accessible compartment and nonaccessible compartment in its glucose-insulin dynamics. Consequently, it gives a better control algorithm so as to yield optimum performance in regulating BGL for people with T1D.

9. The authors used “workers” in several places in the manuscript. Did they mean “works”?

Response: Workers means researchers who carry out the works.

10. Lines 43 and 44: Equation 1 seems to be missing the last term.

Response: Equation 1 has been corrected. Please refer to line 80 in the revised manuscript.

11. Lines 63 and 64: I think equation 8 contains a typo. Should the second “=” be removed?

Response: Equation 8 has been corrected. Please refer to line 100 in the revised manuscript.

12. Figures 2-4: The collected clinical data comprises six glucose samples per patient. However, data was represented with a continuous line. In my opinion, this representation leads to a misleading interpretation (eg, glucose follows a horizontal line in some periods, which is unrealistic). I suggest the authors mark the actual blood samples as in a scatter plot.

Response: Thank you for the constructive comments. Data analyses in determining percentage on target (normoglycemic range) for both clinical and in-silico works were carried out using Microsoft Excel, and it requires a continuous line; therefore, we decided to stick to our decision. We might shift to CGM applications once available in our clinic.

13. Figures 2-4: I think the x-axes should be in hours, not in minutes.

Response: Thank you for the comments. All figures mentioned have been changed to “hours” in the revised manuscript.

Round 2 Review

Anonymous

First, I would like to thank the authors for completing the clinical trial and statistical analysis description and for addressing all my comments on the previous submission. Unfortunately, I am afraid that I still have some methodological doubts regarding the comparison between the in-silico results and the clinical results. Therefore, I suggest a further revision of the manuscript.

Response: Thank you for the compliments and constructive comments.

Specific Comments

My principal methodological concerns are listed in the Major Comments section. Other doubts or typos are presented in the Minor Comments section.

Major Comments

1. If I understood well, the insulin infusion rate used to simulate the “Improved Hovorka model” (variable u(t) in equation 6) is different from the infusion rate administered to the actual patients in the clinical trial. On the one hand, virtual patients have received an insulin infusion rate calculated from a closed-loop algorithm (the eMPC). On the other hand, actual patients seem to follow an open-loop therapy (multiple drug injections). Lastly, the authors also state (lines 361 and 362) that the timing of insulin bolus is different in both settings. Could the authors explain these discrepancies, please? If the goal is to compare the prediction ability of the “Improved Hovorka model,” why have the authors not simulated the model with the same insulin therapy used for the clinical trial?

Response: Thank you for your comment. We fully agree and appreciate your deep concern on the design of the methodology adopted in this study. Yes, there are so many limitations and constraints faced throughout the course of this study; among others, CGM was not used, limited data was available for patients, etc, which, in turn, forced us to use two different protocols; namely, the closed-loop algorithm (eMPC) was used for the in-silico test, whereas an open-loop therapy was used for the clinical validation. These issues have been clarified in different sections of the manuscript as follows:

Abstract (has been inserted, accordingly): “Methods: Three actual patients’ data were collected from Clinic 1, Clinical Training Centre, Universiti Teknologi MARA (UiTM) Hospital, Sungai Buloh, Selangor upon getting approval from UiTM Ethics Committee. The inclusion criteria of subjects were namely; T1D patients, age range between 11 to 14 years old, highly dependent on insulin injection with four or more finger pricks or self-monitoring of blood glucose (SMBG) for BGL measurements per day. The T1D patients typically receive meals three times per day; breakfast, lunch, and dinner. In a nutshell, closed-loop algorithm (eMPC) was used for the in-silico test whereas an open-loop therapy was used for the clinical validation. As for data analysis of patients, P-value via multiple linear regression (MLR) was used to model the relationship between meal, insulin, and BGL.

“Results: In order of breakfast, lunch, and dinner: the optimum bolus insulins for patient 1 were 83.33, 33.33 and 16.67 mU/min; patient 2 were 66.67, 50.01 and 33.33 mU/min, and patient 3 were 100.02, 83.33, and 66.67 mU/min, respectively. As for the in-silico works; results revealed that the percentages of time for their BGL on target in patients 1, 2, and 3 were at 79.59%, 87.76%, and 71.43%, respectively, as compared to the clinical works with less than 50%. A small P-value (P<0.01) indicated that the variables were significant. However, when comparison was made on the BGL profile; both profiles were not comparable due to different methodology adopted in the design of the study.

“Conclusions: In conclusion, the in-silico work using the improved Hovorka model equations was not comparable to the clinical works to simulate BGL with meal disturbances for people with T1D.”

Discussion section (lines 396 and 397): “However, when comparison was made on the BGL profile; both profiles were not comparable due different methodology adopted in the design of the study.”

Conclusion section (lines 414-416): “In conclusion, the in-silico work using the improved Hovorka model equations was not comparable to the clinical works to simulate BGL with meal disturbances for people with T1D due to different methodology adopted for both works.”

2. From the regression analysis results (lines 406-409), the authors seem to conclude that the model is “applicable in predicting BGL” because the P value is <.01. However, I cannot see the relation between a significant P value and a better prediction. From the clear description the authors provided of the multiple regression analysis, I think the authors fitted the following linear model:

improved_hovorka_glucose = insulin·beta1 + meal·beta2 + e

where “improved_hovorka_glucose” is the output of the “Improved Hovorka model,” “insulin” and “meal” correspond to the values of the infusion rate and meal amount in that model, “beta1” and “beta2” are coefficients to be estimated in the analysis, and “e” is the normal distributed residuals. A P value <.05 means that data supports the rejection of the null hypothesis that beta1 = beta2 = 0 [9]. Thus, a significant P value indicates that the “insulin” and/or the “meal” inputs can explain the variations observed in “improved_hovorka_glucose.” However, I cannot see how one can conclude anything from the prediction accuracy of the “Improved Howorka model” from the fact that beta1≠0 or beta2≠0. Could the author explain this point, please?

Response: Thank you for your comment. I am sorry that I may have to reserve my answer for this specific comment. Due to weaknesses in the methodology being adopted for comparison purposes for both works, these results might have happened. However, I have corrected my statements in different sections on the revised manuscript in order to clarify those issues as follows:

Lines 394-397: “Based on Table 14, all patients in both clinical and in-silico works have a small P-value (P<0.01), which indicates the variables are significant. However, when comparison was made on the BGL profile; both profiles were not comparable due different methodology adopted in the design of the study.”

Lines 414-416: “In conclusion, the in-silico work using the improved Hovorka model equations was not comparable to the clinical works to simulate BGL with meal disturbances for people with T1D due to different methodology adopted for both works.”

Minor Comments

3. Equation 5: The term exp(t/maxG) should be exp(-t/maxG).

Response: Thank you for your comment. Equation 5 has been corrected as suggested. Please refer to lines 89 and 90.

4. Equation 6: In line 103, the authors define u(t) as insulin bolus. However, in line 180, the authors refer to infusion rates. Could the author check the consistency of this definition?

Response: Thank you for your comment. The u(t) definition has been used consistently as “Insulin infusion rate” throughout the manuscript. Therefore, the said term has been changed which can be found in line 98 as follows: “u(t) (mU/min) is insulin infusion rate.”

Round 3 Review

Anonymous

I would like to thank the authors for their efforts in replying to my comments. Unfortunately, I still do not understand the article’s contribution regarding comparing clinical data. As stated by the authors, clinical data and in-silico results are not comparable due to the different methodologies and protocols applied to obtain the data; therefore, I wonder if including the clinical data set analysis is justified. In addition, I have doubts about under which conditions the authors have simulated the model, for instance, if the simulation included any kind of variability.

Response: Thank you for the compliments and constructive comments.

We thought that we had answered these queries in the first and second detailed response reports in greater lengths (please refer to those reports), and we reserve not to deliberate them again at this point.

Specific Comments

Major Comments

1. Since clinical data results and in-silico data are incomparable, could the authors justify the motivation for including the clinical data in the article?

Response: Thank you for the comments. Yes, the clinical data are equally important in this study since the actual patients’ demographic profiles are based on their age group, body weight, daily meal intakes, etc. All these information have been taken into account in the calculations for the in-silico results, accordingly.

2. In lines 191-193, the authors indicate that comparing the in-silico data with the clinical data would help determine the model’s accuracy in mimicking the actual glucose. I believe this statement is incompatible with the fact that both clinical and in-silico data were obtained following incomparable protocols and methodologies.

Response: The authors are still in the opinion that this work is considered as a preliminary study attempting to apply the improved Hovorka equations (the authors’ own previous work) using actual patients’ data in the calculations of the in-silico works for glucose mimicking purposes. Even though both studies (in-silico vs clinical) have used different methodologies and protocols due to study/data limitation, the same actual patients’ datasets are used throughout the simulation study, and we believe it should be acceptable for mathematical modeling purposes (in-silico). However, further research needs to be carried out when CGM is available in our clinics.

3. Have the authors thought about modifying the simulation of the model to make the results more comparable with the clinical data? For instance, I would suggest they simulate the model with the same bolus, basal insulin, and meal carbohydrates utilized in the clinical trial. Then, they could compare the model output with each glucose measurement.

Response: Thank you for the compliments and constructive comments.

Yes, we have thought about it, and this should be the way forward for our further research work.

4. In the Discussion section (lines 374-378) and the conclusion (line 404), the authors concluded from Table 13 that the patients have less sensitivity in the morning. Since Table 13 corresponds to the results of the virtual patients in the in-silico analysis, I wonder whether the authors have included any kind of circadian variability in the simulation, for instance, some sinusoidal variability in ka1, kw1, kw11, ka2, kw2, kw22, ka3, kw3, or kw33. If this is not the case and these parameters were kept constant in the simulation, I suggest authors better justify this apparent increase in insulin sensitivity.

Response: Thank you for the compliments and constructive comments.

Since this study is very preliminary in nature, we are unable to conduct our simulation as suggested. However, we will take that into consideration seriously into our research works in the future.

5. The authors said the insulin bolus was computed by trial and error. Since one of the article’s goals is determining the optimal bolus, it would be advisable to detail the method followed to calculate it.

Response: We thought that we had answered these queries in the first and second detailed response reports in greater lengths (please refer to those reports), and we reserve not to deliberate them again at this point.

Round 4 Review

Anonymous

I would like to thank the authors for replying to my comments. Unfortunately, I still believe the work has two principal limitations preventing me from accepting the manuscript. The main one is that the differences in protocols and conditions between the clinical and simulation works make it, in my opinion, unfeasible to address the goal of determining “the accuracy and effectiveness of the in-silico model in mimicking real-world BGL dynamics.” The second one is that insufficient information is reported to reproduce the calculation of the optimal bolus in the in-silico simulations.

Response: Thank you for the comments. All corrections stated as limitations of the study are included in the revised manuscript.

Limitations of the Study (lines 398-414): “Apparently, there are two main limitations discovered from the study which, in turn, make it unfeasible to address the goal of determining the accuracy and effectiveness of the in-silico model in mimicking real-world BGL dynamics. Firstly, it was due to different protocols and conditions adopted in the methodology for the clinical and simulation works. As stated earlier, the open loop therapy was used in the clinical work for evaluation purposes; whereas the closed loop algorithm with MPC was used in the in-silico test. In order to address this limitation for future works, it is primarily essential to modify the simulation of the model to make the results more comparable with the clinical data. For instance, it is suggested that in the simulation work, the improved Hovorka equations model could be simulated using the same bolus, basal insulin, and meal CHO utilized in the clinical trial. By doing so, they could compare the model output with each glucose measurement, preferably when CGM is available at our clinic for future use. Secondly, insufficient information was reported to reproduce the calculation of the optimal bolus in the in-silico simulations. This happened due to the insulin bolus was computed by trial and error as programmed earlier in its closed loop algorithm. Since one of the main goals of the study is to determine the optimal bolus insulin, it would be advisable to detail the method followed to calculate it in the development of its new control algorithm for future works.”