Keywords: Black Africans, cardiovascular events, hemodialysis, kidney disease, proprotein convertase subtilisin, kexin type 9
Published online 2021 July 20. doi: 10.5041/RMMJ.10443.
Circulating Proprotein Convertase Subtilisin/Kexin Type 9 Levels Predict Future Cardiovascular Event Risks in Hemodialyzed Black African Patients
1Department of Internal Medicine, Division of Nephrology–Dialysis, University of Kinshasa Hospital, Kinshasa, Democratic Republic of the Congo
2Faculty of Family Medicine and Primary Care, Protestant University of Congo, Kinshasa, Democratic Republic of the Congo
3Department of Pediatrics, Division of Nephrology–Dialysis, University of Kinshasa Hospital, Kinshasa, Democratic Republic of the Congo
*To whom correspondence should be addressed. E-mail: email@example.com
Copyright : © 2021 Kajingulu et al.
This is an open-access article. All its content, except where otherwise noted, is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Context and ObjectiveCardiovascular diseases are the leading cause of mortality in patients. In this context, proprotein convertase subtilisin/kexin type 9 (PCSK9) appears to be the new biomarker identified as interfering in lipid homeostasis. This study aimed to investigate the association between PCSK9, dyslipidemia, and future risk of cardiovascular events in a population of black Africans.
MethodsA cross-sectional study was conducted between August 2016 and July 2020 in six hemodialysis centers in the city of Kinshasa, Democratic Republic of the Congo. Serum PCSK9 was measured by ELISA; lipid levels of 251 chronic kidney disease grade 5 (CKD G5) hemodialysis patients and the Framingham predictive instrument were used for predicting cardiac events.
ResultsTotal cholesterol (TC), low-density lipoprotein cholesterol (LDL-c), and triglycerides (TG) were significantly increased in the tertile with the highest PCSK9. By contrast, high-density lipoprotein cholesterol (HDL-c) was significantly decreased in the same tertile. A strong positive and significant correlation was found between PCSK9 and TC, TG, and LDL-c. Negative and significant correlation was observed between PCSK9 and HDL-c. The levels of PCSK9, smoking, overweight, and atherogenic dyslipidemia were associated with future risks for cardiovascular events in univariate analysis. After adjustment, all these variables persisted as independent determinants of future risk for cardiovascular events. The probability of having a cardiovascular event in this population was independently associated with PCSK9 levels. Compared to the patients in the lowest PCSK9 tertile, patients with PCSK9 levels in the middle (aOR 5.9, 95% CI 2.06–17.3, P<0.001) and highest tertiles (aOR 8.9, 95% CI 3.02–25.08, P<0.001) presented a greater risk of cardiac event.
ConclusionIncreased PCSK9 serum levels are associated with higher levels of TC, LDL-c, and TG and lower levels of HDL-c in black African hemodialysis patients. Serum PCSK9 levels in these patients predict increased risk of cardiovascular events, independent of traditional potential confounders.
Keywords: Black Africans, cardiovascular events, hemodialysis, kidney disease, proprotein convertase subtilisin, kexin type 9
In chronic kidney disease (CKD), patients die more likely from cardiovascular complications than from progression to kidney failure with replacement therapy. Indeed, clinical evidence shows that cardiovascular disease begins as soon as kidney function declines, and its severity increases with the progression of CKD.1 However, the cardiovascular risk remains high even in patients undergoing hemodialysis or peritoneal dialysis.2,3
Among the modifiable cardiovascular risk factors identified, dyslipidemia remains one of the major risk factors in chronic kidney disease grade 5 (CKD G5) hemodialysis patients or CKD without kidney failure with replacement therapy.4 Indeed, it has been reported in the literature that the high cardiovascular risk associated with CKD and kidney replacement therapy could be partially explained by the acceleration of the dynamic process of atherosclerosis induced by dyslipidemia.5–7 In this regard, serum protein proprotein convertase subtilisin/kexin type 9 (PCSK9) has been identified as the main factor inducing the increased synthesis of very-low-density lipoprotein cholesterol (triglyceride-rich lipoprotein), a factor underlying the development of atherogenic dyslipidemia and the subsequent elevated cardiovascular risk.8 The plasma protein PCSK9 acts, in large part, by targeting and reducing the hepatic degradation of very-low-density lipoprotein, a lipoprotein containing apolipoprotein B (apo-B) and low-density lipoprotein cholesterol (LDL-c).9,10
Dyslipidemia (particularly atherogenic dyslipidemia) does not respond well to statins, which partly explains the residual cardiovascular risk observed in patients on statins.8,9 Several studies report that inhibitors of the plasma protein PCSK9 substantially reduce the residual cardiovascular risk when combined with statins.8,10–12 However, almost all of the studies on the relationship between PCSK9 and cardiovascular disease are conducted in developed countries. Considering the geographic and racial disparity in the prevalence and relative burden of cardiovascular risk factors, it is appropriate for low- and middle-income countries to determine this relationship from a perspective of prevention, screening, and early management of cardiovascular diseases in black African patients. Little is known about PCSK9 levels in CKD,13–15 and in the single study on this topic, PCSK9 plasma concentration does not predict cardiovascular events in CKD stage 2–4 patients.16 Therefore, the present study aimed to investigate the association between PCSK9, dyslipidemia, and future risks of cardiovascular events in a population of black Africans, particularly among patients on chronic kidney disease hemodialysis.
Study Population and DesignIn a cross-sectional study, consecutive CKD G5 hemodialysis patients aged >16 years were examined between August 2016 and July 2020 in the following six hemodialysis centers in the city of Kinshasa, the Democratic Republic of Congo: University Hospital of Kinshasa; General Provincial Referral Hospital of Kinshasa; General Referral Hospital of the Congolese National Police; Ngaliema Medical Center Clinic; HJ Hospitals; and AFIA Medical Center.
Data Collection and ProcedureVariables of interests included: age, sex, CKD etiology, diabetes mellitus (DM), high blood pressure (HBP), smoking, alcoholism, physical inactivity, duration on hemodialysis, and current pharmacological treatment. The physical examination was performed before the hemodialysis session and focused on the following parameters: weight (kg), height (cm), systolic blood pressure (SBP) (mmHg), diastolic blood pressure (DBP) (mmHg), mean arterial pressure (MAP) (mmHg), waist circumference (cm), and pulse and heart rate (beat/min). Laboratory parameters of interest in this study encompassed: serum hemoglobin, hematocrit, urea, creatinine, glycemia, uric acid, total cholesterol (TC), low-density lipoprotein cholesterol (LDL-c), high-density lipoprotein cholesterol (HDL-c), triglycerides (TG), non-HDL-c, calcium, phosphorus, parathyroid hormone (PTHi), vitamin D, and PCSK9. The non-biological parameters included: ankle–brachial pressure index (ABI) (obtained on the basis of the ratio of the systolic pressure of the ankle and that of the brachial) and the body composition of the study population (including: sex, age, height, weight, body fat, muscle mass, and body mass index [BMI]).
The PCSK9 concentration was measured by competitive immunoenzymatic inhibition in accordance with the sandwich enzyme-linked immunosorbent assay (ELISA) human proprotein convertase subtilisin/kexin type 9, MBS920252 Kit (MyBioSource, San Diego, CA, USA). Protein concentrations in the range 0.45 ng/mL to 30 ng/mL were detected. In this study, PCSK9 levels were divided into tertiles: tertile 1: PCSK9 <9.56 ng/L; tertile 2: PCSK9 9.56–23.1 ng/L; and tertile 3: PCSK9 >23.1 ng/L. The lipid fractions were assayed according to the enzymatic colorimetric method on the Cobas C 311, Roche-Paris/France revised version 2010 automated system. Isolated dyslipidemia was defined by a TC ≥200 mg/dL; HDL-c <50 mg/dL in women and <40 mg/dL in men; LDL-c ≥100 mg/dL; or TG ≥150 mg/dL.17 Combined dyslipidemia was defined according to the international Frederickson classification: type I or IV dyslipidemia corresponded to a level of LDL-c <100 mg/dL and TG ≥150 mg/dL; type IIa: LDL-c ≥100 mg/dL and TG <150 mg/dL; type IIb: LDL-c ≥100 mg/dL and TG ≥150 mg/dL.18 We estimated the risk of cardiovascular events using the Framingham equation adopted by the World Health Organization (WHO) and the Agence nationale d’accréditation et d’évaluation en santé (ANAES) in 2004, which made it possible to predict the 10-year risk of a major cardiovascular event (myocardial infarction, cerebrovascular accident, angina, or sudden death). This model takes into account the following parameters: sex, age, systolic blood pressure, TC, HDL-c, smoking, antihypertensive treatment, diabetes mellitus, and family history of hypertension. Framingham scores of <20%, 20%–30%, and >30% were considered as low risk, high risk, and very high risk, respectively.
Written informed consent was obtained from all participants before enrollment. This study protocol was submitted to the ethics committee of the School of Public Health of Kinshasa/DRC for analysis and received approval registered at number: ESP/CE/053/2016.
Statistical AnalysisAll data were analyzed using SPSS for Windows version 24 software. The descriptive statistics were: the mean and standard deviation for quantitative variables with Gaussian distribution; the median and interquartile range (IQR) for variables not normally distributed; and the relative (%) and absolute (n) proportions for categorical data. Student’s t test was performed to compare two means. The ANOVA test was used for multiple comparisons. ANOVA tests which were significant at the P<0.05 level were supplemented by a post hoc Scheffé test. For the comparison of the medians of two groups, we used the Mann–Whitney U test, and for more than two medians the Kruskal–Wallis H test. The linear regression test was applied to check the correlation between the different lipid fractions and PCSK9. Pearson’s correlation coefficients (r) were calculated to assess this association. The search for the determinants of high future risks of cardiovascular event was carried out using the multivariate binary logistic regression model. The adjusted ORs and their 95% CIs were calculated to determine the degree of association between elevated future cardiovascular risks and the independent variables. The significance level retained was P value <0.05.
In total, 251 adult CKD G5 hemodialysis patients were enrolled: 182 (72.5%) men and 69 (27.5%) women. Their average age was 55.8±13.6 years (Table 1). The main cardiovascular risk factors encountered in this population were hypertension (87.3%), diabetes mellitus (28.3%), alcohol consumption (47.0%), smoking (20.3%), and gout (7.2%). The distribution of lipid particles studied was: low HDL-c 74.5%; high LDL-c 33.1%; high TG 29.1%; and high TC 15.1%. Atherogenic dyslipidemia, abnormal ABI, and high cardiovascular risk were observed in 77.3%, 24.3%, and 25.9%, respectively. Considering the clinical and biological characteristics (Table 2), TC, LDL-c, and TG increase significantly from tertile 1 to tertile 3, while HDL-c decreases significantly across the same tertiles. A strong positive and significant correlation was found between PCSK9 and TC, TG, and LDL-c (Figure 1). On the other hand, a negative and significant correlation was shown between PCSK9 and HDL-c.
Table 3 shows that the high cardiovascular risk was observed in patients having higher frequency of alcohol consumption, smoking, overweight, and abnormal ABI. This risk was proportional to the increase in plasma TC, LDL-c, and TG levels. The risk was significantly elevated for low HDL-c levels. Patients of the study with PCSK9 level in tertile 3 also had an elevated cardiovascular risk.
In a univariate analysis, smoking, overweight, atherogenic dyslipidemia, and PCSK9 levels were associated with future risks of cardiovascular events (Table 4). Following the adjustment, all these variables persisted as independent determinants of future risks for cardiovascular events.
Indeed, the probability of having a cardiovascular event in this population is strongly associated with the PCSK9 level. Hence, compared to patients in tertile 1, patients with PCSK9 levels in tertile 2 and tertile 3 had a 6- and 9-fold higher OR, respectively.
This study identified, for the first time, to the best of our knowledge, a positive and significant correlation between PCSK9 and TC, LDL-c, and TG in sub-Saharan African hemodialysis patients. In addition, plasma PCSK9 levels are independently associated with future risks of cardiovascular events. The strong positive and significant correlation found in this study between PCSK9 and TC, TG, and LDL-c is similar to that reported elsewhere.13–15,19–21 In some studies of non-diabetic obese, this correlation has not been shown.22,23 This result suggests that this correlation requires certain metabolic conditions. The emerging role of PCSK9 is observed in different conditions of cardiovascular homeostasis focused on dyslipidemia, glomerular proteinuria, insulin secretion, regulation of blood pressure, and inflammation.24 Patients included in our study had a high frequency of hypertension, DM, tobacco consumption, and alcohol intake. This correlation found between PCSK9 and the lipid fractions would explain the cardiovascular risk encountered in hemodialysis patients.25 Indeed, given that PCSK9’s main site of action is the membrane LDL-c receptor where a growth factor acts as a ligand in the presence of the catalytic domain of PCSK9, this synergy catabolizes the LDL-c receptors. Therefore, the plasma level of LDL-c increases by decreasing the LDL-c receptors. The LDL-c particles containing apo-B exhibit a high concentration of triglycerides which are very sensitive to oxidation. These particles are the basis of the atherogenic process,19 the final consequence of which is cardiovascular disease. In addition, PCSK9, associated with the metabolism of triglycerides, has the ability to hydrolyze lipoprotein lipase, an enzyme that breaks down triglycerides into fatty acids. Hydrolysis of lipoprotein lipase is the basis for the increase in plasma triglycerides.19 Indeed, in patients without cardiac disease, it has been noted that any increase in the level of PCSK9 implies an increase in the cardiovascular risk.26 In our study (Table 4), the probability of having a cardiovascular event in this population is associated with the PCSK9 level. Compared to tertile 1, patients with PCSK9 levels in tertiles 2 and 3 had 6- and 9-fold higher OR, respectively. A regulator of LDL-c receptors, PCSK9 is involved in cholesterol homeostasis. The increase of its level is influenced by the activation of platelets,27 vascular stiffness,28 and coronary plaque instability.29 Therefore, the level of PCSK9 is not a predicting factor of cardiovascular disease in CKD patients at early stages, such as stages 2–3 of CKD,16,30 but in patients with stages 4–5 of CKD. In our study, the patients included are black African CKD patients undergoing CKD G5 hemodialysis in whom traditional and non-traditional black African CKD patients undergoing CKD G5 hemodialysis in whom traditional and non-traditional cardiovascular risk factors coexist. The reason that the PCSK9 level is associated with dyslipidemia in our study is not elucidated and could be explained by genetic or environmental factors. However, these possibilities and other explanations require continued investigation. Indeed, genetic variants significantly associated with dyslipidemia have already been identified: in a Canadian study, loss of gene function was found in the Caucasian population but not in the population of African origin.31,32 Considering these observations, our study suggests that the relationship between PCSK9 and dyslipidemia in black Africans undergoing hemodialysis may be due to a genetic predisposition that could be gains-of-function of PCSK9 in these patients. This observation is consistent with the study conducted in Kenya in a black African population that showed that the plasma PCSK9 level was significantly associated with plasma LDL-c level, and a strong association with plasma LDL-c level, and a strong association was observed between PCSK9 and triglycerides.33
Several limitations exist in this study, including its cross-sectional design, the relatively small sample size, and the method used to predict risk of future cardiovascular events. Thus, these findings require confirmation in a larger cohort of prospective African samples.
This study suggests that a higher plasma level of PCSK9 is associated with a higher Framingham score. Therefore, PCSK9 could be used as a biomarker for the prediction of cardiovascular events in black African hemodialysis patients. This preliminary study suggests the importance of future studies on the effect of PCSK9 in cardiovascular events as well the worse survival encountered in African hemodialysis patients.
The authors gratefully thank the Research Laboratory of the School of Public Health, University of Kinshasa as well as the Laboratory of HJ Hospitals, Democratic Republic of Congo.
Author Contributions: FPMK participated in protocol elaboration, data collection and analysis and wrote the manuscript; EKS, FBL, ANN, JRRM, and PME participated in the study design; VMM, JBB, ALL, YMN, and NMN reviewed the manuscript.
Hermans MP, Ahn SA, Rousseau MF. The atherogenic dyslipidemia ratio [log (TG)/HDL-c] is associated with residual vascular risk, beta-cell function loss and microangiopathy in type 2 diabetes females. Lipids Health Dis. 2012;11:132. https://doi.org/10.1186/1476-511X-11-132.
Chandrashekar A, Ramakrishnan S, Rangarajan D. Survival analysis of patients on maintenance hemodialysis. Indian J Nephrol. 2014;24:206–13. https://doi.org/10.4103/0971-4065.132985.
Neves M, Machado S, Rodrigues L, et al. Cardiovascular risk in peritoneal dialysis - a Portuguese multicenter study. Nefrologia. 2014;34:205–11. https://doi.org/10.3265/nefrologia.pre2013.dec.12131.
Wolf G, Ziyadeh FN. Molecular mechanisms of diabetic renal hypertrophy. Kidney International. 1999;56:393–405. https://doi.org/10.1046/j.1523-1755.1999.00590.x.
Shirai K. Obesity as the core of the metabolic syndrome and the management of coronary heart disease. Curr Med Res Opin. 2004;20:295–304. https://doi.org/10.1185/030079903125003008.
Suzuki M, Takamisawa I, Suzuki K, et al. Close association of endothelial dysfunction with insulin resistance and carotid wall thickening in hypertension. Am J Hypertens. 2004;17:228–32. https://doi.org/10.1016/j.amjhyper.2003.11.013.
Engole, YM.; Lepira, FB.; Nlandu, YM., et al. Prognostic significance of abnormal ankle–brachial index among long-term hemodialysis patients in Kinshasa, the Democratic Republic of the Congo. Rambam Maimonides Med J. 2021. p. e0001. https://doi.org/10.5041/rmmj.10427.
Rashid S, Kastelein JJP. PCSK9 and resistin at the crossroads of the atherogenic dyslipidemia. Expert Rev Cardiovasc Ther. 2013;11:1567–77. https://doi.org/10.1586/14779072.2013.839204.
Guardiola, M.; Plana, N.; Ibarretxe, D., et al. Circulating PCSK9 levels are positively correlated with NMR-assessed atherogenic dyslipidaemia in patients with high cardiovascular risk. Clin Sci (Lond). 2015. pp. 877–82. https://doi.org/10.1042/cs20140832.
Tavori H, Rashid S, Fazio S. On the function and homeostasis of PCSK9: reciprocal interaction with LDLR and additional lipid effects. Atherosclerosis. 2015;238:264–70. https://doi.org/10.1016/j.atherosclerosis.2014.12.017.
Berman AN, Blankstein R. Optimizing dyslipidemia management for the prevention of cardiovascular disease: a focus on risk assessment and therapeutic options. Curr Cardiol Rep. 2019;21:110. https://doi.org/10.1007/s11886-019-1175-z.
Brandts J, Müller-Wieland D. PCSK9 inhibition: new treatment options and perspectives to lower atherosclerotic lipoprotein particles and cardiovascular risk. Curr Atheroscler Rep. 2019;21:40. https://doi.org/10.1007/s11883-019-0802-x.
Konarzewski M, Szolkiewicz M, Sucajtys-Szulc E, et al. Elevated circulating PCSK-9 concentration in renal failure patients is corrected by renal replacement therapy. Am J Nephrol. 2014;40:157–63. https://doi.org/10.1159/000365935.
Rasmussen, LD.; Bøttcher, M.; Ivarsen, P., et al. Association between circulating proprotein convertase subtilisin/kexin type 9 levels and prognosis in patients with severe chronic kidney disease. Nephrol Dial Transplant. 2020. pp. 632–9. https://doi.org/10.1093/ndt/gfy257.
Bermudez-Lopez M, Forne C, Amigo N, et al. An in-depth analysis shows a hidden atherogenic lipoprotein profile in non-diabetic chronic kidney disease patients. Expert Opin Ther Targets. 2019;23:619–30. https://doi.org/10.1080/14728222.2019.1620206.
Rogacev, KS.; Heine, GH.; Silbernagel, G., et al. PCSK9 plasma concentrations are independent of GFR and do not predict cardiovascular events in patients with decreased GFR. PLoS One. 2016. p. e146920. https://doi.org/10.1371/journal.pone.0146920.
Brunezl JD, Davidson M, Furberg CD, et al. Lipoprotein management in patients with cardiometabolic risk: consensus statement from the American Diabetes Association and the American College of Cardiology Foundation. Diabetes Care. 2008;31:811–22. https://doi.org/10.2337/dc08-9018.
Quispe R, Hendrani AD, Baradaran-Noveiry B, et al. Characterization of lipoprotein profiles in patients with hypertriglyceridemic Fredrickson-Levy and Lees dyslipidemia phenotypes: the Very Large Database of Lipids Studies 6–7. Arch Med Sci. 2019;15:1195–202. https://doi.org/10.5114/aoms.2019.87207.
Zhu YM, Anderson TJ, Sikdar K, et al. Association of proprotein convertase subtilisin/kexin type 9 (PCSK9) with cardiovascular risk in primary prevention. Arterioscler Thromb Vasc Biol. 2015;35:2254–9. https://doi.org/10.1161/atvbaha.115.306172.
Baass A, Dubuc G, Tremblay M, et al. Plasma PCSK9 is associated with age, sex, and multiple metabolic markers in a population-based sample of children and adolescents. Clin Chem. 2009;55:1637–45. https://doi.org/10.1373/clinchem.2009.126987.
Cui Q, Ju X, Yang T, et al. Serum PCSK9 is associated with multiple metabolic factors in a large Han Chinese population. Atherosclerosis. 2010;213:632–6. https://doi.org/10.1016/j.atherosclerosis.2010.09.027.
Mayne J, Raymond A, Chaplin A, et al. Plasma PCSK9 levels correlate with cholesterol in men but not in women. Biochem Biophys Res. 2007;361:451–6. https://doi.org/10.1016/j.bbrc.2007.07.029.
Alborn WE, Cao G, Careskey HE, et al. Serum proprotein convertase subtilisin kexin type 9 is correlated directly with serum LDL cholesterol. Clin Chem. 2007;53:1814–19. https://doi.org/10.1373/clinchem.2007.091280.
Hachem A, Hariri E, Saoud P, Lteif C, Lteif L, Welty F. The role of proprotein convertase subtilisin/kexin type 9 (PCSK9) in cardiovascular homeostasis: a non-systematic literature review. Curr Cardiol Rev. 2017;13:274–82. https://doi.org/10.2174/1573403x13666170804150954.
Hwang, HS.; Kim, JS.; Kim, YG., et al. Circulating PCSK9 level and risk of cardiovascular events and death in hemodialysis patients. J Clin Med. 2020. p. 244. https://doi.org/10.3390/jcm9010244.
Leander K, Mälarstig A, Van’t Hooft FM, et al. Circulating proprotein convertase subtilisin/kexin type 9 (PCSK9) predicts future risk of cardiovascular events independently of established risk factors. Circulation. 2016;133:1230–9. https://doi.org/10.1161/circulationaha.115.018531.
Navarese EP, Kolodziejczak M, Winter MP, et al. Association of PCSK9 with platelet reactivity in patients with acute coronary syndrome treated with prasugrel or ticagrelor: the PCSK9-REACT study. Int J Cardiol. 2017;227:644–9. https://doi.org/10.1016/j.ijcard.2016.10.084.
Ruscica, M.; Ferri, N.; Fogacci, F., et al. Circulating levels of proprotein convertase subtilisin/kexin type 9 and arterial stiffness in a large population sample: data from the Brisighella heart study. J Am Heart Assoc. 2017. p. e005764. https://doi.org/10.1161/jaha.117.005764.
Cheng JM, Oemrawsingh RM, Garcia-Garcia HM, et al. PCSK9 in relation to coronary plaque inflammation: results of the ATHEROREMO-IVUS study. Atherosclerosis. 2016;248:117–22. https://doi.org/10.1016/j.atherosclerosis.2016.03.010.
Balla S, Nusair MB, Alpert MA. Risk factors for atherosclerosis in patients with chronic kidney disease: recognition and management. Curr Opin Pharmacol. 2013;13:192–9. https://doi.org/10.1016/j.coph.2012.12.001.
Babiak J, Rudel LL. Lipoproteins and atherosclerosis. Baillieres Clin Endocrinol Metab. 1987;1:515–50. https://doi.org/10.1016/s0950-351x(87)80022-8.
Austin MA, Hutter CM, Zimmern RL, Humphries SE. Genetic causes of monogenic heterozygous familial hypercholesterolemia: a HuGE prevalence review. Am J Epidemiol. 2004;160:407–20. https://doi.org/10.1093/aje/kwh236.
Paquette M, Luna Saavedra YG, Chamberland A, et al. Association between plasma proprotein convertase subtilisin/kexin type 9 and the presence of metabolic syndrome in a predominantly rural-based sub-Saharan African population. Metab Syndr Relat Disord. 2017;15:423–9. https://doi.org/10.1089/met.2017.0027.