Introduction

The practice of Medicine at present demands quality care. Multiple indicators are utilized to measure the latter; however, the clinical results are, in the end, the most relevant.¹ For some investigators, mortality is the first and most important indication of quality in the clinical practice.¹ Crude or non-adjusted mortality has its limitations in that it does not consider, among other aspects, the clinical conditions of the sick person nor his/her comorbilities.^1,2 In heart surgery, statistical programs have been developed from a group of risk factors; these factors are capable of predicting mortality.² For comparing the mortality rate among diverse institutions or for temporary comparison at a same institution, it is recommended to employ some of the mathematical models developed that evaluate mortality adjusted to the clinical conditions and risk profile of the population cared for.^1,2 During the last decade, the Parsonnet score, that of the U.S. Society of Thoracic Surgeons (STS), and the EuroSCORE³ score have been those most frequently utilized. The latter was obtained from the study of 19,030 adult patients submitted to heart surgery (ischemic, valvular, and congenital) with extracorporeal circulation pump in 128 hospitals in eight European countries (Germany, France, the U.K., Italy, Spain, Finland, Sweden, and Switzerland). In this study, based on its objectivity, reliability, and prevalence, the authors analyzed 97 variables or risk factors (68 preoperatory and 29 operatory) were analyzed, among which 18 remained with prognostic value.^2-4 The EuroSCORE model has two variants: additive and logistic. The former awards a numerical value to each variable that the patient presents, and the sum of these values provides the probability of death.⁴ It entertains the advantage of being a simple and uncomplicated instrument that can be applied at the patient’s bedside. The logistic version is more sophisticated. The formula for obtaining the latter is the following: estimated mortality = e (β0+∑βiXi)/ 1+e(β0+∑βiXi), in which β0 is the constant of the logistic regression model (-4.789594) and βi is the coefficient of variable Xi of the logistic-regression information.⁵ The definition, the odds ratio (OR), the value of each variable of the additive model, and coefficient β of each variable of the logistic model appear in Table 1. To utilize a risk score or a predictive model developed to predict mortality in patients submitted to heart surgery, which has shown good performance in the group of patients where it was generated, thus proving its internal validity, it is recommended that it be evaluated at the hospitals where its use is intended to confirm or discard external validity, therefore using it or not.^1,2,4 The objective of the present study was to know the risk profile of the adult population submitted to cardiac surgery and to validate its additive as well as in its logistic version in the population of adult patients submitted to cardiac surgery with and without extracorporeal circulation pump at the Hospital Regional de Alta Especialidad del Bajío (HRAEB) in the city of León, Guanajuato, Mexico.

Material and methods

We conducted an observational, transversal, and retrospective study by means of the review of the clinical files of patients aged of ≥ 16 years submitted to cardiac surgery with and without extracorporeal circulation pump that were effected from January 1, 2008 to December 31, 2013 at the HRAEB. We did not include incomplete files, or those with doubtful information on the variables-of-interest, nor the files of patients who died due to causes not related to the cardiac surgery index. The following variables were obtained: mortality not adjusted up to the patient’s hospital discharge, defined as death occuring during the hospitalization index; type of surgery, defined as the procedure or procedures carried out during the index surgery, whether a) valvular surgery, b) aortic-coronary bypass surgery, c) corrective surgery for some congenital malformation(s), and d) surgery of another different type, including valvular + coronary bypass surgery, aortic surgery, closing of post-infarct interventricular communication, traumatic cardiac lesions, pericardium resection). The required variables were collected and the mortality-risk score was calculated of the EuroSCORE employing the EuroSCORE-program on-line calculator in its additive as well as its logistic version (www.euroscore.org/calsp.html ).

Results

From the list of cardiac-surgery and cardiovascular-surgery procedures that were carried out from January 1, 2008 to December 31, 2013, which were collected by the HRAEB Department of Statistics, we identified 350 surgical procedures. Among the latter, we eliminated eight as follows: two for not having complete data-of-interest in the file; one for having been triplicated and with different information, and four due to death unrelated to the cardiac-surgical-procedure index. A total of 342 procedures remained that complied with the selection criteria and brought together and that are those that comprise the material of the present report. The general characteristics of the study group appear in Table 2. The frequency of each of the variables of the population studied and its comparison with that obtained in the EuroSCORE study are depicted in Table 3. As we can observe, there are notable differences in the prevalence of the majority of the risk factors. The population of the EuroSCORE study is 12 years older than that of our study and presents higher rates of extracardiac arteriopathy and of the left ventricle expulsion fraction (LVEF) of < 30%. On the other hand, our population has the following: a greater percentage of women; of chronic pulmonary disease; neurological dysfunction; prior cardiac surgery; creatinine values above 200 μmol/L (2.26 mg/dL); active endocarditis; critical preoperatory status; unstable angina; pulmonary arterial hypertension; emergent surgeries, and surgeries different from those of coronary bypass and thoracic aorta surgeries. In Table 4, we can appreciate the distribution of patients according to risk. As can be observed, only 16.9% of patients were low risk. The remainder were medium (44.4%) or high risk (38.6%). This is in agreement with the greater prevalence of more than 50% of the risk factors of our studied population in comparison with the EuroSCORE study population. There were 27 deaths, which yielded a crude mortality rate of 10.8%. In Table 4, we may observe the risk-adjusted mortality as follows: low-risk group 1/58 (expected, < 2%; observed, 1.7%), intermediate-risk group 7/152 (expected, <5%; observed, 4.6%), and the high-risk group 29/132 (expected, > 6%; observed, 22.0%). Table 5 presents the comparison of the variables of the following: the EuroSCORE model; type of surgery; the additive and logistic EuroSCORE score; and time of extracorporeal circulation between the group of patients who died and the group who survived. We observed a higher age in the group of patients who died (53.11 ± 13.87 vs. 49.72 ± 17.5 years, p = ns), as well as a notably greater prevalence in 10 variables of the model (extracardiac arteriopathy, neurological dysfunction, prior cardiac surgery, serum creatinine of > 2.2 mg/dL, active endocarditis, critical preoperatory status, LVEF 30-50%, LVEF of < 30%, PAH > 60 mmHg, and emergent surgery). Consequently, the additive EuroSCORE model (7.92 ± 3.63 vs. 5.11 ± 3.04, p = 0.0005) and the logistic EuroSCORE model (15.62 ± 15.99 vs. 6.70 ± 8.65, p = 0.0005) were greater in the group of patients who died. The extracorporeal circulation pump time (ECPT) was also higher in the group of patients who died (106.33 ± 73.76 vs. 82.91 ± 48, p = 0.0005).

Validation of the EuroSCORE model in our studied population yielded the following results: for the additive version, the area under the receiver operating curve (ROC) was 0.763 (Figure 1), and the χ² of the H-L test was 5.30, with p = 0.62 (Figure 2). For the logistic version, the area under the ROC curve was 0.761 (Figure 3) and H-L test was 8.78, with p = 0.36 (Figure 4).

Figure 1: Receiver operating characteristic (ROC) curve. Area under the curve = 0.76, compatible with good discrimination of EuroSCORE additive model. 

Figure 2: The Hosmer-Lemeshow goodness-of-fit test = 5.30, with p = 0.62. The mortality observed was very similar to the mortality estimated in the three risk groups. 

Figure 3: Receiver operating characteristic (ROC) curve. Area under the curve = 0.76, compatible with good discrimination for the EuroSCORE logistic model. 

Figure 4: The data divided into quintiles show the result of the Hosmer-Lemeshow goodness-of fit test at 8.78, with p = 0.36, compatible with good calibration for EuroSCORE logistic model. 

Discussion

The additive version of the EuroSCORE model was evaluated in a sample of 1,497 patients obtained from the population in which the model was developed, noting discrimination with an area under the ROC curve of 0.76 and a calibration measured with the H-L test of 7.5 (p < 0.68), confirming good internal validation.⁴ Later, the equation employed for the calculation of its logistic version was revealed.⁵ The goodness-of-it of the additive and logistic EuroSCORE model has been evaluated in multiple and different populations, in general cardiac surgery and in its subgroups, with and without the extracorporeal circulation pump. Perhaps this is the probabilistic model that is most evaluated to predict mortality in heart surgery worldwide, with studies conducted in Europe, Asia, Australia, and in America, with different results.^6-18 In Europe, in a study that evaluated the performance of the score in populations of the six countries that contributed more than 500 patients to develop the EuroSCORE model, the authors found that, despite the notable epidemiological differences among these populations, the adjustment of the score for predicting mortality was good, with a discrimination evaluated with the area under the ROC curve of 0.82 and calibration evaluated with the H-L goodness-of-fit test of 0.59 (p ≥ 0.05).⁶ In the U.K., the logistic version of the model was evaluated in nearly 10,000 patients, 67.5% with coronary bypass, 15.33% with isolated valvular surgery, and 15.77% valvular aortic surgery + coronary bypass, observing good discrimination, with an area under the ROC curve of 0.79 for the whole group, of 0.77 for the coronary-bypass sub-group, and of 0.79 for valvular surgery; however, calibration was not good. The authors concluded that the logistic version model possesses good discrimination, but that calibration varies among the different risk sub-groups, with overestimation of the mortality observed. These authors recommended recalibration of the model.⁷ In Spain, in a study that included nearly 500 patients submitted to coronary bypass, adjustment of the model was evaluated and the additive and logistic versions were compared. The model’s discrimination was very good, with an area under the ROC curve of 0.830 for the logistic version and of 0.839 for the additive version. The value for the statistical C of the χ² equation of H-L was 11.51 with p = 0.32 for the logistic version. The authors concluded that the logistic model is in greater approximation to the real mortality in the high-risk patient sub-group.⁸ Other studies have also compared the additive and logistic versions, observing that both versions possess satisfactory discrimination, with areas under the ROC curve of 0.80 (logistic) and 0.79 (additive). However, calibration was better with the logistic version (p = 0.12) than with the additive version (p = 0.001).⁹ In the population with highest risk, better performance of the logistic version of the model was observed.⁵ The results obtained in other studies have also found limitations in the model’s additive version. For example, in a study that included six large international samples, the result observed strongly suggest that the additive version overestimates mortality in low-risk groups (EuroSCORE ≤ 6 points) and underestimates mortality in high-risk groups (EuroSCORE ≥ 13 points).¹⁰

In Australia, in a study conducted with 8,331 patients (valvular, coronary bypass, and aortic surgery) to assess the goodness-of-fit of the model in its population, it was found that the discriminatory power of both variants of the model were very good, with an area under the ROC curve for the whole cohort of 0.83, and for the coronary-bypass sub-group, one of 0.82. However, calibration of both variants was poor in terms of predicting mortality in risk sub-groups, in that this was underestimated in nearly all of the risk deciles (χ² of H-L with a p < 0.05).¹¹ In the U.S., based on the STS data of the EuroSCORE model in its additive version, it revealed good to very good discrimination in general cardiac surgery and in coronary-bypass surgery, with areas under the ROC curve of 0.75 and 0.78, respectively. Calibration was evaluated in five risk sub-groups, noting that the mortality observed was nearly identical to that foreseen for the model in all of the sub-groups. The authors concluded that, despite the epidemiological and demographic differences between Europe and North America, the EuroSCORE model can be recommended as a simple and uncomplicated instrument for application on both side of the Atlantic.¹² In Latin America, the EuroSCORE model has been evaluated in diverse studies. In Brazil, Moraes de Carvalho et al., in a study of four hospitals of the city of Rio de Janeiro with a randomized sample of 546 patients of a population submitted to coronary bypass, the model demonstrated poor discriminatory power, with an ROC curve of 0.62.¹³ In Argentina, Scaro and collaborators conducted a study with 123 patients to test the discriminatory power of the EuroSCORE model, concluding that the model was not useful for predicting mortality, in particular in sub-groups of intermediate and high-risk.¹⁴ In Colombia, Parga-Gómez et al., in a study of 498 patients, found good discrimination for the additive as well as for the logistic version (area under the ROC curve of 0.85 in both versions), in addition to good calibration (statistic C of χ² of H-L 3.39 additive version, and 9.99 for the logistic version with p of 0.8 and 0.2, respectively).¹⁵ In Cuba, Chao-García and collaborators, in a study that included 158 patients submitted to mitral valve surgery, reported an area under the ROC curve of 0.97, that is, excellent discrimination.¹⁶ In Mexico, Careaga et al., in a study of 206 patients submitted to valvular surgery, the authors found an area under the ROC curve of 0.77 for the additive version and of 0.97 for the logistic version, values that are compatible with good and excellent discrimination, respectively. The χ² of H-L was 6.7 (p = 0.034) and 2.86 (p = 0.99), the latter compatible with very good calibration.¹⁷ Rodríguez Chávez et al., in another study in Mexico, evaluated the model in 1,188 patients submitted to valvular surgery and found an Area Under the ROC Curve of 0.707 and of 0.694 for the additive and logistic versions, respectively, while the value of the χ² of H-L was 63.15 (p < 0.001) and 45.6 (p < 0.001) for the corresponding additive and logistic versions. These authors concluded that prediction of mortality in this population of patients, in the additive as well as in the logistic version, was inadequate and that discrimination was scarcely borderline with the additive and poor for the logistic version.¹⁸

In our study, we observed that, while the population evaluated was 12 years younger than the European population from which the model arose, the prevalence of diabetes mellitus (DM) was greater, as well as of the majority of the prognostic variables of the score already cited in the results section. However, despite the epidemiological differences between the EuroSCORE development group and those of our population, the results obtained in the present study are compatible with good discrimination and good calibration of the model, in its additive as well as in its logistic version, and are similar to the results reported in Europe,^6,8 the U.S.,¹² Colombia,¹⁵ and Careaga in our environment.¹⁷ It is noteworthy that our evaluated population included vascular surgery, coronary bypass, and congenital surgery in adults, and also other types of surgery that included infrequent cardiac pathologies such as those produced by non-shooting weapons and/or by firearms, that is, it was a population submitted to cardiac surgery for all of the pathologies.

Why are there discrepancies in the results in different studies? The explanations can be several. On the one hand, the continuous progress in cardiovascular surgery (technological, better organization of medical groups, acquisition of experts of surgical groups and of the groups charged with perioperatory management, among others) and, on the other hand, improvement in the coverage of the health systems in countries on various continents, as well as a population with a higher degree of education in health, have yielded as results the progressive diminution of perioperatory mortality in these patients (despite their being older and having more comorbidities), more notorious in developed countries, which has originated that a determined score that has shown a good prognostic performance during the time in which it was developed, can, a few years later, result in poor discrimination and/or calibration, as suggested in some studies.¹⁹ From this arises the suggestion to update these instruments, as has already been done in some models (in the Parssonnet in its different versions and in the EuroSCORE model, which has already developed model II, and in the STS score, which performs this periodically and in a dynamic manner).³ Perhaps it would also be convenient to develop models for some surgical groups in particular, such as valvular surgery²⁰ and infectious endocarditis surgery,^21-23 as has been carried out in some places, with the purpose of possessing more reliable instruments for utilization in these specific pathologies.

Conclusions

The EuroSCORE model showed to be, at present, a reliable predictive instrument for estimating the probabilities of death in patients submitted to cardiac surgery with extracorporeal pump or without the latter in adult population cared for at the HRAEB.