Introduction

Cardiovascular diseases (CVD) are the leading cause of morbidity and mortality in the world and it is known that more people die every year from CVD than from any other cause. According to the World Health Organization,¹ just in 2015, an estimated 17.7 million people died from CVD, which represents 31% of the total deaths recorded in the world. Some estimates indicate that an acute myocardial infarction occurs every four seconds and a cerebral vascular event every five seconds, and at least one in three people lose their lives due to some pathology related to CVD.²

More than three quarters of CVD deaths occur in low and middle-income countries, particularly in Latin America, ischemic heart disease and CVD represent 8.96 and 6.11% respectively of the causes of death in the region, and these chronic problems along with diabetes, cause more deaths in Latin America than in the United States of America and Canada.³

In México, the National Institute of Statistics and Geography (INEGI)⁴ reported 141,619 deaths from cardiovascular diseases and 101,877 from ischemic heart disease, which together accounts for 34.63% of all deaths in 2017. Some studies with a population registered at IMSS (government health insurance) in Mexico have found that since 1990 there has been a decrease in the prevalence of lethality and mortality due to CVD, however it is recognized that there is not a decrease in incidence yet.⁵,⁶ This agrees with statistics of the Organization for Economic Cooperation and Development, which shows that México does not have the highest death rate due to this cause among the member countries of this organization, but it ranks second with the highest percentage of incidence rates between 1990 and 2011.⁷

As in any public health problem, cardiovascular diseases require screening measures for a more simple and rapid detection and with a positive relationship in terms of effectiveness/cost and efficiency/time. In this sense, questionnaires and subjective self-reports of illness can represent an additional relevant contribution to physical medical examinations and laboratory tests, especially at the level of primary or secondary prevention. These questionnaires and scales do not substitute any clinical diagnosis properly, but they should be understood as general screening tests, and in this sense, their usefulness can be an advantage at prevention level, as long as their validity, sensitivity, and specificity are acceptable. The self-reports scales of subjective symptoms have been a useful and widely used measures in the assessment of different health problems and especially have been useful in the field of epidemiological research because of the logistical facilities they represent.

However, for the specific case of CVD, the use of questionnaires or self-report scales are scarce in the literature, perhaps due to the nature of such diseases that represent underlying alterations of blood vessels that are often asymptomatic and their first manifestation can be directly a heart attack or a cerebrovascular event.⁸ Some studies have found weak or absent relationships between the subjective self-report and the clinical diagnosis of CVD or its risk factors; for example, Tenkorang,⁹ and Dave¹⁰ located inconsistencies between self-reported hypertension and the one obtained in biometric data (sensitivity of 33%); Similarly, Molenaar et al¹¹ in a population-based prospective study, obtained for self-reports only a sensitivity of 33.3% in the case of hypertension, and 58.9% for diabetes mellitus type II. On the other hand, Natarajan et al¹² and Dey et al,¹³ found that self-reported hypercholesterolemia also has low levels of sensitivity (44 and 57.5% respectively).

Notwithstanding the foregoing, in a German study, the medical condition of more than 7,000 participants was assessed by means of physical and laboratory examinations at the same time as the subjective self-report of diseases such as hypertension, diabetes, cardiac arrhythmias and angina pectoris. Among others; they found substantial levels of agreement between both measurements with significant percentages ranging from 83 to 96%.¹⁴ In another study,¹⁵ they found sensitivity levels of up to 92% in the subjective self-report of hypertension, and others have also found reasonable consistencies, including Hispanic populations,¹⁶ which means that there are still inconsistencies in the literature regarding this topic.

Several authors have pointed out that the accuracy of a subjective self-report of the medical condition of the same patient may be influenced by factors such as: age, sex, race, knowledge and understanding of relevant medical information, memory capacity, disposition to communicate what it feels and, of course, the specific cardiovascular disease or risk in question.¹⁰,¹²-¹⁴,¹⁷

Although some authors agree that the subjective self-report is slightly more consistent for diabetes than any other disease,¹⁰,¹⁸ other follow-up studies showed that the self-report is a better predictor of the incidence of cerebrovascular accident (CVA) and myocardial infarction (MI), compared to those obtained in other diseases.¹⁴,¹⁹

This has some relation with what has been obtained in experimental studies of cardiovascular activation response, where for several decades it has been concluded that the subjective cardiovascular self-report provides a more accurate indicator of the overall level of arousal of the whole body in relation to that obtained in other specific biomarkers.²⁰,²¹ So perhaps the perception of the health of the heart in a global way is more accurate than the one present in some of its risk factors separately.

In this sense, Bowlin et al²² have suggested combining different repeated measures in a single indicator to improve the sensitivity and specificity of CVD self-reports, which may partly explain the limited results of previous studies that have used a single measurement or an only question to assess the self-report of a disease, such is the case of questions like «Do you suffer from hypertension?». In psychometrics it is well known that the accuracy of a measurement using a single question is weak, and therefore the use of «scales» of subjective self-report through the use of several items (which can be summative) allows not only to ensure a greater a view of what is valued, but the content validity is increased and greater consistency is obtained in the measurement, which gives rise to the concept of reliability. Reliability theory is based on the assumption that a score observed in a particular value of a random variable represents all possible scores that could have been repeatedly reported by a person.²³

The available literature does not show any proposal for validation of a self-report scale with several items that explore cardiovascular symptoms for the overall assessment of heart disease, which is of great need and can represent great advantages for population screening. Among the benefits of having a tool of this nature are: 1) low cost, 2) speed and ease of identification of possible risk without large logistical deployments, 3) decrease in the use of specialized human resources and devices, 4 ) avoidance of invasive procedures, 5) ease of people having global and first-hand information about their possible cardiovascular risk, especially in cases where another form of medical assessment is inaccessible, and 6) the possible availability of a scale of cardiovascular screening for research purposes in large samples.

Given the above, the present work has two objectives: the first one is to propose a scale of self-report of cardiovascular symptoms and analyze their psychometric properties through the analysis of internal consistency (reliability) and its factorial structure (construct validity); and second, to explore the criterion validity of the scale through correlation analysis with different cardiovascular indicators, estimating in turn the sensitivity and specificity through ROC curves.

To achieve these objectives, two studies were carried out with designs and samples that were distinguishable in nature: the first study in a sample of apparently healthy people from whom various biomarkers of their cardiovascular health were obtained as validation criteria, and a second comparative study of cases (with cardiopathy) and not cases (healthy), which are described below.

Material and methods

Results (Study 1)

With respect to the proposed scale of 10 items, the descriptive statistics showed an average of 1.41, which in the frequency scale refers to symptomatology between «never» (1) and «some occasions» (2). The asymmetry indicators showed a positive bias, which together with the kurtosis in relation to their errors, indicate low frequencies and lack of compliance with the assumption of normality. It is worth mentioning that some respondents (n = 9) showed understanding difficulties in items 9 (sudden weakness or going to faint) and 10 (chest pain in the face of exertion that causes them to stop). The item-total correlations (not shown by space issues) showed that the lowest relations were precisely with items 9 and 10 (r = 0.49 and r = 0.43 respectively), which also had the lowest means (1.24 and 1.15 respectively), thus showing some psychometric difference with the rest of the items, so it was decided to also try a version of the scale without those two items (ESCV10 vs ESCV 8) (Table 2).

Eventhough it was not the main objective of this work it is worth it to point out that the descriptive statistics show a prevalence of total cholesterol in the drivers of 41.7%, which was higher than the general population. Likewise, the prevalences of the body mass index (86.1%) and triglycerides (66.9%) in the participants were also high, although the prevalences of other biomarkers such as blood pressure were not so high.

With respect to the psychometric behavior of the scale, the internal consistency indices showed that the 10-item version obtained a α = 0.80 (CI 0.73-0.85) and a ω = 0.85; while for the 8-item version a α = 0.78 (IC 0.70-0.83) and a ω = 0.84. These coefficients were satisfactory, including the minimum value within the confidence interval, and without statistically significant differences between them (χ² = 0.27, p = 0.60). On the other hand, the fit indices of the confirmatory factor analyses showed the expected one-dimensional structure satisfactorily, but only for the 8-item version (CFI = 0.96, B-NNFI = 0.94 and RMSEA ≤ 0.04), which confirmed the limited contribution of items 9 and 10 to the psychometric performance of the scale and the greater effectiveness of the 8-item version. In this respect, the comparative χ² between Satorra Bentler indices showed a statistically significant difference (p = 0.003) between both models (version ESCV-8 vs ESCV10), being in principle, more acceptable in the 8-item version (χ² = 25.96, p = 0.16). In general, the factorial loads were moderate to high (0.42-0.87) in almost all the items in both versions (Table 3).

As for the convergent validity of the proposed scale, both versions correlated in similar ways, with some cardiovascular risk indicators, although, the 8-item version (ESCV-8) showed slightly superior correlations. Meaningul statistical correlations (low magnitude, although) were observed with high fat diets, exercise frequency, low density cholesterol (LDL), high density cholesterol (HDL) and total cholesterol, the last one only in the 8-item version. Even that a higher number of correlations were expected, the ones obtained were in the expected directions, with the only exception of the HDL correlation, which was positively correlated with other lipids (Table 4).

LDL cholesterol was chosen as a possible standard to estimate the sensitivity and specificity of the scale since it was the cardiovascular risk indicator with the highest correlation and the most objective of the criteria with which it correlated the proposed scale. For the ESCV-10 version, there was an area under the curve (AUC) = 0.61 not significant (95% CI = 0.49-0.72), and for the ESCV-8 the ROC curve obtained indicated AUC = 0.63 which was statistically significant (95% CI = 0.51-0.74), which represented greater accuracy compared to the 10-item version (Table 1). The Youden index was calculated to obtain the optimal cut-off point in each scale,³⁵ being 1.45 for the ESCV-10 (sensitivity = 0.50 and specificity = 0.69) and of 1.43 for the ESCV-8 (sensitivity = 0.61 and specificity = 0.61), both very close to the obtained means (Figure 1).

Figure 1: ROC curves of sensitivity and specificity of ESCV-8 and ESCV-10 (study 1). 

For a better reflection of the present work, the discussion and conclusion of both studies (1 and 2) were carried out jointly at the end, so that the study is described below 2.

Results (Study 2)

The confirmatory factorial analyzes that sought to prove the same one-dimensional structure of the scale, showed acceptable fit indices for the 8-item version (CFI = 0.93, B-NNFI = 0.90, RMSEA = 0.09) and less acceptable for the version of 10 items (CFI = 0.92, B-NNFI = 0.89, RMSEA = 0.09). Likewise, although the Satorra Bentler fit indexes were inadequate in both versions (p < 0.05), it has been observed that this can be due to large sample sizes or strong correlations between the indicators, so as corrected additional fit index³³ a lower value than three in the ratio between the coefficient χ² and the degrees of freedom (gl) has been suggested, a criterion that was fulfilled in the two versions. Notwithstanding the foregoing, the χ² test of differences between both models was statistically significant (p = 0.01), thus showing a relative improvement in the 8-item version. In both versions, the factorial loads were moderate to high (0.46-0.88) in all the items. In the internal consistency we found a α = 0.87 and ω = 0.90 for the 10-item version, and for the 8-item version it was 0.84 and 0.88 respectively, without statistically significant differences in the α of the 2 versions (p = 0.31), which together can confirm a high internal consistency or reliability in both versions (Table 6).

The t-test of differences between cases and non-cases showed statistically significant differences in both versions of the total scale (ESCV10 and ESCV8) in practically all items (except item 2), thus demonstrating greater symptomatology in people diagnosed with heart disease in comparison to healthy ones, which confirms the scale’s ability to discriminate them. On the other hand, the point biserial correlations were of moderate to high magnitude, thus confirming the criterion validity of the scale, even with a stronger and clearer tendency than in study 1 (Table 7).

Finally, the ROC curve analyzes in this second study showed, for the ESCV-10 version, an area under the curve (AUC) = 0.81 statistically significant (CI 95% = 0.73 -0.88, p = 0.0001), a sensitivity of 84% and specificity of 70%, with the cut-off point of 1.80 (Youden index = 0.50). For the ESCV-8 version, the ROC curve obtained indicated an AUC = 0.80 which was also statistically significant (95% CI = 0.72-0.88), a sensitivity of 80% and specificity of 70% with a cut-off point of 1.80 (Youden index = 0.50), similar to that obtained in the 10-item version (Figure 2).

Figure 2: ROC curves of sensitivity and specificity of ESCV-8 and ESCV-10 (study 2). 

Discussion

The main purpose of this study was to analyze the validity of a proposed scale to measure cardiovascular symptoms in two ways: by analyzing its factorial structure and internal consistency, on the one hand, and on the other, its convergence with external criteria of cardiovascular health, allowing in turn to explore its sensitivity and specificity in two different studies. The first study involved apparently healthy people who were unaware of their cardiovascular health, and as validity criteria, different biomarkers were used as cardiovascular risk factors, and in the second study, as a comparative validation strategy, healthy and patients with heart disease were evaluated.

The initial exploration of the first study showed two psychometrically vulnerable items (9.- «He has felt sudden weakness or that he is going to faint» and 10.-«He has felt pain in the chest before an effort that has caused him to stop»), so it was decided to try an 8-item version (ESCV-8), in addition to the original version of 10 items (ESCV-10), in which the same validity scans were performed.

With regard to the reliability of the scale, the calculations of omega (ω) and alpha Cronbach (α) showed a fairly satisfactory internal consistency of the two versions, in both studies. The value ω reached a value of 0.90 which is considered «high» and on the other hand, the minimum value in the confidence intervals of α was always greater than 0.70, that is, that even the lowest value estimate possible in the population parameter (with 95% certainty) achieves the minimum recommended to consider this coefficient as acceptable.²⁸,³¹ With the above, it can be concluded that the scores of the proposed scale have satisfactory internal consistency, reliability or accuracy in both versions.

Regarding factorial validity, confirmatory analyzes generally confirmed a one-dimensional structure of the scale in both versions, although the adjustment indices obtained and comparison of Satorra Bentler adjustment between models, showed that the 8-item version had better psychometric performance in the two studies. In particular, the 8-item version in study 1 (apparently healthy people) had the highest adjustment indices, which fall within the most demanding criteria³⁶ (e.g. CFI = 0.96, RMSEA = 0.04). On the other hand, study two showed a marginal to acceptable compliance in the adjustment indices despite the release of the covariance restriction between the errors of items 6 and 7, in particular the RMSEA (0.09), which refers to the amount of variance not explained by the model by degree of freedom. One possible explanation for this is the heterogeneity of the participants (with heart disease and healthy) who had to mix for reasons of maintaining statistical power in the factor analyzes. Notwithstanding the above, considering both studies, the indices were acceptable in general (in particular the ESCV-8 version) and although the factorial loads were heterogeneous, they were always higher than 0.40 in all the analyzes, which shows a good representation of the construction of all the indicators and their evident structural validity.

Regarding criterion validity or convergent, study 1 showed that ESCV-8 had significant correlations with 5 of the 11 cardiovascular risk indicators evaluated in the participants and the ESCV-10 with 4 of 11, however, the correlations were of low magnitudes. The area under the curve (AUC) was only 0.61 for the ESCV-10 and did not reach statistical significance, and although in the ESCV-8 it did reach significance, it was barely 0.63, with a sensitivity of 0.61 and specificity of 0.61, so that using a cut-off point of 1.43 (ESCV-8), approximately 61% of the participants had a «high» score (≥ 1.43) when the biomarker (LDL) was abnormal and 61 % had a «low» score (< 1.42) when it was normal. This means that 39% of the participants who had pathological levels in the chosen biomarker had a low score on the scale, which shows a weak convergent validity when CVD risk factors were used as criterion (particularly LDL), this was just in the case of the «apparently healthy» participants in study 1.

In study 2, a more forceful validity scenario was presented, since the scale in its two versions and each one of the items managed to discriminate in an outstanding way to healthy and unhealthy people with similar sociodemographic characteristics, and the specific biserial correlations showed effects of moderate to high magnitude when cardiovascular disease was used as a criterion, thus confirming the criterion validity of the scale. The area under the curve was above 80% with statistical significance, and the specificity was 70% in both versions, the sensitivity was slightly higher in the 10-item version (84 vs. 80), so using a point of cut of 1.80, more than 80% of patients with heart disease had high scores, while 70% of participants who were healthy had low scores, this means that at most 20% of patients with heart disease (using the ESCV-8 version) obtained low scores, which means a satisfactory sensitivity at the screening level, of the proposed scale.

The vulnerable criterion validity found in study 1 (unlike study 2), recalls the inconsistency in the literature of the accuracy of self-reports to assess the objective medical condition of people, however, this may perhaps be explained in part by the traditional influence of social desirability or willingness to communicate the symptoms in the participants, who were all men in a work context where culturally it is inappropriate to show weakness (public transport drivers). Notwithstanding the foregoing, it could also be valid to interpret that the proposed scale of cardiovascular symptoms allows more discernment of cardiovascular disease in more chronic and/or more advanced stages and not in people with CVD risks factors, since the biomarkers used in this study do not represent a cardiovascular disease by itself but just a «proxy», and also they were taken in a transectional way, without the certainty that their values were maintained over time, and therefore, that high levels of any of them mean a chronic condition (eg: high blood pressure ≠ hypertension). Also, these results could be expected since some studies have shown low sensitivities (around 33%) in the case of hypertension 9-11 and between 44 and 57% for biomarkers such as hypercholesterolemia,¹²,¹³ while for cardiac arrhythmias and clinically diagnosed angina pectoris have found sensitivities above 80%¹⁴ and above all, in the case of more serious diseases such as cerebrovascular accidents or myocardial infarctions.¹⁴,¹⁹ This is consistent with the present work in the sense that the scale proposed was able to better discern when the criteria were participants diagnosed with the specific disease of coronary heart disease by a certified cardiologist (and the respective clinical trial cabinet) (study 2), that when only some blood markers of the apparently healthy participants were considered, which they could be very non-specific (unstable levels) and at the same time very general (cardi o-metabolic in general) (study 1), which also confirms what was said by Mackay et al²⁰ and Cacioppo et al²¹ on the accuracy of the self-report as a better indicator of overall cardiovascular health of a person (arousal level) and at the same time, it is a specific indicator (cardiovascular and not any metabolic alteration).

It is worth discussing that the apparent dysfunction of items 9 and 10 that led to the formulation of an 8-item version with better psychometric performance may be due to an error detected in its design, as they are the only ones that include two or more simultaneous affirmations in its content: 9.-weakness-faint-, and 10.-pain-effort-stop-. Given the above, it is suggested to rethink it in future studies and to assess its elimination, since its indicators of internal consistency and validity in general, show the possibility of improvement.

Conclusions

Finally, it can be concluded that, in general, the proposed scale presents satisfactory construct and convergent validity as a screening measure, although we consider that further future studies are necessary. In the meantime, in particular the ESCV-8 version could be used for screening or cardiovascular health screening in general populations with the reserves of the case, considering the advantages already mentioned (costs, speed, resources, etc.), and with cut-off points. that could place people in the following categories: «without risk» up to 1.43; «possible risk» of 1.44 to 1.79 and «high risk» of 1.80 and up.