Investigación clínica

 

Effects of presenting risk information in different formats to cardiologists. A Latin American survey

 

Encuesta sobre los efectos de la presentación de los riesgos y beneficios en distintos formatos a cardiólogos latinoamericanos

 

Raúl A. Borracci^a,b*, Daniel J. Piñeiro^a, Eduardo B. Arribalzaga^b,c

 

^a Sociedad Interamericana de Cardiología, México, D.F., México

^b Bioestadística, Facultad de Ciencias Biomédicas, Universidad Austral, Argentina

^c Facultad de Medicina, Universidad de Ciencias Empresariales y Sociales (UCES), Argentina

 

* Corresponding author at:
La Pampa 3030, 1428
Buenos Aires, Argentina.
Tel.: +54 11 4784 5917; fax: +54 11 4961 6027.
E-mail address: raborracci@gmail.com (R.A. Borracci).

 

Received July 22, 2013
Accepted September 2, 2014

 

Abstract

Objective: Previous publications demonstrated that the presentation of treatment benefits in terms of relative risk reduction (RRR) rather than in terms of absolute risk reduction (ARR) or number of patients to treat (NNT) might favor the perception of outcome effectiveness. The objective was to perform a cognitive evaluation to assess how the manner in which risks and benefits of screening methods and treatments are presented can affect medical care decision-taking in a sample of cardiologists.

Methods: Four-hundred and six Latin American cardiologists answered a questionnaire reporting the results of clinical trials presented as RRR, ARR or NNT, associated or not to biased graphs.

Results: Cardiologists' decision-taking was different when comparing treatment benefits presented as RRR (62.2%) vs. ARR (40.4%) (p = 0.000000), and as RRR vs. NNT (44.4%) (p = 0.000000). However, their decision-taking was similar when information was presented as NNT or ARR (p = 0.073). The inclusion of biased graphs was misinterpreted as an actual data difference (RRR: 61.6% vs. ARR: 14.0%, p = 0.000000).

Conclusions: This study demonstrated that Latin American cardiologists could misinterpret statistical data when information of clinical trials is presented in terms of RRR. We emphasize the need to enhance cardiologists' training in quantitative techniques, to improve medical care decision-making.

Keywords: Biostatistics; Risks; Decision-taking; Cardiology; Latin America; Argentina.

 

Resumen

Objetivo: Publicaciones previas demostraron que la presentación de los beneficios terapéuticos en términos de reducción del riesgo relativo (RRR) en lugar de en términos de reducción del riesgo absoluto (RRA) o número necesario a tratar (NNT) favorecería la percepción de que los resultados son más efectivos. El objetivo fue realizar una evaluación en una muestra de cardiólogos para determinar cómo afecta a la decisión médica la manera como se presentan los riesgos y beneficios de métodos diagnósticos y terapéuticos.

Métodos: Cuatrocientos seis cardiólogos latinoamericanos respondieron a un cuestionario que contenía los resultados de ensayos clínicos como RRR, RRA y/o NNT, asociados o no a gráficos sesgados.

Resultados: La toma de decisión de los cardiólogos fue diferente al comparar los beneficios terapéuticos presentados como RRR (62.2%) vs. RRA (40.4%) (p = 0.000000), y como RRR vs. NNT (44.4%) (p = 0.000000); sin embargo, fue similar cuando la información se presentó como NNT o RRA (p = 0.073). La inclusión de gráficos sesgados fue malinterpretada como una diferencia real (RRR: 61.6% vs. RRA: 14.0%, p = 0.000000).

Conclusiones: Se demostró que los cardiólogos latinoamericanos podrían confundir resultados estadísticos cuando la información de los ensayos clínicos se presenta como RRR. Hacemos hincapié en la necesidad de entrenar a los cardiólogos en las técnicas cuantitativas para mejorar la toma de decisiones.

Palabras clave: Bioestadística; Riesgos; Toma de decisión; Cardiología; América Latina; Argentina.

 

Introduction

Misinterpretation of numerical data can be induced by the form in which the risks and benefits of clinical trials are reported. These different interpretations influence not only the opinion of medical students and patients, but also the physician care decision-taking process.^1–3 Previous research demonstrated that presenting treatment benefits in terms of relative risk reduction (RRR) rather than in terms of absolute risk reduction (ARR) or number needed to treat (NNT) might favor the perception of outcome effectiveness.⁴ Clinical experience and medical training do not seem to be sufficient to protect physicians against the tendency to interpret data erroneously. A recent meta-analysis by Covey⁵ has shown that these framing effects are heterogeneous and seem to be moderated by key differences between the methodologies used in each study. Several authors have speculated that differences in the presentation of outcomes may lead to discrepancies in decision-making and treatments, but previous studies have reported a wide variety of results. Therefore, considering the importance of the problem in cardiological medical practice, a cognitive evaluation study was performed in a group of Latin American cardiologists to assess how the manner in which risks and benefits of screening methods and treatments are presented can affect medical care decision-making.

 

Methods

This study was performed in a non-randomized sample of 406 Spanish speaking cardiologists included in the Interamerican Society of Cardiology register, proportionally representing the regional distribution of Cardiology specialists. They underwent via e-mail an anonymous and voluntary assessment on the interpretation of results of several medical investigations. The questionnaire was in Spanish and consisted of nine questions with multiple choice answers. The questions presented the summary of clinical results from different controlled trials, showing the data as relative risk reduction (RRR), absolute risk reduction (ARR) or number needed to treat (NNT), with or without associated graphs. Questions were randomly ordered and were associated as follows:

Group 1: a question expressed with RRR and NNT, the same question with RRR, the same question with ARR and the same question with NNT.

Group 2: a second question expressed with AAR, the same question with RRR and the same question with NNT.

Group 3: finally, a third question presented with RRR and the same question with ARR, together with corresponding graphic representations. In this case, the relative proportion of the graphs was biased to favor the choice of RRR. The abridged questions are shown in Table 1. To compare with other studies, test questions were formulated according to three previous studies on the same subject.^4,6,7 The wording of these questions intended to overcome previous study problems, mainly through the addition of examples containing NNT, the combination of RRR + NNT and supporting graphs. At the end of each example, the physician was asked to consider these results as sufficient evidence favoring the proposed treatment. The multiple choice answer to each question could be answered on some occasions with "yes, no, or I do not have sufficient information", and in others with "very useful, moderately or slightly useful, not useful, or I do not have sufficient information".

The analysis of results was performed comparing the rate of favorable choice for RRR presentations vs. ARR and NNT. The study hypothesis postulated that respondent cardiologists would choose more easily RRR rather than ARR or NNT as evidence favoring a treatment. Moreover, questions from groups 1 and 2 were conjointly analyzed. Thus, the total number of answers was 812 (406 respondents with 2 questions each) in the case of ARR, 812 (406 respondents with 2 questions each) in the case of RRR and 1218 (406 respondents with 3 questions each) in the case of NNT and RRR + NNT (2 questions in group 1 and one question in group 2).

Groups were compared using χ². The odds ratio logarithm (log-OR) with its 95% confidence interval (CI_95%) was calculated to compare the study results with those previously published with the same format.

 

Results

Fig. 1 shows the participating countries and the proportion of respondent cardiologists is compared to the total number of cardiologists in each country, except for Brazil, as the proportion of Spanish speaking physicians is negligible in this country. The same graph shows the proportional distribution of cardiologists in the region.

In the conjoint analysis of group 1 and 2 questions, 62.2% (505/812) of the respondents accepted the proposals as valid when the data were presented as RRR, whereas only 40.4% (328/812) accepted it when the results were presented as ARR. On the other hand, 44.4% (541/1218) cardiologists validated the proposals when the data were expressed as NNT (p = 0.000000). The comparison between answers when the propositions were made with RRR and ARR was log-OR = 0.89, CI_95% 0.683–1.092, p = 0.000000, and the comparison of NNT vs. ARR was log-OR = 0.17, CI_95% -0.020 to 0.351, p = 0.073, indicating that the respondent cardiologists managed the statistical evidence in the same manner both with ARR as with its equivalent NNT. Finally, the comparison between RRR and NNT was log-OR = 0.72, CI_95% 0.536–0.908, p = 0.000000. In group 3, where questions were presented with biased bar charts, 61.6% (250/406) cardiologists accepted as recommendable the proposed screening method when the data were presented in the first place as RRR; however, when the same problem was presented as ARR, only 14.0% (57/406) cardiologists accepted the method as useful (log-OR = 2.28, CI_95% 1.926–2.643, p = 0.000000) (Fig. 2).

 

Discussion

Studies on cognitive psychology demonstrate that the possible interpretation of quantitative evidence is deeply affected by the way in which it is presented.^8,9 Thus, the interpretation of numerical information may differ depending on the data presentation format. For example, a 25% RRR can be generated both from the difference in event rates between two groups of 40% and 30% as well as from the difference between two groups of 3% and 4%; which in the first case represents an absolute difference of 10% and in the second case only 1%. If the data were only presented as RRR, the tendency would be to accept these differences as having clinical relevance, neglecting or ignoring the real event rate reduction. Although medical journals require in general the publication of the real event rates in terms of ARR and/or NNT, there is no uniform policy in the presentation of clinical study results, either among researchers or in the pharmaceutical industry marketing.^6,10 The "illusion of benefit" produced when results are presented as RRR has strong implications in the acceptance and adoption of innovations. In our work, although the respondents must already be trained to identify different measurement presentations of effect magnitude, faults were found in the interpretations of the results. When the benefits were presented in terms of RRR, more than 60% cardiologists accepted as valid evidence which should have been rejected or at least questioned. This percentage was significantly reduced both in the presentation of data as ARR or NNT. Graphics intentionally biased, generated even more significant confusion in the study. The magnification of relative proportions in the bar graphs was easily confused with real data difference.

It is essential to train students and physicians in the quantitative analysis of statistical techniques in order to correct these issues and improve decision-taking. The results of other studies as well as ours, demonstrate that the current learning process of these methods is apparently inadequate and unsatisfactory.^11–13 The meta-analysis performed by Covey⁵ concluded that physicians as well as students and public in general may be equally influenced by the manner in which results are presented; in other words, medical training and clinical experience are not enough to immunize subjects against the tendency to interpret data differently when it is presented in terms of RRR, ARR or NNT. This difficulty to acquire the adequate knowledge in mathematics and statistics to interpret health decision making has been thoroughly studied under the name of numeracy.^14,15 The importance of this concept is based on the fact that the training and comprehension of statistical tools might not be within the reach of many health-care decision takers, and particularly the patients who might wish to make their own health decisions.¹⁶

Fourteen former studies analyzing the effects of presenting treatment benefits or therapeutic risks in different formats to physicians belonging to several fields of medicine, have reported a wide dispersion in the results.^{4–6,17–27} Grouped together, the weighted log-OR was 1.19 when comparing RRR vs. ARR and 1.08 in the comparison of RRR vs. NNT, which would indicate a higher tendency to accept a result as valid when it is presented in terms of relative risk. Although these overall findings differ slightly from those observed in our work, there were minimal differences when the individual results of Lacy¹⁸ and Ward²³ for RRR vs. ARR and of Bucher,²⁰ Nikolajevic²² and Nexoe^25,26 for RRR vs. NNT were considered. Specifically, in a study performed by cardiologists in Spain, Brotons et al.²⁴ reported that presenting clinical trials results in the form of RRR, as compared with presenting results in terms of ARR or NNT, led to overestimation of the efficacy of interventions, without influencing the likelihood of prescribing a given drug therapy.

 

Conclusions

The assessment of a Latin-American sample of cardiologists showed a different decision-taking in the comparison of the therapeutic benefits observed in a series of clinical trials when the presentation was done in terms of relative risks instead of absolute risks. There was a tendency to favor outcomes when the data was presented as RRR. The inclusion of biased graphics was also interpreted as real data difference. This study demonstrated the risk of statistical data misinterpretation and the need to improve cardiologists' training in quantitative techniques, in order to ameliorate the care decision-making process.

 

Funding

No endorsement of any kind received to conduct this study/article.

 

Conflict of interest

The authors declare no conflict of interest.

 

References

1. McGettigan P., Sly K., O'Connell D. The effects of information framing on the practices of physicians. J Gen Intern Med 1999;14:633-42.         [ Links ]

2. Edwards A., Elwyn G., Covey J. Presenting risk information: a review of the effects of "framing" and others manipulations on patients outcomes. J Health Commun 2001;6:61-82.         [ Links ]

3. Moxey A., O'Connell D., McGettigan P. Describing treatment effects to patients: how they are expressed makes a difference. J Gen Intern Med 2003;18:948-59.         [ Links ]

4. Forrow L., Taylor W.C., Arnold R.M. Absolutely relative: how research results are summarized can affect treatment decisions. Am J Med 1992;92:121-4.         [ Links ]

5. Covey J. A meta-analysis of the effects of presenting treatment benefits in different formats. Med Decis Making 2007;27:638-54.         [ Links ]

6. Naylor C.D., Chen E., Strauss B. Measured enthusiasm: does the method of reporting trial results alter perceptions of therapeutic effectiveness?. Ann Intern Med 1992;117:916-21.         [ Links ]

7. Wolf A.M.D., Schorling J.B. Does informed consent alter elderly patients' preferences for colorectal cancer screening? Results of a randomized trial. J Gen Intern Med 2000;15:24-30.         [ Links ]

8. Tversky A., Kahneman D. The framing of decisions and the rationality of choice. Science 1981;211:453-8.         [ Links ]

9. Kahneman D., Tversky A. Choices, values, and frames. Am Psychol 1984;39:341-50.         [ Links ]

10. Dryver E., Hux J.E. Reporting of numerical and statistical differences in abstracts: improving but not optimal. J Gen Intern Med 2002;17:203-6.         [ Links ]

11. Woloshin S., Schwartz L.M. Press releases: translating research into news. JAMA 2002;287:2856-8.         [ Links ]

12. Moynihan R.L., Bero L., Ross-Degnan R. Coverage of the news media of the benefits and risks of medications. N Engl J Med 2000;342:1645-50.         [ Links ]

13. Thomson R., Edwards A., Grey J. Risk communication in the clinical consultation. Clin Med 2005;5:465-9.         [ Links ]

14. Nelson W., Reyna V.F., Fagerlin A. Clinical implications of numeracy: theory and practice. Ann Behav Med 2008;35:261-74.         [ Links ]

15. Lexchin J. Statistics can affect treatment decisions. Can Fam Physician 2003;49:1587-8.         [ Links ]

16. Reyna V.F., Nelson W.L., Han P.K. How numeracy influences risk comprehension and medical decision making. Psychol Bull 2009;135:943-73.         [ Links ]

17. Damur C., Steurer J. Do doctors Judie therapy results differently from students?. Schweizerische Medizinische Wochenschrift 2000;130:171-6.         [ Links ]

18. Lacy C.R., Barone J.A., Suh D.C. Impact of presentation of research results on likelihood of prescribing medications to patients with left ventricular dysfunction. Am J Cardiol 2001;87:203-7.         [ Links ]

19. Bobbio M., Demichelis B., Giustetto G. Completeness of reporting trial results: effect on physicians' willingness to prescribe. Lancet 1994;343:1209-11.         [ Links ]

20. Bucher H.C., Weinbacher M., Gyr K. Influence of method of reporting study results on decision of physicians to prescribe drugs to lower cholesterol concentration. Br Med J 1994;309:761-4.         [ Links ]

21. Cranney M., Walley T. Same information, different decisions: the influence of evidence on the management of hypertension in the elderly. Br J Gen Pract 1996;46:661-3.         [ Links ]

22. Nikolajevic-Sarunac J., Henry D.A., O'Connell D.L. Effects of information framing on the intention of family physicians to prescribe long-term hormone replacement therapy. J Gen Intern Med 1999;14:591-8.         [ Links ]

23. Ward J.E., Shah S., Donnelly N. Resource allocation in cardiac rehabilitation: Muir Gray's aphorisms might apply in Australia. Clin Manag 1999;8:24-5.         [ Links ]

24. Brotons C., Moral I., Ribera A. Methods of reporting research results and their influence on decision-making by cardiologists prescribing drugs for primary and secondary prevention. Rev Esp Cardiol 2002;55:1042-51.         [ Links ]

25. Nexoe J., Gyrd-Hansen D., Kragstrup J. Danish GPs' perception of disease risk and benefit of prevention. Fam Pract 2002;19:3-6.         [ Links ]

26. Nexoe J., Oltarzewska A.M., Sawicka-Powierza J. Perception of risk information: similarities and differences between Danish and Polish general practitioners. Scand J Prim Health Care 2002;20:183-7.         [ Links ]

27. Heller R.F., Sandars J.E., Patterson L. GPs' and physicians' interpretation of risks, benefits and diagnostic test results. Fam Pract 2004;21:155-9.         [ Links ]