Defining decision thresholds for judgments on health benefits and harms using the grading of recommendations assessment, development, and evaluation (GRADE) evidence to decision (EtD) frameworks: a randomized methodological study (GRADE-THRESHOLD)
Por:
Morgano GP, Wiercioch W, Piovani D, Neumann I, Nieuwlaat R, Piggott T, Alonso-Coello P, Mbuagbaw L, Rigoni M, Bognanni A, Celedon N, Mustafa RA, Pottie K, Leontiadis GI, Akl EA, Bonovas S, Schünemann HJ
Publicada:
1 mar 2025
Ahead of Print:
1 ene 2025
Resumen:
Background and Objective: GRADE and other evidence to decision (EtD) frameworks are widely used by guideline development groups (GDG) and other decision-makers. When GDGs judge the magnitude of desirable and undesirable health outcomes on EtDs, they typically categorize them as trivial, small, moderate, or large. However, generic judgment or decision thresholds (DTs) that could guide the user about such estimates of effect size or serve as references for interpretation of findings are not yet available. The objective of this study was to empirically derive DTs for EtD judgments about the magnitude of dichotomously assessed health benefits and harms. Methods: We conducted a methodological randomized controlled trial to derive empirical DTs across conditions and health outcomes. We invited stakeholders, including clinicians, epidemiologists, decision scientists, health research methodologists, experts in health technology assessment (HTA), members of GDGs, patient representatives, and the public to participate in the trial. We employed randomly assigned case scenarios to elicit ranges of absolute risk differences judged as small and moderate effects from study participants. We then used the collected data to derive empirical DTs. We also investigated the validity of our DTs by measuring the agreement between judgments that were made by GDGs in the past and the judgments that our DTs approach would suggest if applied to the same guideline data. Results: A total of 445 stakeholders accessed the survey of which 409 were randomised and 288 rated at least one case scenario. Based on these participants, the study findings support our a priori hypothesis of a difference in the DTs for trivial, small, moderate, and large effects and are suggestive of a relation between raters' judgments and the joint measure of absolute effects and outcome values. The results permit the use and calculation of DTs for a variety of scenarios and we present three ways of how to use the results practically. Conclusions: In this trial we confirmed that empirically derived DTs discriminate between judgments on the EtDs. These DTs can be used for judgments about desirable and undesirable health effects in systematic reviews or to initiate and inform a discussion with a GDG. This ensures consistency in judgments across different guideline questions and promotes transparency in judgments. (c) 2024 The Authors. Published by Elsevier Inc. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
Filiaciones:
Morgano GP:
European Commission, Joint Research Centre (JRC), Ispra, Italy
Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada
Wiercioch W:
Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada
Michael G. DeGroote Cochrane Canada & McMaster GRADE Centres, McMaster University, Hamilton, Ontario, Canada
Piovani D:
Clinical Epidemiology and Research Center (CERC), Humanitas University and IRCCS Humanitas Research Hospital, Milan, Italy
Neumann I:
School of Medicine, Universidad San Sebastián, Santiago, Chile
Nieuwlaat R:
Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada
Michael G. DeGroote Cochrane Canada & McMaster GRADE Centres, McMaster University, Hamilton, Ontario, Canada
Piggott T:
Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada
Department of Family Medicine, Queens University, Kingston, Ontario, Canada
Alonso-Coello P:
Institut de Recerca Sant Pau (IR Sant Pau-CIBERESP), Barcelona, Spain
Mbuagbaw L:
Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada
Rigoni M:
Department of Biomedical, Surgical and Dental Sciences, Università Degli Studi Di Milano Statale, Milan, Italy
Bognanni A:
Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada
Michael G. DeGroote Cochrane Canada & McMaster GRADE Centres, McMaster University, Hamilton, Ontario, Canada
Celedon N:
Department of Health Technology Assessments, Ministerio de Salud de Chile, Santiago, Chile
Mustafa RA:
Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada
Department of Internal Medicine, University of Kansas Medical Centre, Kansas City, Kansas, USA
Pottie K:
Department of Family Medicine, Epidemiology and Biostatistics, Western University, London, Ontario, Canada
Leontiadis GI:
Department of Medicine, McMaster University, Hamilton, Ontario, Canada
Akl EA:
Department of Internal Medicine, American University of Beirut, Beirut, Lebanon
Bonovas S:
Clinical Epidemiology and Research Center (CERC), Humanitas University and IRCCS Humanitas Research Hospital, Milan, Italy
Schünemann HJ:
Michael G. DeGroote Cochrane Canada & McMaster GRADE Centres, McMaster University, Hamilton, Ontario, Canada
Clinical Epidemiology and Research Center (CERC), Humanitas University and IRCCS Humanitas Research Hospital, Milan, Italy
Green Submitted, hybrid
|